box/docs/BOX-API-DESIGN-TIMERS.md

# BOX-API-DESIGN-TIMERS.md: Timer & Job Endpoint Design

**Status:** Design only. No implementation.
**Owner:** operator-main. Sanctioned direction from human 2026-10-04.
**Supersedes:** the endpoint sketches in `BOX-API-SPEC.md` §§ "Endpoints"
(which are kept as background; this doc is the buildable contract).

## 1. Scope

Design the request/response schemas and security model for the
agentic timer/job management plane on `box.muse-dev.online`:

- `GET/POST/DELETE /api/box/timers` — systemd timer lifecycle
- `GET/POST/PUT/DELETE /api/box/jobs` — job definition lifecycle
- `POST /api/box/jobs/{name}/trigger` — manual job dispatch
- `GET/POST /api/box/requests` — the `[REQ]`/`[CONFIRM]` flow

Out of scope: `/api/box/dms`, `/api/box/board/*`, `/api/box/chat/*`,
`/api/box/message` (covered by BOX-API-SPEC.md appendices B/C; they
reuse the auth and audit patterns defined here).

## 2. Ground truths (read before implementing)

Reality on 2026-10-04, verified live. The design conforms to these;
where BOX-API-SPEC.md v0.1 disagrees, this doc wins.

1. **Job definitions are JSON, not YAML.** bl has Python 3.14.5 and no
   PyYAML. Canonical path: `/home/super/Projects/NetVM/jobs/<name>.json`.
   The API accepts and returns `application/json` only. (A YAML
   frontend may be added later with a vendored parser; it is not
   part of v1.)
2. **The dispatcher is `bin/job-dispatch.py <job_name> [--dry-run]`.**
   It renders the prompt template, handles sidechat create/reuse
   (via `job-sidechats.json`: `reuse_key → thread UUID`), sends via
   `dm.py` / direct sidechat send, and logs to `job-log.jsonl`.
3. **Timers are systemd user units** at
   `~/.config/systemd/user/job-<name>.service` and
   `~/.config/systemd/user/job-<name>.timer`, owned by `super` on bl.
   The service `ExecStart` calls `job-dispatch.py <name>` — nothing else.
4. **The API server is `/srv/board/server.py` on the VM** (port 8090,
   behind Caddy; `box.muse-dev.online` routes to its box host branch).
   New endpoints plug into the existing `_box_api` dispatch, next to
   `/api/box/nodes`, `/api/box/chats`, etc.
5. **Auth already exists:** `_box_tier()` returns
   `("operator", actor)` for PIN-session or operator bearer token, or
   `("agent", ident)` for a valid box-signature
   (`identity`+`ts`+`sig` query params, skew-checked). Anything else
   gets 403. Audit helper `_box_audit(event, actor, tier, detail)`
   appends to `/srv/box/audit.jsonl`
   (`{ts, event, actor, tier, detail}`).
6. **VM → bl path exists:** `super@100.123.153.75` over tailnet,
   passwordless SSH from the VM. This is the only channel the API
   server uses to touch bl.

## 3. Authorization model

Timer management is **privileged**: creating a timer schedules future
code execution on shared infrastructure. The model follows the fleet
trust tiers (operator ≥ agent-identity > dev > anonymous).

### 3.1 Matrix

| Endpoint | super (PIN) | operator (bearer) | agent (box-sig, allowlisted identity) | dev tier | anonymous |
|---|---|---|---|---|---|
| `GET /timers`, `GET /timers/{name}` | ✅ | ✅ | ✅ | ❌ | 403 |
| `POST /timers` (create) | ✅ | ✅ | ✅ * | ❌ | 403 |
| `DELETE /timers/{name}` | ✅ | ✅ | ✅ * | ❌ | 403 |
| `POST /timers/{name}/{start,stop,enable,disable}` | ✅ | ✅ | ✅ * | ❌ | 403 |
| `GET /jobs`, `GET /jobs/{name}` | ✅ | ✅ | ✅ | ❌ | 403 |
| `POST /jobs`, `PUT /jobs/{name}` | ✅ | ✅ | ✅ * | ❌ | 403 |
| `DELETE /jobs/{name}` | ✅ | ✅ | ✅ * | ❌ | 403 |
| `POST /jobs/{name}/trigger` | ✅ | ✅ | ✅ * | ❌ | 403 |
| `GET /requests`, `GET /requests/{id}` | ✅ | ✅ | ✅ (own only) | ❌ | 403 |
| `POST /requests` | ✅ | ✅ | ✅ * | ❌ | 403 |

`*` Agent box-signatures are accepted **only if the identity is in the
operator allowlist** (`/srv/board/allowed_signers` entries flagged
`box_mutate`). A box-signature from a non-operator agent (e.g. a dev
agent) is treated as read-capable only and gets 403 on mutations.
Rationale: box-signatures prove *which* agent, not *what tier* — the
allowlist binds identity to privilege.

### 3.2 Why devs get nothing here

Devs execute jobs; they do not schedule them (JOB-SPEC.md § Security:
"Agents cannot create jobs"). A dev that could create timers could
schedule DM spam or exfiltrate via prompt templates. Read access is
also withheld: timer definitions reveal fleet operational tempo.
The human may grant exceptions via the duties system; the API checks
the grant at request time, it is not hardcoded.

### 3.3 What "operator" means for agents

`operator-main`, `operator-646`, and the node agents (`muse`, `pip`,
`646`, `opm`) authenticate with box-signatures. Their tier for box
mutations comes from the allowlist, not the signature itself. The
human (super) uses the PIN session. There is no separate "admin"
tier — super is the trust root and can already do everything via
other channels.

## 4. The constrained bl helper: `bin/box-ctl.py`

**The board server never runs raw `systemctl` or shell over SSH.**
All bl mutations go through one allowlisted helper:

```
/home/super/Projects/NetVM/bin/box-ctl.py <action> [args...]
```

Invoked from the VM as:

```
ssh -o BatchMode=yes super@100.123.153.75 \
  /home/super/Projects/NetVM/bin/box-ctl.py timer-status heartbeat
```

### 4.1 Why a helper instead of `systemctl` over SSH

- BOX-API-SPEC.md proposed SSH with a restricted key. A restricted
  key still permits `systemctl --user <anything>` — including
  `systemctl --user edit`, `link`, or `set-property` on attacker-named
  units. An allowlist *inside* the helper is a smaller, auditable
  surface: the helper accepts a fixed verb set and validates every
  argument before acting.
- The helper also owns the **safe unit-file generation**
  (template-only, no string interpolation of user content into unit
  files) and the **cron→OnCalendar conversion**, keeping that logic
  next to the dispatcher it serves.
- Defense in depth: the API server validates first (fast 400s), the
  helper validates again (it is the trust boundary; the API server
  is reachable from the internet via Caddy).

### 4.2 Allowlisted actions

| Action | Args | Effect |
|---|---|---|
| `timer-list` | — | JSON list of `job-*.timer` units + state |
| `timer-status` | `<name>` | JSON: active/enabled/last/next/result |
| `timer-create` | `<name>` | Read `jobs/<name>.json`, write `.service`+`.timer` from template, `daemon-reload`, `enable --now` |
| `timer-delete` | `<name> [--keep-job]` | stop, disable, remove unit files, `daemon-reload` |
| `timer-start` / `timer-stop` / `timer-enable` / `timer-disable` | `<name>` | the obvious systemctl op on `job-<name>.timer` |
| `job-list` | — | JSON list of `jobs/*.json` (parsed summaries) |
| `job-get` | `<name>` | print the job JSON |
| `job-put` | `<name>` | read job JSON from **stdin**, schema-validate, write file, `git add`+`git commit` |
| `job-delete` | `<name>` | `git rm` the job JSON + commit |
| `job-trigger` | `<name>` | run `job-dispatch.py <name>` (foreground; caller decides sync/async) |

The helper prints JSON to stdout (`{"ok": true, ...}` or
`{"ok": false, "error": "...", "code": "..."}`) and exits nonzero on
failure. It never prints secrets. It never invokes a shell.

### 4.3 Validation inside the helper (non-bypassable)

- `<name>` must match `^[a-z0-9-]{1,64}$`. Anything else → error,
  no action. (This kills path traversal: no `/`, no `..`.)
- Unit files are generated from a **fixed template**; the only
  interpolated value is `<name>` (already regex-validated). The
  `ExecStart` is always the absolute path to `job-dispatch.py` plus
  the name. User-controlled strings (prompt templates, schedules)
  never appear in unit files.
- `job-put` validates the full job schema (§6) before writing,
  including: `agent` ∈ known nodes, `schedule` parses as cron and
  converts to a valid `OnCalendar` (checked with
  `systemd-analyze calendar --iterations=2`), `prompt_template`
  length ≤ 4000 chars, `timeout` 60–3600, `on_failure` ∈
  `{retry, alert, ignore}`.
- `timer-create` refuses if the job JSON is missing or invalid, and
  refuses if a `job-<name>.timer` already exists (use update flow).
- Every action appends to the bl-side log
  (`/home/super/Projects/NetVM/box-ctl.jsonl`):
  `{ts, action, name, caller}` — `caller` is passed by the API
  server as `BOX_CALLER` env (the authenticated actor).

### 4.4 SSH hardening (recommended, not load-bearing)

- The VM→bl SSH uses `BatchMode=yes`, no agent forwarding, no X11.
- Optional hardening: a dedicated `authorized_keys` entry on bl
  with `command="/home/super/Projects/NetVM/bin/box-ctl.py"`
  for a `box-api` SSH key held on the VM, so even a compromised
  board server process can only invoke the helper. The helper's
  own validation remains the real boundary.

## 5. Endpoint designs

Base: `https://box.muse-dev.online/api/box`.
Auth: §3. All responses are JSON. Mutations are audit-logged
(`_box_audit("box.timer.create", actor, tier, {...})` etc.).

Conventions used below:

- `{name}` is the **job name** (`heartbeat`), not the unit name.
  The API maps `name` → `job-<name>.timer` internally. Clients
  never construct unit names.
- `dry_run=true` query param: validate everything, report the
  planned actions, change nothing. Returns `200` with
  `"dry_run": true` and a `plan` array.
- `Idempotency-Key` header on POST: the server stores
  `{key → response}` for 24h in `/srv/box/idempotency.jsonl`;
  replays return the original response with `200` (not 201).
- Timestamps are UTC ISO-8601 (`2026-10-04T03:40:00Z`).

### 5.1 Timers

#### `GET /api/box/timers`

List all job timers on bl.

**Response `200`:**
```json
{
  "timers": [
    {
      "name": "heartbeat",
      "unit": "job-heartbeat.timer",
      "active": true,
      "enabled": true,
      "last_run": "2026-10-04T03:40:00Z",
      "next_run": "2026-10-04T03:45:00Z",
      "last_result": "success",
      "job_name": "heartbeat",
      "schedule": "*/5 * * * *"
    }
  ]
}
```

- `last_result`: `success` | `failed` | `unknown`, derived from the
  most recent `job_sent`/`job_result`/`job_timeout` entries for the
  job in `job-log.jsonl`.
- Timers with no corresponding job JSON are listed with
  `"orphan": true` (unit exists, definition missing) — never silently
  dropped.

**Errors:** `502` (`BL_UNREACHABLE`) if bl SSH fails.

#### `GET /api/box/timers/{name}`

Full status for one timer, including the parsed job definition.

**Response `200`:**
```json
{
  "name": "heartbeat",
  "unit": "job-heartbeat.timer",
  "active": true,
  "enabled": true,
  "last_run": "2026-10-04T03:40:00Z",
  "next_run": "2026-10-04T03:45:00Z",
  "last_result": "success",
  "oncalendar": "*:0/5",
  "job_definition": {
    "name": "heartbeat",
    "agent": "opm",
    "schedule": "*/5 * * * *",
    "timeout": 300
  }
}
```

**Errors:** `404` (`NOT_FOUND`) — no such timer (or no such job).

#### `POST /api/box/timers`

Create a job **and** its timer atomically. Body is the job
definition JSON (§6). This is the primary creation path; it keeps
"job exists but timer missing" states unrepresentable.

**Request:**
```http
POST /api/box/timers
Content-Type: application/json
Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000

{
  "name": "board-watch",
  "description": "Check board for new posts every 5 minutes",
  "schedule": "*/5 * * * *",
  "agent": "muse",
  "prompt_template": "Check the board for posts since {last_run}. ...",
  "timeout": 300,
  "on_failure": "alert",
  "chain_next": null,
  "sidechat": { "create": true, "name_template": "job-{job_name}-{date}", "reuse_key": "board-watch-muse" }
}
```

**Backend sequence** (each step validated before the next):
1. API server validates JSON schema + authz (§3). `400` on failure.
2. `box-ctl.py job-put <name>` (stdin = body) → writes + validates
   on bl, git-commits. Failure → `400`/`502`, nothing else happens.
3. `box-ctl.py timer-create <name>` → writes units from template,
   `daemon-reload`, `enable --now`. Failure → job JSON is kept
   (it is valid and committed) and the response is `502` with
   `"timer_created": false` so the client can retry creation
   without resubmitting the job.
4. Audit log entry. Optional `[REQ]` confirmation DM (§8).

**Response `201`:**
```json
{
  "name": "board-watch",
  "unit": "job-board-watch.timer",
  "created": true,
  "timer_created": true,
  "active": true,
  "next_run": "2026-10-04T04:00:00Z",
  "request_id": "a1b2c3d4"
}
```

**Errors:**
- `400` (`INVALID_JOB`) — schema validation failed; `detail` names
  the offending field and reason.
- `409` (`ALREADY_EXISTS`) — job name or timer already exists.
  The client should `PUT /jobs/{name}` or `DELETE` first.
- `502` (`BL_UNREACHABLE`, `TIMER_CREATE_FAILED`).

#### `DELETE /api/box/timers/{name}`

Stop, disable, and remove the timer. The job definition is kept by
default (timers are runtime state; definitions are config).

**Query params:** `keep_job=true|false` (default `true`).

**Backend:** `box-ctl.py timer-delete <name> [--keep-job]`.

**Response `200`:**
```json
{
  "name": "heartbeat",
  "deleted": true,
  "job_kept": true
}
```

**Errors:** `404` (`NOT_FOUND`), `502`.

#### `POST /api/box/timers/{name}/start` (also `/stop`, `/enable`, `/disable`)

Control a timer. One action per endpoint; the action is in the path,
not the body, so there is nothing to inject.

**Response `200`:**
```json
{
  "name": "heartbeat",
  "action": "stop",
  "success": true,
  "active": false,
  "enabled": true
}
```

**Errors:** `404`, `502`. `POST .../start` on an already-active
timer is `200` with `"success": true` (idempotent).

### 5.2 Jobs

Jobs are definitions; timers are runtime. These endpoints manage
definitions **without** touching timers (use §5.1 to create both
together).

#### `GET /api/box/jobs`

**Response `200`:**
```json
{
  "jobs": [
    {
      "name": "heartbeat",
      "description": "Heartbeat job - verifies DM system ...",
      "schedule": "*/5 * * * *",
      "agent": "opm",
      "has_timer": true,
      "timer_active": true,
      "timeout": 300
    }
  ]
}
```

#### `GET /api/box/jobs/{name}`

**Response `200`:** the full job JSON (`{"name": ..., "definition":
{...}}` — or the definition object directly; v1 uses
`{"name": "<name>", "definition": {<full job json>}}`).

**Errors:** `404` (`NOT_FOUND`).

#### `POST /api/box/jobs`

Create (or replace, with `?replace=true`) a job definition **without**
creating a timer. For definitions that will be wired to a timer
later, or timerless jobs triggered manually / by chains.

Same body schema as `POST /timers` (§6). Backend: `box-ctl.py job-put`.

**Response `201`:** `{"name": "...", "created": true, "has_timer": false}`.
`409` without `?replace=true` if the job exists.

#### `PUT /api/box/jobs/{name}`

Replace the definition of an existing job. If a timer exists, its
schedule is regenerated from the new definition
(`timer-create` semantics on the existing unit: rewrite + reload).
The `{name}` in the path must equal `body.name` (`400` otherwise).

**Response `200`:**
```json
{
  "name": "heartbeat",
  "updated": true,
  "timer_reloaded": true
}
```

#### `DELETE /api/box/jobs/{name}`

Delete the definition. Refuses (`409` `TIMER_STILL_ACTIVE`) if a
timer exists unless `?force=true`, in which case the timer is
deleted first (same backend as `DELETE /timers/{name}?keep_job=false`).

**Response `200`:** `{"name": "...", "deleted": true, "timer_deleted": false}`.

#### `POST /api/box/jobs/{name}/trigger`

Run the dispatcher **now**, outside the schedule. For testing and
for chain steps.

**Backend:** `box-ctl.py job-trigger <name>` runs
`job-dispatch.py <name>` on bl. The API server runs it **in the
background** and returns immediately — dispatch involves browser
automation that can take 30–60s; the HTTP request must not block.

**Response `202`:**
```json
{
  "name": "heartbeat",
  "job_id": "heartbeat-20261004-034737-63b57a71",
  "triggered": true,
  "status_url": "/api/box/requests/<request_id>"
}
```

The `job_id` is minted by the API server (same format as the
dispatcher: `{name}-{YYYYMMDD-HHMMSS}-{8hex}`) and passed to the
dispatcher via a `--job-id` flag (dispatcher change required; until
then the response omits `job_id` and only returns `request_id`).
Completion is tracked through the confirmation system (§8): the
trigger creates a request of type `job_trigger`, and the
`[RESULT job_id]` collector marks it confirmed.

**Errors:** `404` (`NOT_FOUND` — no job definition), `502`.

### 5.3 Requests (`[REQ]`/`[CONFIRM]`)

Side-effecting API calls (§5.1–5.2 mutations, `trigger`) create a
**request** record. The server notifies the responsible agent with a
`[REQ {request_id}]` DM; the agent's `[CONFIRM {request_id}]` reply
(or the JOB system's `[RESULT {job_id}]`) closes the loop. This is
the generalization of `[JOB]`→`[RESULT]` from BOX-API-SPEC.md.

#### Request record

```json
{
  "request_id": "a1b2c3d4",
  "type": "timer_create",
  "status": "pending",
  "actor": "operator-main",
  "tier": "agent",
  "job_name": "board-watch",
  "created_at": "2026-10-04T04:00:00Z",
  "confirmed_at": null,
  "attempts": 1,
  "timeout_s": 300,
  "detail": { "unit": "job-board-watch.timer" },
  "confirm_payload": null
}
```

- `type`: `timer_create` | `timer_delete` | `timer_control` |
  `job_create` | `job_update` | `job_delete` | `job_trigger` |
  `generic`.
- `status`: `pending` → `confirmed` | `failed` | `timeout`.
- Store: `/srv/box/requests.jsonl` (append-only log) plus a
  companion index `/srv/box/requests-index.json`
  (`{request_id → latest status}`) rebuilt from the log on startup.
  The log is the source of truth; the index is a cache.

#### `POST /api/box/requests`

Create a generic confirmation-tracked request (for flows the typed
endpoints don't cover). The server sends the `[REQ]` DM immediately.

**Request:**
```json
{
  "type": "generic",
  "title": "Rotate webhook secret",
  "assignee": "opm",
  "body": "Rotate the X secret and confirm.",
  "timeout_s": 600
}
```

**Response `201`:**
```json
{
  "request_id": "f7e8d9c0",
  "status": "pending",
  "dm_sent": true,
  "status_url": "/api/box/requests/f7e8d9c0"
}
```

#### `GET /api/box/requests`

List requests, newest first. Query params:
`status=pending|confirmed|failed|timeout`,
`actor=<name>`, `type=<type>`, `limit` (default 20, max 100).

**Response `200`:**
```json
{
  "requests": [
    {
      "request_id": "a1b2c3d4",
      "type": "timer_create",
      "status": "confirmed",
      "actor": "operator-main",
      "job_name": "board-watch",
      "created_at": "2026-10-04T04:00:00Z",
      "confirmed_at": "2026-10-04T04:00:22Z"
    }
  ]
}
```

Agents with box-signatures see only requests where they are the
actor or the assignee (privacy: one agent's pending work is not
another's business). Operators and super see all.

#### `GET /api/box/requests/{request_id}`

**Response `200`:** the full record (§5.3), including
`confirm_payload` when confirmed:

```json
{
  "request_id": "a1b2c3d4",
  "status": "confirmed",
  "confirm_payload": {
    "text": "Timer job-board-watch.timer created and active.",
    "at": "2026-10-04T04:00:22Z"
  }
}
```

**Errors:** `404` (`NOT_FOUND`), `403` (agent requesting
another agent's request).

## 6. Job definition JSON schema (canonical)

This is the contract for `POST /timers`, `POST /jobs`,
`PUT /jobs/{name}`, and what `GET /jobs/{name}` returns.

```json
{
  "name": "board-watch",
  "description": "Check board for new posts every 5 minutes",
  "schedule": "*/5 * * * *",
  "scheduler": "systemd",
  "agent": "muse",
  "prompt_template": "Check the board for posts since {last_run}. ...",
  "timeout": 300,
  "on_failure": "alert",
  "chain_next": null,
  "sidechat": {
    "create": true,
    "name_template": "job-{job_name}-{date}",
    "reuse_key": "board-watch-muse"
  }
}
```

| Field | Required | Rules |
|---|---|---|
| `name` | ✅ | `^[a-z0-9-]{1,64}$`. Immutable after creation. |
| `description` | ❌ | ≤ 280 chars. |
| `schedule` | ✅ | Cron `M H dom mon dow`. Each field validated (ranges, steps, lists). Must convert to a valid systemd `OnCalendar` (§7). |
| `scheduler` | ❌ | Default `"systemd"`. v1 accepts **only** `"systemd"`; `"cron"`/`"at"` return `400` `UNSUPPORTED_SCHEDULER` (appendix A of BOX-API-SPEC.md is v2). |
| `agent` | ✅ | ∈ `{"muse","pip","646","opm"}` — must be a node in the machine registry with a healthy chromebox. |
| `prompt_template` | ✅ | 1–4000 chars. May use `{job_id}`, `{job_name}`, `{datetime}`, `{date}`, `{last_run}`. Unknown `{placeholders}` → `400`. Must not contain `[REQ`/`[CONFIRM`/`[JOB`/`[RESULT` literals (those are protocol framing; the dispatcher adds them). |
| `timeout` | ❌ | Default 300. Integer, 60–3600. |
| `on_failure` | ❌ | Default `"alert"`. ∈ `{"retry","alert","ignore"}`. |
| `chain_next` | ❌ | Default `null`. If set, must name an existing job (`400` otherwise; prevents dangling chains). |
| `sidechat.create` | ❌ | Default `false`. Boolean. |
| `sidechat.name_template` | ❌ | Only meaningful if `create` is true. ≤ 120 chars; allowed placeholders `{job_name}`, `{date}`, `{job_id}`. |
| `sidechat.reuse_key` | ❌ | `^[a-z0-9-]{1,64}$`. Enables the `job-sidechats.json` UUID mapping so recurring jobs reuse one thread instead of spawning one per run. **Recommended for every recurring job.** |

Validation happens twice: in the API server (fast, user-friendly
400s) and in `box-ctl.py job-put` (authoritative — §4.3).

## 7. Cron → systemd OnCalendar conversion

The helper converts the cron `schedule` to `OnCalendar=` when
generating the `.timer` unit. v1 supports the patterns jobs
actually use; anything else is rejected with a clear error
rather than silently mis-scheduled.

| Cron | OnCalendar |
|---|---|
| `*/5 * * * *` | `*:0/5` |
| `*/15 * * * *` | `*:0/15` |
| `0 * * * *` | `hourly` |
| `0 */2 * * *` | `*:0/120` (see note) |
| `30 2 * * *` | `02:30` |
| `0 9 * * 1` | `Mon 09:00` |

- Conversion is done by a small pure function in `box-ctl.py`
  (no croniter — not installed on bl, and adding a dependency for
  this is overkill).
- The generated string is validated with
  `systemd-analyze calendar "<expr>" --iterations=2`; nonzero exit
  → `400 INVALID_SCHEDULE`.
- Note on `*/2` hours: systemd `OnCalendar` supports `/` repetition
  on the hour field (`0/2:00`); the converter emits that form.
- `Persistent=true` is always set (catch-up after downtime —
  matches the existing heartbeat timer).
- `AccuracySec=1min` default (no need for sub-minute precision;
  reduces wakeups).

## 8. Confirmation flow mechanics

### 8.1 Who sends the `[REQ]` DM

The API server (VM), via a new box-ctl action or via the existing
DM path. Two options; the design picks (a):

- (a) **The helper sends it.** New action `box-ctl.py notify <agent>
  <request_id> <type>`: renders
  `[REQ {request_id}] {title}\n{body}\nReply with [CONFIRM {request_id}] ...`
  and sends via `dm.py` on bl (it is already there, next to the
  dispatcher). Keeps DM-sending on bl where the chromeboxes live.
- (b) The VM sends via a future `/api/box/dms` endpoint. Deferred.

The `[REQ]` DM goes to the **assignee**: for typed requests, the
agent that owns the job (`job.agent`); for generic requests, the
`assignee` field. If the request was made by an agent about its own
job, the `[REQ]` still goes out — the confirmation is about the
*action completing on bl*, not about the requester's intent.

### 8.2 Who sends `[CONFIRM]`

- For `timer_create/delete/control`, `job_create/update/delete`:
  the confirmation is **server-side and automatic**. The helper
  already verified the outcome (unit active, file written). The
  request is marked `confirmed` when the helper reports success —
  no agent round-trip needed. The `[REQ]` DM is then informational
  ("this happened"), and the request record is the audit trail.
  Rationale: the agent being asked to confirm is often the same
  agent whose chromebox just did the work; a human-style
  ack adds latency without signal. The spec's "DMs must result in
  confirmations" is satisfied by the verified helper result +
  audit log.
- For `job_trigger` and `POST /requests` (generic): confirmation
  is **agent-driven**. The assignee replies `[CONFIRM {request_id}]`
  (or `[RESULT {job_id}]` for triggers, which the result collector
  translates). The pending-request sweeper (§8.3) handles silence.

This split keeps the common path fast (verified writes need no
chat round-trip) while keeping genuinely asynchronous work
(job execution) explicitly confirmed.

### 8.3 Pending-request sweeper

A systemd timer on the VM (`box-request-sweeper`, every 60s) scans
`requests-index.json` for `pending` records where
`now - created_at > timeout_s` (default 300s):

1. If `attempts < 3`: re-send the `[REQ]` DM (`attempts += 1`,
   log `request_retry`).
2. Else: mark `timeout`, log `request_timeout`, and — if the
   request type is `job_trigger` — apply the job's `on_failure`
   policy (alert → DM to `#operators` / the requesting actor).

The sweeper only touches records it owns (VM-side); it never
calls bl.

### 8.4 Reading confirmations

The result collector (JOB-SPEC.md §5, on bl) already watches for
`[RESULT {job_id}]`. It gains a second pattern: when a
`[CONFIRM {request_id}]` DM is seen in `dm-log.jsonl`, it appends
to a `confirmations.jsonl` which the VM sweeper (or a
`box-ctl.py confirm-pull` action) ships back to `/srv/box/`.
Simpler v1 alternative: the collector POSTs directly to the VM's
`POST /api/box/requests/{id}/confirm` (box-signed, operator
allowlist) — one new endpoint, no file shipping:

```http
POST /api/box/requests/a1b2c3d4/confirm
Content-Type: application/json

{ "ok": true, "text": "Timer job-board-watch.timer created and active." }
```

Response `200`: `{"request_id": "...", "status": "confirmed"}`.
`ok: false` marks `failed` with the text as reason. This endpoint
is allowlisted to the same identities as mutations (§3.1).

## 9. Security details

1. **No arbitrary command execution.** The API server's bl access
   is exactly: `ssh … box-ctl.py <allowlisted verb> <validated name>`.
   There is no endpoint that passes a shell string, no `ExecStart`
   interpolation, no `systemctl` subcommand passthrough.
2. **Two-layer validation.** API server validates for UX (400s);
   `box-ctl.py` re-validates as the trust boundary (§4.3). The
   helper does not trust the API server.
3. **Name regex everywhere.** `^[a-z0-9-]{1,64}$` on job names,
   reuse keys, and request IDs (server-generated hex for the
   latter). No path traversal, no unit-name smuggling
   (`job-../../x` is impossible).
4. **Prompt templates are data.** They are stored in JSON and
   rendered by the dispatcher with a fixed placeholder set.
   They never reach a shell, a unit file, or SQL.
5. **Git auditability preserved.** `job-put`/`job-delete` commit
   on bl with author `<actor> via box-api`. `git log` on the jobs
   dir remains the definition history (JOB-SPEC.md § Security).
6. **Rate limiting.** 10 mutations/min/agent (token bucket,
   shared `rate_limiter.py` semantics ported to the API server).
   Reads: 60/min/agent. `429` with `Retry-After`.
7. **Dry-run.** `?dry_run=true` on all mutations: runs validation
   + helper in check mode, returns the plan, changes nothing.
8. **Idempotency.** `Idempotency-Key` on POST (§5); safe retries
   for flaky mobile clients.
9. **Audit everything.** Every mutation → `/srv/box/audit.jsonl`
   via `_box_audit` with actor, tier, action, name, request_id,
   and result. Reads of timer/job data are audit-logged too
   (existing `_box_api` behavior).
10. **Secrets never in responses.** Job definitions contain no
    credentials by construction (dispatcher handles none).
    DM wire metadata is returned without signature blocks unless
    the caller is super/operator (signatures are verification
    material, not secret, but redaction is cheap).
11. **Bl unreachable ≠ silent.** SSH failures return `502
    BL_UNREACHABLE` with a `detail`; mutations are never
    half-reported as success.

## 10. Interaction diagram

```
Agent / human UI
      │  HTTPS + auth (§3)
      ▼
box.muse-dev.online → Caddy → board/server.py::_box_api (VM :8090)
      │  validate, authz, audit, rate-limit
      │  SSH: box-ctl.py <verb> <name>        (mutations)
      ▼
bl: bin/box-ctl.py (allowlist + re-validate)
      ├── jobs/<name>.json  (job-put/get/delete, git-committed)
      ├── ~/.config/systemd/user/job-<name>.{service,timer}
      │        │  daemon-reload / enable / start / ...
      │        ▼
      │   systemd user manager
      │        │  OnCalendar fires
      │        ▼
      │   job-dispatch.py <name>  →  dm.py / sidechat send
      │        │  [JOB job_id] DM to agent
      │        ▼
      │   job-log.jsonl, job-sidechats.json
      │
      └── [REQ]/notify path: dm.py → agent chromebox
```

Reads (`GET /timers`, `GET /jobs`) follow the same SSH path with
read-only verbs; nothing on bl is modified.

## 11. Error codes

| HTTP | `code` | Meaning |
|---|---|---|
| 400 | `INVALID_JOB` | schema validation failed (`detail.field`, `detail.reason`) |
| 400 | `INVALID_SCHEDULE` | cron didn't parse or convert |
| 400 | `UNSUPPORTED_SCHEDULER` | `scheduler` ≠ `"systemd"` |
| 400 | `NAME_MISMATCH` | path name ≠ body name |
| 400 | `BAD_NAME` | name fails `^[a-z0-9-]{1,64}$` |
| 403 | `FORBIDDEN` | auth missing/insufficient; agent not in mutate allowlist |
| 404 | `NOT_FOUND` | timer / job / request unknown |
| 409 | `ALREADY_EXISTS` | job or timer already present |
| 409 | `TIMER_STILL_ACTIVE` | job delete while timer exists (use `?force=true`) |
| 429 | `RATE_LIMITED` | too many ops (`Retry-After` header) |
| 502 | `BL_UNREACHABLE` | SSH to bl failed |
| 502 | `TIMER_CREATE_FAILED` | job written but unit creation failed |
| 502 | `DISPATCH_FAILED` | trigger ran but dispatcher errored |

All error bodies: `{"error": "<human text>", "code": "<CODE>", "detail": {...?}}`.

## 12. What changes in existing components

1. **`bin/box-ctl.py` (new, on bl).** The allowlisted helper (§4).
   Lives in the NetVM repo, deployed to bl with everything else.
2. **`bin/job-dispatch.py` (small change).** Accept `--job-id` so
   `POST /jobs/{name}/trigger` can mint the ID up front and return
   it in the `202` (§5.2). Backward compatible (flag optional).
3. **Result collector (small change).** Recognize
   `[CONFIRM {request_id}]` and forward to
   `POST /api/box/requests/{id}/confirm` (§8.4).
4. **`/srv/board/server.py` (VM).** New `_box_api` branches for
   `/timers`, `/jobs`, `/requests`; `_box_audit` calls; requests
   store (`/srv/box/requests.jsonl` + index); idempotency store;
   the 60s sweeper timer unit on the VM.
5. **Box web UI.** Timer/job/request pages call these endpoints
   (thin client; "UI for Creativity, API for Steering" — every UI
   action shows its curl equivalent).

Nothing changes in `dm.py`, `muse-chat-api.py`, or the systemd
units of existing jobs.

## 13. Open questions (for the human, not blockers)

1. Should `POST /timers` send the informational `[REQ]` DM at all,
   given §8.2 makes creation confirmations automatic? (Lean: no DM
   for auto-confirmed actions; the audit log + request record is
   the trail. The UI/API response already tells the caller.)
2. `chain_next` validation requires the target job to exist — should
   creation order then force topological creation, or should
   dangling `chain_next` be allowed with a warning? (Lean: `400`,
   explicit is better than dangling.)
3. The `agent` allowlist (`muse/pip/646/opm`) is hardcoded in §6.
   Should it be read live from the machine registry instead?
   (Lean: yes at implementation time — one source of truth.)
4. v2 schedulers (`cron`, `at`): still wanted, or is systemd
   sufficient for the fleet's needs?

## 14. Relationship to prior specs

- `BOX-API-SPEC.md` — background and appendices (DM metadata,
  board/chat APIs, confirmation philosophy). Its endpoint sketches
  are superseded by §5 here; its auth, rate-limit, and audit
  sections are adopted unchanged.
- `JOB-SPEC.md` — the job YAML/JSON format, dispatcher, collector.
  This doc defines the API that manages those artifacts; the job
  schema in §6 is the JSON rendering of JOB-SPEC.md's YAML.
- `DM-SPEC.md` / `DM_SPEC.md` — `[from:X] [id:Y]` attribution and
  signing. `[REQ]`/`[CONFIRM]` DMs reuse that envelope.