Box API design: timer and job endpoints\n\n- 14 endpoints with full schemas, JSON canonical (not YAML)\n- Allowlisted box-ctl.py helper (no raw systemctl over SSH)\n- Auto-confirmed vs agent-confirmed split for [REQ]/[CONFIRM]
This commit is contained in:
operator-main
@@ -0,0 +1,848 @@
|
||||
# BOX-API-DESIGN-TIMERS.md: Timer & Job Endpoint Design
|
||||
|
||||
**Status:** Design only. No implementation.
|
||||
**Owner:** operator-main. Sanctioned direction from human 2026-10-04.
|
||||
**Supersedes:** the endpoint sketches in `BOX-API-SPEC.md` §§ "Endpoints"
|
||||
(which are kept as background; this doc is the buildable contract).
|
||||
|
||||
## 1. Scope
|
||||
|
||||
Design the request/response schemas and security model for the
|
||||
agentic timer/job management plane on `box.muse-dev.online`:
|
||||
|
||||
- `GET/POST/DELETE /api/box/timers` — systemd timer lifecycle
|
||||
- `GET/POST/PUT/DELETE /api/box/jobs` — job definition lifecycle
|
||||
- `POST /api/box/jobs/{name}/trigger` — manual job dispatch
|
||||
- `GET/POST /api/box/requests` — the `[REQ]`/`[CONFIRM]` flow
|
||||
|
||||
Out of scope: `/api/box/dms`, `/api/box/board/*`, `/api/box/chat/*`,
|
||||
`/api/box/message` (covered by BOX-API-SPEC.md appendices B/C; they
|
||||
reuse the auth and audit patterns defined here).
|
||||
|
||||
## 2. Ground truths (read before implementing)
|
||||
|
||||
Reality on 2026-10-04, verified live. The design conforms to these;
|
||||
where BOX-API-SPEC.md v0.1 disagrees, this doc wins.
|
||||
|
||||
1. **Job definitions are JSON, not YAML.** bl has Python 3.14.5 and no
|
||||
PyYAML. Canonical path: `/home/super/Projects/NetVM/jobs/<name>.json`.
|
||||
The API accepts and returns `application/json` only. (A YAML
|
||||
frontend may be added later with a vendored parser; it is not
|
||||
part of v1.)
|
||||
2. **The dispatcher is `bin/job-dispatch.py <job_name> [--dry-run]`.**
|
||||
It renders the prompt template, handles sidechat create/reuse
|
||||
(via `job-sidechats.json`: `reuse_key → thread UUID`), sends via
|
||||
`dm.py` / direct sidechat send, and logs to `job-log.jsonl`.
|
||||
3. **Timers are systemd user units** at
|
||||
`~/.config/systemd/user/job-<name>.service` and
|
||||
`~/.config/systemd/user/job-<name>.timer`, owned by `super` on bl.
|
||||
The service `ExecStart` calls `job-dispatch.py <name>` — nothing else.
|
||||
4. **The API server is `/srv/board/server.py` on the VM** (port 8090,
|
||||
behind Caddy; `box.muse-dev.online` routes to its box host branch).
|
||||
New endpoints plug into the existing `_box_api` dispatch, next to
|
||||
`/api/box/nodes`, `/api/box/chats`, etc.
|
||||
5. **Auth already exists:** `_box_tier()` returns
|
||||
`("operator", actor)` for PIN-session or operator bearer token, or
|
||||
`("agent", ident)` for a valid box-signature
|
||||
(`identity`+`ts`+`sig` query params, skew-checked). Anything else
|
||||
gets 403. Audit helper `_box_audit(event, actor, tier, detail)`
|
||||
appends to `/srv/box/audit.jsonl`
|
||||
(`{ts, event, actor, tier, detail}`).
|
||||
6. **VM → bl path exists:** `super@100.123.153.75` over tailnet,
|
||||
passwordless SSH from the VM. This is the only channel the API
|
||||
server uses to touch bl.
|
||||
|
||||
## 3. Authorization model
|
||||
|
||||
Timer management is **privileged**: creating a timer schedules future
|
||||
code execution on shared infrastructure. The model follows the fleet
|
||||
trust tiers (operator ≥ agent-identity > dev > anonymous).
|
||||
|
||||
### 3.1 Matrix
|
||||
|
||||
| Endpoint | super (PIN) | operator (bearer) | agent (box-sig, allowlisted identity) | dev tier | anonymous |
|
||||
|---|---|---|---|---|---|
|
||||
| `GET /timers`, `GET /timers/{name}` | ✅ | ✅ | ✅ | ❌ | 403 |
|
||||
| `POST /timers` (create) | ✅ | ✅ | ✅ * | ❌ | 403 |
|
||||
| `DELETE /timers/{name}` | ✅ | ✅ | ✅ * | ❌ | 403 |
|
||||
| `POST /timers/{name}/{start,stop,enable,disable}` | ✅ | ✅ | ✅ * | ❌ | 403 |
|
||||
| `GET /jobs`, `GET /jobs/{name}` | ✅ | ✅ | ✅ | ❌ | 403 |
|
||||
| `POST /jobs`, `PUT /jobs/{name}` | ✅ | ✅ | ✅ * | ❌ | 403 |
|
||||
| `DELETE /jobs/{name}` | ✅ | ✅ | ✅ * | ❌ | 403 |
|
||||
| `POST /jobs/{name}/trigger` | ✅ | ✅ | ✅ * | ❌ | 403 |
|
||||
| `GET /requests`, `GET /requests/{id}` | ✅ | ✅ | ✅ (own only) | ❌ | 403 |
|
||||
| `POST /requests` | ✅ | ✅ | ✅ * | ❌ | 403 |
|
||||
|
||||
`*` Agent box-signatures are accepted **only if the identity is in the
|
||||
operator allowlist** (`/srv/board/allowed_signers` entries flagged
|
||||
`box_mutate`). A box-signature from a non-operator agent (e.g. a dev
|
||||
agent) is treated as read-capable only and gets 403 on mutations.
|
||||
Rationale: box-signatures prove *which* agent, not *what tier* — the
|
||||
allowlist binds identity to privilege.
|
||||
|
||||
### 3.2 Why devs get nothing here
|
||||
|
||||
Devs execute jobs; they do not schedule them (JOB-SPEC.md § Security:
|
||||
"Agents cannot create jobs"). A dev that could create timers could
|
||||
schedule DM spam or exfiltrate via prompt templates. Read access is
|
||||
also withheld: timer definitions reveal fleet operational tempo.
|
||||
The human may grant exceptions via the duties system; the API checks
|
||||
the grant at request time, it is not hardcoded.
|
||||
|
||||
### 3.3 What "operator" means for agents
|
||||
|
||||
`operator-main`, `operator-646`, and the node agents (`muse`, `pip`,
|
||||
`646`, `opm`) authenticate with box-signatures. Their tier for box
|
||||
mutations comes from the allowlist, not the signature itself. The
|
||||
human (super) uses the PIN session. There is no separate "admin"
|
||||
tier — super is the trust root and can already do everything via
|
||||
other channels.
|
||||
|
||||
## 4. The constrained bl helper: `bin/box-ctl.py`
|
||||
|
||||
**The board server never runs raw `systemctl` or shell over SSH.**
|
||||
All bl mutations go through one allowlisted helper:
|
||||
|
||||
```
|
||||
/home/super/Projects/NetVM/bin/box-ctl.py <action> [args...]
|
||||
```
|
||||
|
||||
Invoked from the VM as:
|
||||
|
||||
```
|
||||
ssh -o BatchMode=yes super@100.123.153.75 \
|
||||
/home/super/Projects/NetVM/bin/box-ctl.py timer-status heartbeat
|
||||
```
|
||||
|
||||
### 4.1 Why a helper instead of `systemctl` over SSH
|
||||
|
||||
- BOX-API-SPEC.md proposed SSH with a restricted key. A restricted
|
||||
key still permits `systemctl --user <anything>` — including
|
||||
`systemctl --user edit`, `link`, or `set-property` on attacker-named
|
||||
units. An allowlist *inside* the helper is a smaller, auditable
|
||||
surface: the helper accepts a fixed verb set and validates every
|
||||
argument before acting.
|
||||
- The helper also owns the **safe unit-file generation**
|
||||
(template-only, no string interpolation of user content into unit
|
||||
files) and the **cron→OnCalendar conversion**, keeping that logic
|
||||
next to the dispatcher it serves.
|
||||
- Defense in depth: the API server validates first (fast 400s), the
|
||||
helper validates again (it is the trust boundary; the API server
|
||||
is reachable from the internet via Caddy).
|
||||
|
||||
### 4.2 Allowlisted actions
|
||||
|
||||
| Action | Args | Effect |
|
||||
|---|---|---|
|
||||
| `timer-list` | — | JSON list of `job-*.timer` units + state |
|
||||
| `timer-status` | `<name>` | JSON: active/enabled/last/next/result |
|
||||
| `timer-create` | `<name>` | Read `jobs/<name>.json`, write `.service`+`.timer` from template, `daemon-reload`, `enable --now` |
|
||||
| `timer-delete` | `<name> [--keep-job]` | stop, disable, remove unit files, `daemon-reload` |
|
||||
| `timer-start` / `timer-stop` / `timer-enable` / `timer-disable` | `<name>` | the obvious systemctl op on `job-<name>.timer` |
|
||||
| `job-list` | — | JSON list of `jobs/*.json` (parsed summaries) |
|
||||
| `job-get` | `<name>` | print the job JSON |
|
||||
| `job-put` | `<name>` | read job JSON from **stdin**, schema-validate, write file, `git add`+`git commit` |
|
||||
| `job-delete` | `<name>` | `git rm` the job JSON + commit |
|
||||
| `job-trigger` | `<name>` | run `job-dispatch.py <name>` (foreground; caller decides sync/async) |
|
||||
|
||||
The helper prints JSON to stdout (`{"ok": true, ...}` or
|
||||
`{"ok": false, "error": "...", "code": "..."}`) and exits nonzero on
|
||||
failure. It never prints secrets. It never invokes a shell.
|
||||
|
||||
### 4.3 Validation inside the helper (non-bypassable)
|
||||
|
||||
- `<name>` must match `^[a-z0-9-]{1,64}$`. Anything else → error,
|
||||
no action. (This kills path traversal: no `/`, no `..`.)
|
||||
- Unit files are generated from a **fixed template**; the only
|
||||
interpolated value is `<name>` (already regex-validated). The
|
||||
`ExecStart` is always the absolute path to `job-dispatch.py` plus
|
||||
the name. User-controlled strings (prompt templates, schedules)
|
||||
never appear in unit files.
|
||||
- `job-put` validates the full job schema (§6) before writing,
|
||||
including: `agent` ∈ known nodes, `schedule` parses as cron and
|
||||
converts to a valid `OnCalendar` (checked with
|
||||
`systemd-analyze calendar --iterations=2`), `prompt_template`
|
||||
length ≤ 4000 chars, `timeout` 60–3600, `on_failure` ∈
|
||||
`{retry, alert, ignore}`.
|
||||
- `timer-create` refuses if the job JSON is missing or invalid, and
|
||||
refuses if a `job-<name>.timer` already exists (use update flow).
|
||||
- Every action appends to the bl-side log
|
||||
(`/home/super/Projects/NetVM/box-ctl.jsonl`):
|
||||
`{ts, action, name, caller}` — `caller` is passed by the API
|
||||
server as `BOX_CALLER` env (the authenticated actor).
|
||||
|
||||
### 4.4 SSH hardening (recommended, not load-bearing)
|
||||
|
||||
- The VM→bl SSH uses `BatchMode=yes`, no agent forwarding, no X11.
|
||||
- Optional hardening: a dedicated `authorized_keys` entry on bl
|
||||
with `command="/home/super/Projects/NetVM/bin/box-ctl.py"`
|
||||
for a `box-api` SSH key held on the VM, so even a compromised
|
||||
board server process can only invoke the helper. The helper's
|
||||
own validation remains the real boundary.
|
||||
|
||||
## 5. Endpoint designs
|
||||
|
||||
Base: `https://box.muse-dev.online/api/box`.
|
||||
Auth: §3. All responses are JSON. Mutations are audit-logged
|
||||
(`_box_audit("box.timer.create", actor, tier, {...})` etc.).
|
||||
|
||||
Conventions used below:
|
||||
|
||||
- `{name}` is the **job name** (`heartbeat`), not the unit name.
|
||||
The API maps `name` → `job-<name>.timer` internally. Clients
|
||||
never construct unit names.
|
||||
- `dry_run=true` query param: validate everything, report the
|
||||
planned actions, change nothing. Returns `200` with
|
||||
`"dry_run": true` and a `plan` array.
|
||||
- `Idempotency-Key` header on POST: the server stores
|
||||
`{key → response}` for 24h in `/srv/box/idempotency.jsonl`;
|
||||
replays return the original response with `200` (not 201).
|
||||
- Timestamps are UTC ISO-8601 (`2026-10-04T03:40:00Z`).
|
||||
|
||||
### 5.1 Timers
|
||||
|
||||
#### `GET /api/box/timers`
|
||||
|
||||
List all job timers on bl.
|
||||
|
||||
**Response `200`:**
|
||||
```json
|
||||
{
|
||||
"timers": [
|
||||
{
|
||||
"name": "heartbeat",
|
||||
"unit": "job-heartbeat.timer",
|
||||
"active": true,
|
||||
"enabled": true,
|
||||
"last_run": "2026-10-04T03:40:00Z",
|
||||
"next_run": "2026-10-04T03:45:00Z",
|
||||
"last_result": "success",
|
||||
"job_name": "heartbeat",
|
||||
"schedule": "*/5 * * * *"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- `last_result`: `success` | `failed` | `unknown`, derived from the
|
||||
most recent `job_sent`/`job_result`/`job_timeout` entries for the
|
||||
job in `job-log.jsonl`.
|
||||
- Timers with no corresponding job JSON are listed with
|
||||
`"orphan": true` (unit exists, definition missing) — never silently
|
||||
dropped.
|
||||
|
||||
**Errors:** `502` (`BL_UNREACHABLE`) if bl SSH fails.
|
||||
|
||||
#### `GET /api/box/timers/{name}`
|
||||
|
||||
Full status for one timer, including the parsed job definition.
|
||||
|
||||
**Response `200`:**
|
||||
```json
|
||||
{
|
||||
"name": "heartbeat",
|
||||
"unit": "job-heartbeat.timer",
|
||||
"active": true,
|
||||
"enabled": true,
|
||||
"last_run": "2026-10-04T03:40:00Z",
|
||||
"next_run": "2026-10-04T03:45:00Z",
|
||||
"last_result": "success",
|
||||
"oncalendar": "*:0/5",
|
||||
"job_definition": {
|
||||
"name": "heartbeat",
|
||||
"agent": "opm",
|
||||
"schedule": "*/5 * * * *",
|
||||
"timeout": 300
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Errors:** `404` (`NOT_FOUND`) — no such timer (or no such job).
|
||||
|
||||
#### `POST /api/box/timers`
|
||||
|
||||
Create a job **and** its timer atomically. Body is the job
|
||||
definition JSON (§6). This is the primary creation path; it keeps
|
||||
"job exists but timer missing" states unrepresentable.
|
||||
|
||||
**Request:**
|
||||
```http
|
||||
POST /api/box/timers
|
||||
Content-Type: application/json
|
||||
Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000
|
||||
|
||||
{
|
||||
"name": "board-watch",
|
||||
"description": "Check board for new posts every 5 minutes",
|
||||
"schedule": "*/5 * * * *",
|
||||
"agent": "muse",
|
||||
"prompt_template": "Check the board for posts since {last_run}. ...",
|
||||
"timeout": 300,
|
||||
"on_failure": "alert",
|
||||
"chain_next": null,
|
||||
"sidechat": { "create": true, "name_template": "job-{job_name}-{date}", "reuse_key": "board-watch-muse" }
|
||||
}
|
||||
```
|
||||
|
||||
**Backend sequence** (each step validated before the next):
|
||||
1. API server validates JSON schema + authz (§3). `400` on failure.
|
||||
2. `box-ctl.py job-put <name>` (stdin = body) → writes + validates
|
||||
on bl, git-commits. Failure → `400`/`502`, nothing else happens.
|
||||
3. `box-ctl.py timer-create <name>` → writes units from template,
|
||||
`daemon-reload`, `enable --now`. Failure → job JSON is kept
|
||||
(it is valid and committed) and the response is `502` with
|
||||
`"timer_created": false` so the client can retry creation
|
||||
without resubmitting the job.
|
||||
4. Audit log entry. Optional `[REQ]` confirmation DM (§8).
|
||||
|
||||
**Response `201`:**
|
||||
```json
|
||||
{
|
||||
"name": "board-watch",
|
||||
"unit": "job-board-watch.timer",
|
||||
"created": true,
|
||||
"timer_created": true,
|
||||
"active": true,
|
||||
"next_run": "2026-10-04T04:00:00Z",
|
||||
"request_id": "a1b2c3d4"
|
||||
}
|
||||
```
|
||||
|
||||
**Errors:**
|
||||
- `400` (`INVALID_JOB`) — schema validation failed; `detail` names
|
||||
the offending field and reason.
|
||||
- `409` (`ALREADY_EXISTS`) — job name or timer already exists.
|
||||
The client should `PUT /jobs/{name}` or `DELETE` first.
|
||||
- `502` (`BL_UNREACHABLE`, `TIMER_CREATE_FAILED`).
|
||||
|
||||
#### `DELETE /api/box/timers/{name}`
|
||||
|
||||
Stop, disable, and remove the timer. The job definition is kept by
|
||||
default (timers are runtime state; definitions are config).
|
||||
|
||||
**Query params:** `keep_job=true|false` (default `true`).
|
||||
|
||||
**Backend:** `box-ctl.py timer-delete <name> [--keep-job]`.
|
||||
|
||||
**Response `200`:**
|
||||
```json
|
||||
{
|
||||
"name": "heartbeat",
|
||||
"deleted": true,
|
||||
"job_kept": true
|
||||
}
|
||||
```
|
||||
|
||||
**Errors:** `404` (`NOT_FOUND`), `502`.
|
||||
|
||||
#### `POST /api/box/timers/{name}/start` (also `/stop`, `/enable`, `/disable`)
|
||||
|
||||
Control a timer. One action per endpoint; the action is in the path,
|
||||
not the body, so there is nothing to inject.
|
||||
|
||||
**Response `200`:**
|
||||
```json
|
||||
{
|
||||
"name": "heartbeat",
|
||||
"action": "stop",
|
||||
"success": true,
|
||||
"active": false,
|
||||
"enabled": true
|
||||
}
|
||||
```
|
||||
|
||||
**Errors:** `404`, `502`. `POST .../start` on an already-active
|
||||
timer is `200` with `"success": true` (idempotent).
|
||||
|
||||
### 5.2 Jobs
|
||||
|
||||
Jobs are definitions; timers are runtime. These endpoints manage
|
||||
definitions **without** touching timers (use §5.1 to create both
|
||||
together).
|
||||
|
||||
#### `GET /api/box/jobs`
|
||||
|
||||
**Response `200`:**
|
||||
```json
|
||||
{
|
||||
"jobs": [
|
||||
{
|
||||
"name": "heartbeat",
|
||||
"description": "Heartbeat job - verifies DM system ...",
|
||||
"schedule": "*/5 * * * *",
|
||||
"agent": "opm",
|
||||
"has_timer": true,
|
||||
"timer_active": true,
|
||||
"timeout": 300
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### `GET /api/box/jobs/{name}`
|
||||
|
||||
**Response `200`:** the full job JSON (`{"name": ..., "definition":
|
||||
{...}}` — or the definition object directly; v1 uses
|
||||
`{"name": "<name>", "definition": {<full job json>}}`).
|
||||
|
||||
**Errors:** `404` (`NOT_FOUND`).
|
||||
|
||||
#### `POST /api/box/jobs`
|
||||
|
||||
Create (or replace, with `?replace=true`) a job definition **without**
|
||||
creating a timer. For definitions that will be wired to a timer
|
||||
later, or timerless jobs triggered manually / by chains.
|
||||
|
||||
Same body schema as `POST /timers` (§6). Backend: `box-ctl.py job-put`.
|
||||
|
||||
**Response `201`:** `{"name": "...", "created": true, "has_timer": false}`.
|
||||
`409` without `?replace=true` if the job exists.
|
||||
|
||||
#### `PUT /api/box/jobs/{name}`
|
||||
|
||||
Replace the definition of an existing job. If a timer exists, its
|
||||
schedule is regenerated from the new definition
|
||||
(`timer-create` semantics on the existing unit: rewrite + reload).
|
||||
The `{name}` in the path must equal `body.name` (`400` otherwise).
|
||||
|
||||
**Response `200`:**
|
||||
```json
|
||||
{
|
||||
"name": "heartbeat",
|
||||
"updated": true,
|
||||
"timer_reloaded": true
|
||||
}
|
||||
```
|
||||
|
||||
#### `DELETE /api/box/jobs/{name}`
|
||||
|
||||
Delete the definition. Refuses (`409` `TIMER_STILL_ACTIVE`) if a
|
||||
timer exists unless `?force=true`, in which case the timer is
|
||||
deleted first (same backend as `DELETE /timers/{name}?keep_job=false`).
|
||||
|
||||
**Response `200`:** `{"name": "...", "deleted": true, "timer_deleted": false}`.
|
||||
|
||||
#### `POST /api/box/jobs/{name}/trigger`
|
||||
|
||||
Run the dispatcher **now**, outside the schedule. For testing and
|
||||
for chain steps.
|
||||
|
||||
**Backend:** `box-ctl.py job-trigger <name>` runs
|
||||
`job-dispatch.py <name>` on bl. The API server runs it **in the
|
||||
background** and returns immediately — dispatch involves browser
|
||||
automation that can take 30–60s; the HTTP request must not block.
|
||||
|
||||
**Response `202`:**
|
||||
```json
|
||||
{
|
||||
"name": "heartbeat",
|
||||
"job_id": "heartbeat-20261004-034737-63b57a71",
|
||||
"triggered": true,
|
||||
"status_url": "/api/box/requests/<request_id>"
|
||||
}
|
||||
```
|
||||
|
||||
The `job_id` is minted by the API server (same format as the
|
||||
dispatcher: `{name}-{YYYYMMDD-HHMMSS}-{8hex}`) and passed to the
|
||||
dispatcher via a `--job-id` flag (dispatcher change required; until
|
||||
then the response omits `job_id` and only returns `request_id`).
|
||||
Completion is tracked through the confirmation system (§8): the
|
||||
trigger creates a request of type `job_trigger`, and the
|
||||
`[RESULT job_id]` collector marks it confirmed.
|
||||
|
||||
**Errors:** `404` (`NOT_FOUND` — no job definition), `502`.
|
||||
|
||||
### 5.3 Requests (`[REQ]`/`[CONFIRM]`)
|
||||
|
||||
Side-effecting API calls (§5.1–5.2 mutations, `trigger`) create a
|
||||
**request** record. The server notifies the responsible agent with a
|
||||
`[REQ {request_id}]` DM; the agent's `[CONFIRM {request_id}]` reply
|
||||
(or the JOB system's `[RESULT {job_id}]`) closes the loop. This is
|
||||
the generalization of `[JOB]`→`[RESULT]` from BOX-API-SPEC.md.
|
||||
|
||||
#### Request record
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "a1b2c3d4",
|
||||
"type": "timer_create",
|
||||
"status": "pending",
|
||||
"actor": "operator-main",
|
||||
"tier": "agent",
|
||||
"job_name": "board-watch",
|
||||
"created_at": "2026-10-04T04:00:00Z",
|
||||
"confirmed_at": null,
|
||||
"attempts": 1,
|
||||
"timeout_s": 300,
|
||||
"detail": { "unit": "job-board-watch.timer" },
|
||||
"confirm_payload": null
|
||||
}
|
||||
```
|
||||
|
||||
- `type`: `timer_create` | `timer_delete` | `timer_control` |
|
||||
`job_create` | `job_update` | `job_delete` | `job_trigger` |
|
||||
`generic`.
|
||||
- `status`: `pending` → `confirmed` | `failed` | `timeout`.
|
||||
- Store: `/srv/box/requests.jsonl` (append-only log) plus a
|
||||
companion index `/srv/box/requests-index.json`
|
||||
(`{request_id → latest status}`) rebuilt from the log on startup.
|
||||
The log is the source of truth; the index is a cache.
|
||||
|
||||
#### `POST /api/box/requests`
|
||||
|
||||
Create a generic confirmation-tracked request (for flows the typed
|
||||
endpoints don't cover). The server sends the `[REQ]` DM immediately.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"type": "generic",
|
||||
"title": "Rotate webhook secret",
|
||||
"assignee": "opm",
|
||||
"body": "Rotate the X secret and confirm.",
|
||||
"timeout_s": 600
|
||||
}
|
||||
```
|
||||
|
||||
**Response `201`:**
|
||||
```json
|
||||
{
|
||||
"request_id": "f7e8d9c0",
|
||||
"status": "pending",
|
||||
"dm_sent": true,
|
||||
"status_url": "/api/box/requests/f7e8d9c0"
|
||||
}
|
||||
```
|
||||
|
||||
#### `GET /api/box/requests`
|
||||
|
||||
List requests, newest first. Query params:
|
||||
`status=pending|confirmed|failed|timeout`,
|
||||
`actor=<name>`, `type=<type>`, `limit` (default 20, max 100).
|
||||
|
||||
**Response `200`:**
|
||||
```json
|
||||
{
|
||||
"requests": [
|
||||
{
|
||||
"request_id": "a1b2c3d4",
|
||||
"type": "timer_create",
|
||||
"status": "confirmed",
|
||||
"actor": "operator-main",
|
||||
"job_name": "board-watch",
|
||||
"created_at": "2026-10-04T04:00:00Z",
|
||||
"confirmed_at": "2026-10-04T04:00:22Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Agents with box-signatures see only requests where they are the
|
||||
actor or the assignee (privacy: one agent's pending work is not
|
||||
another's business). Operators and super see all.
|
||||
|
||||
#### `GET /api/box/requests/{request_id}`
|
||||
|
||||
**Response `200`:** the full record (§5.3), including
|
||||
`confirm_payload` when confirmed:
|
||||
|
||||
```json
|
||||
{
|
||||
"request_id": "a1b2c3d4",
|
||||
"status": "confirmed",
|
||||
"confirm_payload": {
|
||||
"text": "Timer job-board-watch.timer created and active.",
|
||||
"at": "2026-10-04T04:00:22Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Errors:** `404` (`NOT_FOUND`), `403` (agent requesting
|
||||
another agent's request).
|
||||
|
||||
## 6. Job definition JSON schema (canonical)
|
||||
|
||||
This is the contract for `POST /timers`, `POST /jobs`,
|
||||
`PUT /jobs/{name}`, and what `GET /jobs/{name}` returns.
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "board-watch",
|
||||
"description": "Check board for new posts every 5 minutes",
|
||||
"schedule": "*/5 * * * *",
|
||||
"scheduler": "systemd",
|
||||
"agent": "muse",
|
||||
"prompt_template": "Check the board for posts since {last_run}. ...",
|
||||
"timeout": 300,
|
||||
"on_failure": "alert",
|
||||
"chain_next": null,
|
||||
"sidechat": {
|
||||
"create": true,
|
||||
"name_template": "job-{job_name}-{date}",
|
||||
"reuse_key": "board-watch-muse"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Required | Rules |
|
||||
|---|---|---|
|
||||
| `name` | ✅ | `^[a-z0-9-]{1,64}$`. Immutable after creation. |
|
||||
| `description` | ❌ | ≤ 280 chars. |
|
||||
| `schedule` | ✅ | Cron `M H dom mon dow`. Each field validated (ranges, steps, lists). Must convert to a valid systemd `OnCalendar` (§7). |
|
||||
| `scheduler` | ❌ | Default `"systemd"`. v1 accepts **only** `"systemd"`; `"cron"`/`"at"` return `400` `UNSUPPORTED_SCHEDULER` (appendix A of BOX-API-SPEC.md is v2). |
|
||||
| `agent` | ✅ | ∈ `{"muse","pip","646","opm"}` — must be a node in the machine registry with a healthy chromebox. |
|
||||
| `prompt_template` | ✅ | 1–4000 chars. May use `{job_id}`, `{job_name}`, `{datetime}`, `{date}`, `{last_run}`. Unknown `{placeholders}` → `400`. Must not contain `[REQ`/`[CONFIRM`/`[JOB`/`[RESULT` literals (those are protocol framing; the dispatcher adds them). |
|
||||
| `timeout` | ❌ | Default 300. Integer, 60–3600. |
|
||||
| `on_failure` | ❌ | Default `"alert"`. ∈ `{"retry","alert","ignore"}`. |
|
||||
| `chain_next` | ❌ | Default `null`. If set, must name an existing job (`400` otherwise; prevents dangling chains). |
|
||||
| `sidechat.create` | ❌ | Default `false`. Boolean. |
|
||||
| `sidechat.name_template` | ❌ | Only meaningful if `create` is true. ≤ 120 chars; allowed placeholders `{job_name}`, `{date}`, `{job_id}`. |
|
||||
| `sidechat.reuse_key` | ❌ | `^[a-z0-9-]{1,64}$`. Enables the `job-sidechats.json` UUID mapping so recurring jobs reuse one thread instead of spawning one per run. **Recommended for every recurring job.** |
|
||||
|
||||
Validation happens twice: in the API server (fast, user-friendly
|
||||
400s) and in `box-ctl.py job-put` (authoritative — §4.3).
|
||||
|
||||
## 7. Cron → systemd OnCalendar conversion
|
||||
|
||||
The helper converts the cron `schedule` to `OnCalendar=` when
|
||||
generating the `.timer` unit. v1 supports the patterns jobs
|
||||
actually use; anything else is rejected with a clear error
|
||||
rather than silently mis-scheduled.
|
||||
|
||||
| Cron | OnCalendar |
|
||||
|---|---|
|
||||
| `*/5 * * * *` | `*:0/5` |
|
||||
| `*/15 * * * *` | `*:0/15` |
|
||||
| `0 * * * *` | `hourly` |
|
||||
| `0 */2 * * *` | `*:0/120` (see note) |
|
||||
| `30 2 * * *` | `02:30` |
|
||||
| `0 9 * * 1` | `Mon 09:00` |
|
||||
|
||||
- Conversion is done by a small pure function in `box-ctl.py`
|
||||
(no croniter — not installed on bl, and adding a dependency for
|
||||
this is overkill).
|
||||
- The generated string is validated with
|
||||
`systemd-analyze calendar "<expr>" --iterations=2`; nonzero exit
|
||||
→ `400 INVALID_SCHEDULE`.
|
||||
- Note on `*/2` hours: systemd `OnCalendar` supports `/` repetition
|
||||
on the hour field (`0/2:00`); the converter emits that form.
|
||||
- `Persistent=true` is always set (catch-up after downtime —
|
||||
matches the existing heartbeat timer).
|
||||
- `AccuracySec=1min` default (no need for sub-minute precision;
|
||||
reduces wakeups).
|
||||
|
||||
## 8. Confirmation flow mechanics
|
||||
|
||||
### 8.1 Who sends the `[REQ]` DM
|
||||
|
||||
The API server (VM), via a new box-ctl action or via the existing
|
||||
DM path. Two options; the design picks (a):
|
||||
|
||||
- (a) **The helper sends it.** New action `box-ctl.py notify <agent>
|
||||
<request_id> <type>`: renders
|
||||
`[REQ {request_id}] {title}\n{body}\nReply with [CONFIRM {request_id}] ...`
|
||||
and sends via `dm.py` on bl (it is already there, next to the
|
||||
dispatcher). Keeps DM-sending on bl where the chromeboxes live.
|
||||
- (b) The VM sends via a future `/api/box/dms` endpoint. Deferred.
|
||||
|
||||
The `[REQ]` DM goes to the **assignee**: for typed requests, the
|
||||
agent that owns the job (`job.agent`); for generic requests, the
|
||||
`assignee` field. If the request was made by an agent about its own
|
||||
job, the `[REQ]` still goes out — the confirmation is about the
|
||||
*action completing on bl*, not about the requester's intent.
|
||||
|
||||
### 8.2 Who sends `[CONFIRM]`
|
||||
|
||||
- For `timer_create/delete/control`, `job_create/update/delete`:
|
||||
the confirmation is **server-side and automatic**. The helper
|
||||
already verified the outcome (unit active, file written). The
|
||||
request is marked `confirmed` when the helper reports success —
|
||||
no agent round-trip needed. The `[REQ]` DM is then informational
|
||||
("this happened"), and the request record is the audit trail.
|
||||
Rationale: the agent being asked to confirm is often the same
|
||||
agent whose chromebox just did the work; a human-style
|
||||
ack adds latency without signal. The spec's "DMs must result in
|
||||
confirmations" is satisfied by the verified helper result +
|
||||
audit log.
|
||||
- For `job_trigger` and `POST /requests` (generic): confirmation
|
||||
is **agent-driven**. The assignee replies `[CONFIRM {request_id}]`
|
||||
(or `[RESULT {job_id}]` for triggers, which the result collector
|
||||
translates). The pending-request sweeper (§8.3) handles silence.
|
||||
|
||||
This split keeps the common path fast (verified writes need no
|
||||
chat round-trip) while keeping genuinely asynchronous work
|
||||
(job execution) explicitly confirmed.
|
||||
|
||||
### 8.3 Pending-request sweeper
|
||||
|
||||
A systemd timer on the VM (`box-request-sweeper`, every 60s) scans
|
||||
`requests-index.json` for `pending` records where
|
||||
`now - created_at > timeout_s` (default 300s):
|
||||
|
||||
1. If `attempts < 3`: re-send the `[REQ]` DM (`attempts += 1`,
|
||||
log `request_retry`).
|
||||
2. Else: mark `timeout`, log `request_timeout`, and — if the
|
||||
request type is `job_trigger` — apply the job's `on_failure`
|
||||
policy (alert → DM to `#operators` / the requesting actor).
|
||||
|
||||
The sweeper only touches records it owns (VM-side); it never
|
||||
calls bl.
|
||||
|
||||
### 8.4 Reading confirmations
|
||||
|
||||
The result collector (JOB-SPEC.md §5, on bl) already watches for
|
||||
`[RESULT {job_id}]`. It gains a second pattern: when a
|
||||
`[CONFIRM {request_id}]` DM is seen in `dm-log.jsonl`, it appends
|
||||
to a `confirmations.jsonl` which the VM sweeper (or a
|
||||
`box-ctl.py confirm-pull` action) ships back to `/srv/box/`.
|
||||
Simpler v1 alternative: the collector POSTs directly to the VM's
|
||||
`POST /api/box/requests/{id}/confirm` (box-signed, operator
|
||||
allowlist) — one new endpoint, no file shipping:
|
||||
|
||||
```http
|
||||
POST /api/box/requests/a1b2c3d4/confirm
|
||||
Content-Type: application/json
|
||||
|
||||
{ "ok": true, "text": "Timer job-board-watch.timer created and active." }
|
||||
```
|
||||
|
||||
Response `200`: `{"request_id": "...", "status": "confirmed"}`.
|
||||
`ok: false` marks `failed` with the text as reason. This endpoint
|
||||
is allowlisted to the same identities as mutations (§3.1).
|
||||
|
||||
## 9. Security details
|
||||
|
||||
1. **No arbitrary command execution.** The API server's bl access
|
||||
is exactly: `ssh … box-ctl.py <allowlisted verb> <validated name>`.
|
||||
There is no endpoint that passes a shell string, no `ExecStart`
|
||||
interpolation, no `systemctl` subcommand passthrough.
|
||||
2. **Two-layer validation.** API server validates for UX (400s);
|
||||
`box-ctl.py` re-validates as the trust boundary (§4.3). The
|
||||
helper does not trust the API server.
|
||||
3. **Name regex everywhere.** `^[a-z0-9-]{1,64}$` on job names,
|
||||
reuse keys, and request IDs (server-generated hex for the
|
||||
latter). No path traversal, no unit-name smuggling
|
||||
(`job-../../x` is impossible).
|
||||
4. **Prompt templates are data.** They are stored in JSON and
|
||||
rendered by the dispatcher with a fixed placeholder set.
|
||||
They never reach a shell, a unit file, or SQL.
|
||||
5. **Git auditability preserved.** `job-put`/`job-delete` commit
|
||||
on bl with author `<actor> via box-api`. `git log` on the jobs
|
||||
dir remains the definition history (JOB-SPEC.md § Security).
|
||||
6. **Rate limiting.** 10 mutations/min/agent (token bucket,
|
||||
shared `rate_limiter.py` semantics ported to the API server).
|
||||
Reads: 60/min/agent. `429` with `Retry-After`.
|
||||
7. **Dry-run.** `?dry_run=true` on all mutations: runs validation
|
||||
+ helper in check mode, returns the plan, changes nothing.
|
||||
8. **Idempotency.** `Idempotency-Key` on POST (§5); safe retries
|
||||
for flaky mobile clients.
|
||||
9. **Audit everything.** Every mutation → `/srv/box/audit.jsonl`
|
||||
via `_box_audit` with actor, tier, action, name, request_id,
|
||||
and result. Reads of timer/job data are audit-logged too
|
||||
(existing `_box_api` behavior).
|
||||
10. **Secrets never in responses.** Job definitions contain no
|
||||
credentials by construction (dispatcher handles none).
|
||||
DM wire metadata is returned without signature blocks unless
|
||||
the caller is super/operator (signatures are verification
|
||||
material, not secret, but redaction is cheap).
|
||||
11. **Bl unreachable ≠ silent.** SSH failures return `502
|
||||
BL_UNREACHABLE` with a `detail`; mutations are never
|
||||
half-reported as success.
|
||||
|
||||
## 10. Interaction diagram
|
||||
|
||||
```
|
||||
Agent / human UI
|
||||
│ HTTPS + auth (§3)
|
||||
▼
|
||||
box.muse-dev.online → Caddy → board/server.py::_box_api (VM :8090)
|
||||
│ validate, authz, audit, rate-limit
|
||||
│ SSH: box-ctl.py <verb> <name> (mutations)
|
||||
▼
|
||||
bl: bin/box-ctl.py (allowlist + re-validate)
|
||||
├── jobs/<name>.json (job-put/get/delete, git-committed)
|
||||
├── ~/.config/systemd/user/job-<name>.{service,timer}
|
||||
│ │ daemon-reload / enable / start / ...
|
||||
│ ▼
|
||||
│ systemd user manager
|
||||
│ │ OnCalendar fires
|
||||
│ ▼
|
||||
│ job-dispatch.py <name> → dm.py / sidechat send
|
||||
│ │ [JOB job_id] DM to agent
|
||||
│ ▼
|
||||
│ job-log.jsonl, job-sidechats.json
|
||||
│
|
||||
└── [REQ]/notify path: dm.py → agent chromebox
|
||||
```
|
||||
|
||||
Reads (`GET /timers`, `GET /jobs`) follow the same SSH path with
|
||||
read-only verbs; nothing on bl is modified.
|
||||
|
||||
## 11. Error codes
|
||||
|
||||
| HTTP | `code` | Meaning |
|
||||
|---|---|---|
|
||||
| 400 | `INVALID_JOB` | schema validation failed (`detail.field`, `detail.reason`) |
|
||||
| 400 | `INVALID_SCHEDULE` | cron didn't parse or convert |
|
||||
| 400 | `UNSUPPORTED_SCHEDULER` | `scheduler` ≠ `"systemd"` |
|
||||
| 400 | `NAME_MISMATCH` | path name ≠ body name |
|
||||
| 400 | `BAD_NAME` | name fails `^[a-z0-9-]{1,64}$` |
|
||||
| 403 | `FORBIDDEN` | auth missing/insufficient; agent not in mutate allowlist |
|
||||
| 404 | `NOT_FOUND` | timer / job / request unknown |
|
||||
| 409 | `ALREADY_EXISTS` | job or timer already present |
|
||||
| 409 | `TIMER_STILL_ACTIVE` | job delete while timer exists (use `?force=true`) |
|
||||
| 429 | `RATE_LIMITED` | too many ops (`Retry-After` header) |
|
||||
| 502 | `BL_UNREACHABLE` | SSH to bl failed |
|
||||
| 502 | `TIMER_CREATE_FAILED` | job written but unit creation failed |
|
||||
| 502 | `DISPATCH_FAILED` | trigger ran but dispatcher errored |
|
||||
|
||||
All error bodies: `{"error": "<human text>", "code": "<CODE>", "detail": {...?}}`.
|
||||
|
||||
## 12. What changes in existing components
|
||||
|
||||
1. **`bin/box-ctl.py` (new, on bl).** The allowlisted helper (§4).
|
||||
Lives in the NetVM repo, deployed to bl with everything else.
|
||||
2. **`bin/job-dispatch.py` (small change).** Accept `--job-id` so
|
||||
`POST /jobs/{name}/trigger` can mint the ID up front and return
|
||||
it in the `202` (§5.2). Backward compatible (flag optional).
|
||||
3. **Result collector (small change).** Recognize
|
||||
`[CONFIRM {request_id}]` and forward to
|
||||
`POST /api/box/requests/{id}/confirm` (§8.4).
|
||||
4. **`/srv/board/server.py` (VM).** New `_box_api` branches for
|
||||
`/timers`, `/jobs`, `/requests`; `_box_audit` calls; requests
|
||||
store (`/srv/box/requests.jsonl` + index); idempotency store;
|
||||
the 60s sweeper timer unit on the VM.
|
||||
5. **Box web UI.** Timer/job/request pages call these endpoints
|
||||
(thin client; "UI for Creativity, API for Steering" — every UI
|
||||
action shows its curl equivalent).
|
||||
|
||||
Nothing changes in `dm.py`, `muse-chat-api.py`, or the systemd
|
||||
units of existing jobs.
|
||||
|
||||
## 13. Open questions (for the human, not blockers)
|
||||
|
||||
1. Should `POST /timers` send the informational `[REQ]` DM at all,
|
||||
given §8.2 makes creation confirmations automatic? (Lean: no DM
|
||||
for auto-confirmed actions; the audit log + request record is
|
||||
the trail. The UI/API response already tells the caller.)
|
||||
2. `chain_next` validation requires the target job to exist — should
|
||||
creation order then force topological creation, or should
|
||||
dangling `chain_next` be allowed with a warning? (Lean: `400`,
|
||||
explicit is better than dangling.)
|
||||
3. The `agent` allowlist (`muse/pip/646/opm`) is hardcoded in §6.
|
||||
Should it be read live from the machine registry instead?
|
||||
(Lean: yes at implementation time — one source of truth.)
|
||||
4. v2 schedulers (`cron`, `at`): still wanted, or is systemd
|
||||
sufficient for the fleet's needs?
|
||||
|
||||
## 14. Relationship to prior specs
|
||||
|
||||
- `BOX-API-SPEC.md` — background and appendices (DM metadata,
|
||||
board/chat APIs, confirmation philosophy). Its endpoint sketches
|
||||
are superseded by §5 here; its auth, rate-limit, and audit
|
||||
sections are adopted unchanged.
|
||||
- `JOB-SPEC.md` — the job YAML/JSON format, dispatcher, collector.
|
||||
This doc defines the API that manages those artifacts; the job
|
||||
schema in §6 is the JSON rendering of JOB-SPEC.md's YAML.
|
||||
- `DM-SPEC.md` / `DM_SPEC.md` — `[from:X] [id:Y]` attribution and
|
||||
signing. `[REQ]`/`[CONFIRM]` DMs reuse that envelope.
|
||||
Reference in New Issue
Block a user