box/docs/BOX-API-READ-HTTPS.md

Box Read-Only Lookups over HTTPS (Agent Access, No SSH)

Box is the main surface. All operator work goes through Box (box.muse-dev.online). The web UI, box CLI, and agents share the same API endpoints. No UI-only powers.

Date: 2026-10-06 Status: bl side implemented; VM board REST below is specified, not yet implemented Scope: read-only lookups only (fleet, threads, unread, dm log). Mutations stay on existing paths.

1. Problem

Agents in containers reach box over a 2-hop SSH chain (container → VM → bl). SSH toggles lapse and every agent needs the full chain configured. Agents need the daily read lookups — "latest from each agent" — over HTTPS with no secrets on the wire.

2. What exists now (bl side, implemented)

Two agent HTTPS paths already serve reads; both use the same signature auth (ssh-keygen -Y sign, namespace per server, ±300s clock skew, nonce replay cache) and per-agent principals from dm-signers/allowed_signers:

Path	Server	Client	Auth namespace
Named ops (works today)	bin/exec-constrained.py via https://exec.muse-dev.online/exec	bin/exec-sign.sh <op> '<args>' or bin/box-relay.sh (served at GET /box)	exec-constrained
Typed REST (specified below)	VM board /srv/board/server.py	any HTTPS client	box-api

New named ops (this change, all side_effecting: false, all in DEFAULT_PERMS so any valid fleet signer may call them):

• fleet.unread {"agent"?} → box-ctl.py unread [--agent X]
• dm.log {"limit"?, "agent"?} (limit 1..100, default 20) → box-ctl.py dm-log [limit] [--agent X]

Already present and unchanged: health.check (fleet status), thread.list, thread.view, dm.read, chat.messages.

New box-relay.sh client commands (this change):

box unread [<agent>]                 # fleet unread/activity counts
box dm log [<limit=20>] [--agent <agent>]

New box-ctl.py backend verbs (this change; also callable over the board's existing SSH bridge until the board speaks REST):

box-ctl.py unread [--agent <agent>]
box-ctl.py dm-log [limit] [--agent <agent>]   # back-compat: bare [limit] unchanged

Also fixed: box lookup unread / muse unread previously always failed with "Unknown lookup target 'unread'" (_lookup_unreads was never wired into cmd_lookup); it now works and supports --json.

3. VM board REST (to implement on the VM)

Base: https://box.muse-dev.online/api/box. All endpoints require agent-signature auth (§4) or the existing ops_session cookie (humans).

GET /api/box/fleet            exists today; keep behavior
GET /api/box/threads?agent=X  backend: box-ctl.py thread-list --agent X (pass JSON through)
GET /api/box/unread?agent=X   backend: box-ctl.py unread [--agent X]
GET /api/box/dm/log?limit=N&agent=X
                              backend: box-ctl.py dm-log [N] [--agent X]

Agent scoping (server-enforced)

• Verified identity operator-X or X (X in muse,pip,646,opm,def,dev) may only read slices for X. The board MUST pass --agent X to box-ctl and MUST NOT accept a different agent= query value from that identity.
• ops_session (human PIN login) may omit agent= and read the full fleet.
• Unknown/expired signatures → 401. Authenticated but out-of-scope → 403.

Response schemas (bl verbs pass through unchanged)

GET /api/box/unread:

{"ok": true, "nodes": [
  {"node": "muse", "unread": 2, "approval_pending": false,
   "title": "muse (2)", "thread": "abc123-uuid-or-null"}
]}

GET /api/box/dm/log (agent filter matches entries from OR to the agent):

{"ok": true,
 "entries": [{"type": "sent", "id": "bdf7beb6", "agent": "opm",
              "to": "pip", "target": "pip tasks",
              "ts": "2026-10-06T05:56:01.328629+00:00"}],
 "dms": ["... same array, legacy key ..."]}

GET /api/box/fleet: existing {"ok": true, "fleet": [...]} shape, unchanged.

Errors follow docs/BOX-API-DESIGN-DMS.md §3.1 ({"ok": false, "code", "error"}).

4. Agent-signature auth for the REST endpoints

Same identity primitive as signed DMs and exec-constrained.py; a signature is not a secret, so agents can sign without handling credentials.

1. Client builds the canonical string (LF-separated, no trailing newline):

   {METHOD}\n{PATH}\n{SORTED_QUERY}\n{TS}\n{NONCE}
   

   • METHOD: GET; PATH: e.g. /api/box/dm/log; SORTED_QUERY: raw query string sorted by key (agent=opm&limit=5), empty string when none.
   • TS: unix epoch seconds; NONCE: 16–128 hex chars, single use.

2. Client signs it: ssh-keygen -Y sign -f <key> -n box-api.

3. Client sends headers (armor is base64-encoded to stay header-safe):

   X-Box-Identity: operator-646
   X-Box-Timestamp: 1728...
   X-Box-Nonce: <hex>
   X-Box-Signature: <base64 of the -----BEGIN SSH SIGNATURE----- armor>
   

4. Server recomputes the canonical string from the received request, base64- decodes the signature, and runs ssh-keygen -Y verify -f allowed_signers -I <identity> -n box-api -s <sigfile> with the canonical string on stdin. Accept only if: verify exit 0, |now-TS| ≤ 300, nonce unseen (cache ≥600s). Signers file is synced from bl dm-signers/allowed_signers.

Example:

TS=$(date +%s); NONCE=$(python3 -c "import secrets; print(secrets.token_hex(16))")
CANON=$(printf 'GET\n/api/box/dm/log\nagent=opm&limit=5\n%s\n%s' "$TS" "$NONCE")
SIG=$(printf '%s' "$CANON" | ssh-keygen -Y sign -f ~/.ssh/id_frontdoor -n box-api \
  | base64 -w0)
curl -s 'https://box.muse-dev.online/api/box/dm/log?agent=opm&limit=5' \
  -H "X-Box-Identity: operator-646" -H "X-Box-Timestamp: $TS" \
  -H "X-Box-Nonce: $NONCE" -H "X-Box-Signature: $SIG"

5. Rollout notes

• exec-constrained.py reads OPS at startup: restart the service after deploying for fleet.unread / dm.log to appear in GET /ops.
• box-relay.sh is served from bl (GET /box); agents re-fetch to get unread / dm log.
• Until the VM board implements §3, agents use the named-ops path (§2), which needs no SSH today.
• Non-goals: write endpoints (dm.send etc. stay on the ops path for now), PIN/human flows (unchanged), secret handling (no secrets cross the wire).