box/docs/BOX-API-READ-HTTPS.md

# Box Read-Only Lookups over HTTPS (Agent Access, No SSH)

> **Box is the main surface.** All operator work goes through Box (box.muse-dev.online). The web UI, `box` CLI, and agents share the same API endpoints. No UI-only powers.

**Date:** 2026-10-06
**Status:** bl side implemented; VM board REST below is specified, not yet implemented
**Scope:** read-only lookups only (fleet, threads, unread, dm log). Mutations stay on existing paths.

## 1. Problem

Agents in containers reach box over a 2-hop SSH chain (container → VM → bl).
SSH toggles lapse and every agent needs the full chain configured. Agents need
the daily read lookups — "latest from each agent" — over HTTPS with no secrets
on the wire.

## 2. What exists now (bl side, implemented)

Two agent HTTPS paths already serve reads; both use the same signature auth
(`ssh-keygen -Y sign`, namespace per server, ±300s clock skew, nonce replay
cache) and per-agent principals from `dm-signers/allowed_signers`:

| Path | Server | Client | Auth namespace |
|---|---|---|---|
| Named ops (works today) | `bin/exec-constrained.py` via `https://exec.muse-dev.online/exec` | `bin/exec-sign.sh <op> '<args>'` or `bin/box-relay.sh` (served at `GET /box`) | `exec-constrained` |
| Typed REST (specified below) | VM board `/srv/board/server.py` | any HTTPS client | `box-api` |

New named ops (this change, all `side_effecting: false`, all in `DEFAULT_PERMS`
so any valid fleet signer may call them):

- `fleet.unread` `{"agent"?}` → `box-ctl.py unread [--agent X]`
- `dm.log` `{"limit"?, "agent"?}` (limit 1..100, default 20) → `box-ctl.py dm-log [limit] [--agent X]`

Already present and unchanged: `health.check` (fleet status), `thread.list`,
`thread.view`, `dm.read`, `chat.messages`.

New `box-relay.sh` client commands (this change):

```bash
box unread [<agent>]                 # fleet unread/activity counts
box dm log [<limit=20>] [--agent <agent>]
```

New `box-ctl.py` backend verbs (this change; also callable over the board's
existing SSH bridge until the board speaks REST):

```bash
box-ctl.py unread [--agent <agent>]
box-ctl.py dm-log [limit] [--agent <agent>]   # back-compat: bare [limit] unchanged
```

Also fixed: `box lookup unread` / `muse unread` previously always failed with
"Unknown lookup target 'unread'" (`_lookup_unreads` was never wired into
`cmd_lookup`); it now works and supports `--json`.

## 3. VM board REST (to implement on the VM)

Base: `https://box.muse-dev.online/api/box`. All endpoints require
agent-signature auth (§4) or the existing `ops_session` cookie (humans).

```text
GET /api/box/fleet            exists today; keep behavior
GET /api/box/threads?agent=X  backend: box-ctl.py thread-list --agent X (pass JSON through)
GET /api/box/unread?agent=X   backend: box-ctl.py unread [--agent X]
GET /api/box/dm/log?limit=N&agent=X
                              backend: box-ctl.py dm-log [N] [--agent X]
```

### Agent scoping (server-enforced)

- Verified identity `operator-X` or `X` (X in `muse,pip,646,opm,def,dev`)
  may only read slices for X. The board MUST pass `--agent X` to box-ctl and
  MUST NOT accept a different `agent=` query value from that identity.
- `ops_session` (human PIN login) may omit `agent=` and read the full fleet.
- Unknown/expired signatures → `401`. Authenticated but out-of-scope → `403`.

### Response schemas (bl verbs pass through unchanged)

`GET /api/box/unread`:

```json
{"ok": true, "nodes": [
  {"node": "muse", "unread": 2, "approval_pending": false,
   "title": "muse (2)", "thread": "abc123-uuid-or-null"}
]}
```

`GET /api/box/dm/log` (agent filter matches entries from OR to the agent):

```json
{"ok": true,
 "entries": [{"type": "sent", "id": "bdf7beb6", "agent": "opm",
              "to": "pip", "target": "pip tasks",
              "ts": "2026-10-06T05:56:01.328629+00:00"}],
 "dms": ["... same array, legacy key ..."]}
```

`GET /api/box/fleet`: existing `{"ok": true, "fleet": [...]}` shape, unchanged.

Errors follow `docs/BOX-API-DESIGN-DMS.md` §3.1 (`{"ok": false, "code", "error"}`).

## 4. Agent-signature auth for the REST endpoints

Same identity primitive as signed DMs and `exec-constrained.py`; a signature
is not a secret, so agents can sign without handling credentials.

1. Client builds the canonical string (LF-separated, no trailing newline):

   ```text
   {METHOD}\n{PATH}\n{SORTED_QUERY}\n{TS}\n{NONCE}
   ```

   - `METHOD`: `GET`; `PATH`: e.g. `/api/box/dm/log`; `SORTED_QUERY`: raw
     query string sorted by key (`agent=opm&limit=5`), empty string when none.
   - `TS`: unix epoch seconds; `NONCE`: 16–128 hex chars, single use.
2. Client signs it: `ssh-keygen -Y sign -f <key> -n box-api`.
3. Client sends headers (armor is base64-encoded to stay header-safe):

   ```text
   X-Box-Identity: operator-646
   X-Box-Timestamp: 1728...
   X-Box-Nonce: <hex>
   X-Box-Signature: <base64 of the -----BEGIN SSH SIGNATURE----- armor>
   ```

4. Server recomputes the canonical string from the received request, base64-
   decodes the signature, and runs `ssh-keygen -Y verify -f allowed_signers
   -I <identity> -n box-api -s <sigfile>` with the canonical string on stdin.
   Accept only if: verify exit 0, `|now-TS| ≤ 300`, nonce unseen (cache ≥600s).
   Signers file is synced from bl `dm-signers/allowed_signers`.

Example:

```bash
TS=$(date +%s); NONCE=$(python3 -c "import secrets; print(secrets.token_hex(16))")
CANON=$(printf 'GET\n/api/box/dm/log\nagent=opm&limit=5\n%s\n%s' "$TS" "$NONCE")
SIG=$(printf '%s' "$CANON" | ssh-keygen -Y sign -f ~/.ssh/id_frontdoor -n box-api \
  | base64 -w0)
curl -s 'https://box.muse-dev.online/api/box/dm/log?agent=opm&limit=5' \
  -H "X-Box-Identity: operator-646" -H "X-Box-Timestamp: $TS" \
  -H "X-Box-Nonce: $NONCE" -H "X-Box-Signature: $SIG"
```

## 5. Rollout notes

- `exec-constrained.py` reads `OPS` at startup: restart the service after
  deploying for `fleet.unread` / `dm.log` to appear in `GET /ops`.
- `box-relay.sh` is served from bl (`GET /box`); agents re-fetch to get
  `unread` / `dm log`.
- Until the VM board implements §3, agents use the named-ops path (§2),
  which needs no SSH today.
- Non-goals: write endpoints (`dm.send` etc. stay on the ops path for now),
  PIN/human flows (unchanged), secret handling (no secrets cross the wire).