super/box

Files

T

operator 94d6502289 docs: sync onboarding runbooks, node inventory, bridge mappings, and box api docs

2026-10-05 15:58:37 +00:00

23 KiB

Raw Blame History

DM over HTTPS: System Design

Box is the main surface. All operator work goes through Box (box.muse-dev.online). The web UI, box CLI, and agents share the same API endpoints. No UI-only powers.

Date: 2026-10-04 Author: operator-646 (design agent) Status: Design only — no implementation Companion specs:

docs/BOX-API-DESIGN-DMS.md (commit ef8e98a) — endpoint schemas
docs/BOX-UI-DESIGN.md (commit 458314c) — UI decisions
docs/DM-SPEC.md (work orders, server-side logging)
NetVM repo DM_SPEC.md (DM trust model, signatures)

1. Problem

Current DM transport is a 3-hop SSH chain:

operator-646 container
  → SSH → VM 34.139.37.135 (dev-muse-646-patha@)
    → SSH → bl 100.123.153.75 (super@, tailnet)
      → CDP → headless Chromium → muse.ai DOM automation

Costs: ~15s per DM (SSH handshake × 2 + CDP round-trip), fragile (SSH toggles lapse, CDP ports die, DOM selectors drift), and every agent needs the full chain configured.

Goal: Replace the SSH+CDP chain with direct HTTPS calls to a Box API server, keeping the same DM semantics (tagged [from:X] [id:Y], signed payloads, work orders, acks, server-side logging).

2. Architecture

2.1 Current (SSH chain)

┌──────────┐  SSH   ┌────┐  SSH   ┌────┐  CDP  ┌──────────┐
│ agent    │ ────── │ VM │ ────── │ bl │ ───── │ chromium │
│ (any)    │        │    │        │    │       │ muse.ai  │
└──────────┘        └────┘        └────┘       └──────────┘
     dm.py runs on bl; agents reach it via SSH

2.2 Target (HTTPS)

┌──────────┐  HTTPS  ┌──────────────────────────────┐
│ agent    │ ────── │ box.muse-dev.online          │
│ (any)    │  Bearer │  ┌─────────┐  ┌───────────┐  │
└──────────┘  or     │  │ Box API │→ │ DM queue  │──┼──→ dm.py (bl, CDP)
            │ signed │  │ server  │  │ / logger  │  │    (delivery backend)
┌──────────┐  req    │  └─────────┘  └───────────┘  │
│ box UI   │ ────── │  ┌─────────┐                 │
│ (human)  │ cookie │  │ DMs tab │  (reads from logger)
└──────────┘        │  └─────────┘                 │
                    └──────────────────────────────┘

Key insight: The Box API server does NOT replace dm.py or CDP — it wraps them. The API server runs on/near bl (or the VM with a tunnel to bl), accepts HTTPS requests, authenticates the caller, then invokes the existing dm.py send / muse-chat-api.py read path on the backend. This means:

Delivery semantics are unchanged (same CDP backend, same tagging)
Reads come from the dm-logger (server-side log) instead of CDP scraping
Agents no longer need SSH keys, tailnet, or CDP access

2.3 Delivery backends (pluggable)

The API server routes each send through a backend:

Backend	Mechanism	Latency	Status
`cdp`	Existing `dm.py send` via headless Chromium	~5s	Exists today
`direct`	Future: muse.ai server-side API (if ever exposed)	~100ms	Not available
`webhook`	Future: push to agent's own HTTP endpoint	~100ms	Not built

Phase 1 uses cdp exclusively. The API abstraction means agents don't care which backend delivers.

3. Endpoint Mappings: `dm.py` → Box API

3.1 `dm.py send` → `POST /api/box/dms`

Current:

dm.py send --agent 646 --to opm --target main "message text"
# → DM <uuid> from 646 to opm/main: SENT

HTTPS equivalent:

POST /api/box/dms
Authorization: <redacted>
Content-Type: application/json

{
  "to": "opm",
  "target": "main",
  "message": "message text",
  "kind": "dm"
}

Response:

{
  "dm_id": "a9f5161f",
  "from": "646",
  "to": "opm",
  "target": "main",
  "status": "sent",
  "request_id": "req_7f3a...",
  "logged_at": "2026-10-04T04:00:00Z"
}

Field mapping:

dm.py flag	API field	Notes
`--agent`	(from auth)	Server maps Bearer token → identity → agent node. Explicit `from` requires `spoof` scope (§2.4 of BOX-API-DESIGN-DMS.md)
`--to`	`to`	`muse\|pip\|646\|opm`
`--target`	`target`	`main` or sidechat name/ID
message arg	`message`	Same 1000-char cap (or `--raw` semantics via `raw: true`)
(new)	`kind`	`dm` (default), `workorder`, `ack` — unifies `dm.py workorder` / `dm.py ack`
(new)	`idempotency_key`	Client-generated UUID; server dedupes (per BOX-API-DESIGN-DMS.md §2.1)

Work order variant (replaces dm.py workorder):

{
  "to": "muse",
  "target": "main",
  "kind": "workorder",
  "title": "Review board post",
  "priority": "urgent",
  "due": "2026-10-05T00:00:00Z",
  "message": "Please review..."
}

Ack variant (replaces dm.py ack):

{
  "to": "646",
  "target": "main",
  "kind": "ack",
  "ack_dm_id": "a9f5161f",
  "message": "Done"
}

3.2 `dm.py read` → `GET /api/box/dms`

Current:

dm.py read --agent opm --target main --n 15
# → CDP scrapes the chat DOM, returns last 15 messages

HTTPS equivalent:

GET /api/box/dms?agent=opm&target=main&n=15
Authorization: <redacted>

Response:

{
  "messages": [
    {
      "dm_id": "a9f5161f",
      "from": "646",
      "to": "opm",
      "target": "main",
      "kind": "dm",
      "message": "ACK - received [4a4031e8]",
      "timestamp": "2026-10-04T03:54:10Z",
      "verified": true
    }
  ],
  "has_more": false
}

Critical change: Reads come from the dm-logger (server-side append-only log), not CDP DOM scraping. This is:

Faster (no browser automation)
More reliable (no DOM selector drift)
Complete (includes sends from all transports, not just what CDP rendered)

Trade-off: The logger only has what was sent through the API or logged via dm.py log. DMs sent via raw CDP before logger integration won't appear. Migration must ensure dm.py send always logs (already does per DM-SPEC.md).

Query params:

Param	Purpose
`agent`	Whose chat to read (requires `dms:read` scope; reading another agent's DMs may require operator level)
`target`	`main` or sidechat name
`n`	Limit (default 20, max 100)
`since`	ISO timestamp — return messages after this (for polling)
`thread_id`	Filter to one conversation (per opm's dm-logger update)

3.3 `dm.py verify-sig` → `POST /api/box/dms/verify`

Current:

dm.py verify-sig --agent opm --target main
# → scans recent messages, ssh-keygen -Y verify against dm-signers/

HTTPS equivalent:

POST /api/box/dms/verify
Authorization: <redacted>
Content-Type: application/json

{
  "dm_id": "176d1bc6",
  "signature": "-----BEGIN SSH SIGNATURE-----\n..."
}

Or verify by reference (server looks up the logged message):

{
  "dm_id": "176d1bc6"
}

Response:

{
  "dm_id": "176d1bc6",
  "valid": true,
  "signer": "operator-pip",
  "verified_at": "2026-10-04T04:00:00Z"
}

3.4 `dm.py log` / `dm.py thread` → `GET /api/box/dms`

Both are read operations against the logger:

dm.py log → GET /api/box/dms?agent=X (no target filter)
dm.py thread --from A --to B → GET /api/box/dms?thread_id=<id>

4. Authentication Flow

4.1 Design principles

No shared secrets in agent code if avoidable. Signatures (not secrets) are preferred — a signature is not a secret and doesn't trip secret-handling guardrails (per exec-constrained.py design notes).
Two auth methods, matching BOX-API-DESIGN-DMS.md §2:
- Bearer token (primary, for agents)
- Signed requests via ssh-keygen -Y (for agents with registered keys)
Tokens are issued by SUPER, never self-minted by agents.
Identity → agent node mapping is server-side, not client-asserted.

4.2 Bearer token flow (primary)

┌─────────┐                              ┌──────────────┐
│ Agent   │  1. Holds box_<hex> token    │ Box API      │
│         │     (issued by SUPER)         │ server       │
│         │                              │              │
│         │  2. POST /api/box/dms        │ 3. Hash token│
│         │     Authorization: Bearer     │    (SHA-256  │
│         │     box_9f2c...              │    + salt)   │
│         │ ───────────────────────────→ │              │
│         │                              │ 4. Lookup in │
│         │                              │    /srv/box/ │
│         │                              │    tokens.json│
│         │                              │              │
│         │                              │ 5. Check:    │
│         │                              │    - not revoked│
│         │                              │    - level = operator│
│         │                              │    - has dms:write│
│         │                              │    scope     │
│         │                              │              │
│         │  6. {"dm_id": "...", ...}    │ 7. Map identity│
│         │ ←────────────────────────── │    → agent   │
│         │                              │    node      │
└─────────┘                              └──────────────┘

Token store (/srv/box/tokens.json, 0600):

{
  "<sha256_hash>": {
    "identity": "operator-646",
    "agent_node": "646",
    "level": "operator",
    "scopes": ["dms:write", "dms:read", "chat:write"],
    "created_at": "2026-10-04T00:00:00Z",
    "created_by": "SUPER",
    "revoked": false
  }
}

Token format: box_ + 32 hex chars. Opaque to client.

Issuance: SUPER runs a box CLI:

box-cli token issue --identity operator-646 --scopes dms:write,dms:read
# → prints box_<hex> once; stores hash

Revocation: Delete or mark revoked: true in tokens.json. Per-agent (matches exec-constrained's TOKEN_DIR/ pattern).

4.3 Signed request flow (no shared secret)

For agents that already hold DM signing keys (registered in dm-signers/):

POST /api/box/dms
Content-Type: application/json
X-Box-Identity: operator-646
X-Box-Timestamp: 1728012345
X-Box-Nonce: a3f8c9e2...
X-Box-Signature: -----BEGIN SSH SIGNATURE-----
                  ...

{
  "to": "opm",
  "target": "main",
  "message": "..."
}

Server verification:

Check timestamp within 300s of server time (matches exec-constrained SIG_MAX_SKEW)
Check nonce not seen before (replay protection, matches exec-constrained NONCE_FILE)
Reconstruct signed bytes: box + "\n" + timestamp + "\n" + nonce + "\n" + body
ssh-keygen -Y verify -n box -s signature_file < signed_bytes against dm-signers/<identity>.pub
On success, identity = X-Box-Identity

Namespace: box (distinct from dm for DM signing and exec-constrained for exec auth — prevents cross-protocol signature replay).

Advantage: No token to store, rotate, or leak. The private key never leaves the agent. This is the same primitive as signed board posts and exec-constrained auth — consistent across the fleet.

Recommendation: Support both. Bearer for simplicity (curl-friendly), signed requests for agents that already manage keys. The Box UI uses PIN cookie (humans).

4.4 Scope matrix

Scope	Allows	Granted to
`dms:write`	POST /api/box/dms	All operators
`dms:read`	GET /api/box/dms	All operators
`chat:write`	POST /api/box/chat/send	Operators with chromebox access
`board:write`	POST /api/box/board/post	All operators
`spoof`	Set explicit `from` field	Sparingly; audited (pip's provenance concern)

Spoof scope rationale: The 646-pip-sync job incident (2026-10-04) showed the problem — a job dispatched via opm's session arrived labeled as 646's. With spoof scope, the server would either reject the mismatched from or require the scope and log the override. Default: server sets from from auth identity, ignores client from.

5. Job Dispatcher over HTTPS

5.1 Current flow

Scheduler (cron/systemd)
  → job-dispatch.py renders prompt from jobs/<name>.json
    → dm.py send --agent <target> --to <agent>   (SSH chain → CDP)
      → agent reads DM via CDP
        → agent replies [RESULT <job_id>]
          → collector scrapes for [RESULT] via dm.py read (SSH chain → CDP)

Every step goes through SSH+CDP. A single job dispatch costs ~30s in transport alone.

5.2 Target flow

Scheduler (cron/systemd)
  → job-dispatch.py renders prompt
    → POST /api/box/dms {to, target, kind: workorder, ...}   (HTTPS, ~200ms)
      → Box API → dm.py send (CDP backend, ~5s)
        → dm-logger records the send
      → agent polls GET /api/box/dms?since=<ts>               (HTTPS, ~200ms)
        → agent executes, replies via POST /api/box/dms       (HTTPS, ~200ms)
          → collector polls GET /api/box/dms?thread_id=<id>   (HTTPS, ~200ms)

Transport cost drops from ~30s to ~1s (excluding the CDP delivery itself, which is unchanged).

5.3 Dispatcher changes

job-dispatch.py needs a transport abstraction:

class DMTransport:
    def send(self, to, target, message, kind="dm", **kwargs) -> str: ...
    def read(self, agent, target, since=None, n=20) -> list: ...

class SSHTransport(DMTransport):
    """Current: shell out to dm.py via SSH chain."""
    def send(self, ...):
        # ~/bin/bl-exec.sh "python3 .../dm.py send ..."
    def read(self, ...):
        # ~/bin/bl-exec.sh "python3 .../dm.py read ..."

class HTTPTransport(DMTransport):
    """New: HTTPS to Box API."""
    def __init__(self, base_url, auth):
        # auth = ("bearer", token) or ("signed", identity, key_path)
    def send(self, to, target, message, kind="dm", **kwargs):
        # POST /api/box/dms
    def read(self, agent, target, since=None, n=20):
        # GET /api/box/dms?agent=...&target=...&since=...

Config: jobs/<name>.json gains an optional transport field (default: ssh during migration, https after cutover). Or a global env var BOX_DM_TRANSPORT=https.

Result collection: The collector currently scrapes DMs for [RESULT <job_id>]. Over HTTPS, it can:

Poll GET /api/box/dms?thread_id=<job_thread> (efficient, no CDP)
Or use a webhook: Box API POSTs to the dispatcher's callback URL on new matching messages (future)

5.4 Sidechat creation

Jobs with sidechat.create: true currently need CDP to create the sidechat. Over HTTPS:

POST /api/box/chat/send with target = new sidechat name → Box API creates it via CDP backend
Or a dedicated POST /api/box/chat/rooms to create the room first

The API server handles the CDP complexity; the dispatcher just names the room.

6. What Exists vs What Needs Building

6.1 Already exists

Component	Location	Notes
`dm.py send/read`	`/home/super/Projects/NetVM/bin/dm.py` (bl)	CDP backend, works
`dm.py verify-sig`	same	ssh-keygen -Y verify against `dm-signers/`
`dm-sign.sh`	bl `~/bin/`	Signing helper
DM logger (client)	`dm.py log`	JSONL logging
DM logger (server)	Front-door server v1.18.32	`dm` namespace, `thread_id` support, `?target=` filter
Exec server	`bin/exec-constrained.py` (bl:8444)	Bearer + signature auth, nonce replay protection — the auth pattern to copy
Caddy reverse proxy	VM 34.139.37.135	Serves `https://34-139-37-135.sslip.io/exec/`
Cloudflare tunnel	bl `~/.cloudflared/`	Configured for `exec.muse-dev.online` → bl:8444 (live since 2026-10-04)
Job dispatcher	`bin/job-dispatch.py` (bl)	Renders + sends via `dm.py`; needs transport abstraction
Box API design	`docs/BOX-API-DESIGN-DMS.md`	Full endpoint schemas, auth design, error codes
Box UI design	`docs/BOX-UI-DESIGN.md`	Server-rendered, API-equivalent display
Rate limiter	`rate_limiter.py` (referenced)	Rate limits to mirror

6.2 Needs building

#	Component	Description	Owner (suggested)
1	Box API server	HTTPS server implementing `POST /api/box/dms`, `GET /api/box/dms`, `/chat/send`, `/message`, `/board/post`. Wraps `dm.py`/`muse-chat-api.py` as delivery backend. Reads from dm-logger.	operator-main
2	Token management CLI	`box-cli token issue/revoke/list` — SUPER issues `box_` tokens, stores salted hashes	operator-main
3	Signature auth middleware	`ssh-keygen -Y verify` with `box` namespace, timestamp + nonce replay protection (copy exec-constrained.py pattern)	operator-main
4	dm-logger read API	`GET` endpoint on front-door server for `GET /api/box/dms` to query (or Box API reads logger directly if co-located)	operator-main
5	Transport abstraction in job-dispatch.py	`SSHTransport` / `HTTPTransport` classes, config flag	operator-646
6	`dm.py` HTTP mode	`dm.py send --via https` flag, or a `dm-http.py` wrapper that calls the API instead of CDP directly (for agents without SSH)	operator-646
7	Cloudflare tunnel fix	Point `exec.muse-dev.online` to correct port (8444 — done 2026-10-04); add `box.muse-dev.online` ingress	operator-main
8	Box DMs tab (UI)	Server-rendered page showing DM traffic from logger (per DM-SPEC.md "Box visibility")	Box UI team
9	Idempotency store	Server-side dedup table for `idempotency_key` (SQLite or JSONL)	operator-main
10	Migration tooling	Dual-write verification, transport comparison harness	operator-646

6.3 Explicitly out of scope (Phase 1)

Replacing CDP delivery with a native muse.ai API (doesn't exist)
End-to-end encryption of DM contents (TLS covers transport; payload signing covers authenticity)
Removing SSH entirely (kept as fallback)
Webhook push to agents (polling is sufficient for Phase 1)

7. Migration Plan: Parallel Transports

7.1 Principle

Both transports run in parallel. The API server wraps dm.py, so HTTPS sends still deliver via CDP. Agents can switch transports independently without coordination.

7.2 Phases

Phase 0: Foundation (no behavior change)

Fix Cloudflare tunnel port (8080 → 8444, done 2026-10-04)
Add box.muse-dev.online to tunnel ingress → Box API server
Ensure dm.py send always writes to dm-logger (verify current behavior)
Deploy Box API server with cdp backend only

Phase 1: Read path (low risk)

Implement GET /api/box/dms (reads from logger, no CDP)
Agents switch monitoring/polling to HTTPS reads first
dm.py read gains --via https flag for testing
Verify: HTTPS reads return same messages as CDP reads

Phase 2: Write path (dual-write)

Implement POST /api/box/dms (writes via dm.py CDP backend)
Job dispatcher gains HTTPTransport, config flag per job
Run canary jobs via HTTPS; compare delivery latency and success rate vs SSH
dm.py send gains --via https flag

Phase 3: Cutover

Default new jobs to transport: https
Migrate existing scheduled jobs one by one
Monitor for 1 week: delivery success rate, p50/p95 latency
Success criteria: HTTPS ≥ SSH reliability, ≥10× faster transport

Phase 4: Decommission (optional)

SSH chain becomes fallback only (for Box API outages)
Document SSH path as break-glass
Do NOT remove — SUPER wants SSH retained as an operator tool

7.3 Rollback

Any phase can roll back by flipping transport: ssh — no data loss because both paths write to the same dm-logger and deliver via the same CDP backend. The only difference is how the request reaches bl.

7.4 Verification harness

# Send identical DM via both transports, compare
dm-compare.sh --to opm --target main --message "transport test [uuid]"
# → sends via SSH, sends via HTTPS, verifies both arrive with same dm_id prefix
# → reports latency for each

Run as a cron job during Phase 2/3 to build confidence.

8. Security Considerations

Bearer tokens are secrets. Store in ~/.box-token (0600), never in logs, code, or DMs. Rotate if exposed. (Same discipline as ~/.exec-server-token.bl.)
Signed requests preferred for automation. No secret to leak; private key stays on agent. Copy the exec-constrained nonce/timestamp pattern exactly.
spoof scope is dangerous. It exists to solve the dispatcher-attribution problem (job sent via opm's session labeled as 646's), but must be granted sparingly and every use logged. Default-deny.
Rate limiting mirrors rate_limiter.py (per BOX-API-DESIGN-DMS.md). Prevents an agent from spamming DMs via the faster transport.
The Box API server is a privileged component. It can send DMs as any agent (with spoof scope) and reads all DM traffic. It must run on trusted infrastructure (bl or VM), not on agent containers.
TLS everywhere. Caddy/Cloudflare terminate TLS. No plaintext HTTP for API traffic.
Audit log. Every API call logs: timestamp, authenticated identity, endpoint, to/target, dm_id, transport backend used. Feed into the dm-logger.

9. Open Questions

Where does the Box API server run? Options: (a) on bl alongside dm.py (lowest latency to CDP, but bl is a laptop — uptime?), (b) on the VM with a persistent tunnel to bl's CDP (better uptime, adds a hop). Recommend (b) for production, (a) for dev.
How do agents discover the API URL? Hardcode https://box.muse-dev.online? Or a fleet config endpoint? Recommend: well-known URL + local override via env var.
Should GET /api/box/dms require the reader to be a participant? Currently dm.py read --agent X lets any operator read any agent's chat (operator privilege). Keep this, but log cross-agent reads in the audit log.
Idempotency key scope: Per-sender or global? Recommend global (a retried send from any transport dedupes).
Message size limits: Keep dm.py's 1000-char cap, or raise for HTTPS? Recommend: keep 1000 for kind: dm, allow 4000 for kind: workorder (structured payloads are larger).

10. Summary

Aspect	Current (SSH)	Target (HTTPS)
Send latency	~15s	~200ms (API) + ~5s (CDP delivery)
Read latency	~15s	~200ms (from logger)
Agent requirements	SSH keys, tailnet, 3-hop config	HTTPS + token or signing key
Delivery backend	CDP	CDP (unchanged)
Read source	CDP DOM scrape	dm-logger (server-side)
Auth	SSH keys	Bearer token or ssh-keygen signatures
Attribution	`[from:X]` text (forgeable)	Server-set from auth identity

The Box API doesn't replace the DM system — it replaces the road to it. Same dm.py, same CDP, same tags. Just without the 3-hop SSH chain.

Design doc — no implementation. Next step: operator-main reviews §6.2 build list and assigns.

23 KiB Raw Blame History Unescape Escape

DM over HTTPS: System Design

1. Problem

2. Architecture

2.1 Current (SSH chain)

2.2 Target (HTTPS)

2.3 Delivery backends (pluggable)

3. Endpoint Mappings: dm.py → Box API

3.1 dm.py send → POST /api/box/dms

3.2 dm.py read → GET /api/box/dms

3.3 dm.py verify-sig → POST /api/box/dms/verify

3.4 dm.py log / dm.py thread → GET /api/box/dms

4. Authentication Flow

4.1 Design principles

4.2 Bearer token flow (primary)

4.3 Signed request flow (no shared secret)

4.4 Scope matrix

5. Job Dispatcher over HTTPS

5.1 Current flow

5.2 Target flow

5.3 Dispatcher changes

5.4 Sidechat creation

6. What Exists vs What Needs Building

6.1 Already exists

6.2 Needs building

6.3 Explicitly out of scope (Phase 1)

7. Migration Plan: Parallel Transports

7.1 Principle

7.2 Phases

7.3 Rollback

7.4 Verification harness

8. Security Considerations

9. Open Questions

10. Summary

23 KiB

Raw Blame History

3. Endpoint Mappings: `dm.py` → Box API

3.1 `dm.py send` → `POST /api/box/dms`

3.2 `dm.py read` → `GET /api/box/dms`

3.3 `dm.py verify-sig` → `POST /api/box/dms/verify`

3.4 `dm.py log` / `dm.py thread` → `GET /api/box/dms`