feat(supervision): add choice watcher daemon, HTTPS spec docs, and test suites
- bin/muse_choice_watcher.py + systemd/muse-choices-reconcile.*: automatic choice answering and timer reconciliation - bin/digest.py: fleet log and health summarization - docs/BOX-*-HTTPS.md: comprehensive HTTPS execution contracts and API documentation - docs/MUSE-CHOICES-POLICY.md & docs/SUPERVISION-SPEC.md: autonomous execution specs - tests/test_*.py: unit test suites for HTTPS API, choice watcher, fleet heal, and swarm pruning
This commit is contained in:
operator
@@ -0,0 +1,152 @@
|
||||
# Box Read-Only Lookups over HTTPS (Agent Access, No SSH)
|
||||
|
||||
> **Box is the main surface.** All operator work goes through Box (box.muse-dev.online). The web UI, `box` CLI, and agents share the same API endpoints. No UI-only powers.
|
||||
|
||||
**Date:** 2026-10-06
|
||||
**Status:** bl side implemented; VM board REST below is specified, not yet implemented
|
||||
**Scope:** read-only lookups only (fleet, threads, unread, dm log). Mutations stay on existing paths.
|
||||
|
||||
## 1. Problem
|
||||
|
||||
Agents in containers reach box over a 2-hop SSH chain (container → VM → bl).
|
||||
SSH toggles lapse and every agent needs the full chain configured. Agents need
|
||||
the daily read lookups — "latest from each agent" — over HTTPS with no secrets
|
||||
on the wire.
|
||||
|
||||
## 2. What exists now (bl side, implemented)
|
||||
|
||||
Two agent HTTPS paths already serve reads; both use the same signature auth
|
||||
(`ssh-keygen -Y sign`, namespace per server, ±300s clock skew, nonce replay
|
||||
cache) and per-agent principals from `dm-signers/allowed_signers`:
|
||||
|
||||
| Path | Server | Client | Auth namespace |
|
||||
|---|---|---|---|
|
||||
| Named ops (works today) | `bin/exec-constrained.py` via `https://exec.muse-dev.online/exec` | `bin/exec-sign.sh <op> '<args>'` or `bin/box-relay.sh` (served at `GET /box`) | `exec-constrained` |
|
||||
| Typed REST (specified below) | VM board `/srv/board/server.py` | any HTTPS client | `box-api` |
|
||||
|
||||
New named ops (this change, all `side_effecting: false`, all in `DEFAULT_PERMS`
|
||||
so any valid fleet signer may call them):
|
||||
|
||||
- `fleet.unread` `{"agent"?}` → `box-ctl.py unread [--agent X]`
|
||||
- `dm.log` `{"limit"?, "agent"?}` (limit 1..100, default 20) → `box-ctl.py dm-log [limit] [--agent X]`
|
||||
|
||||
Already present and unchanged: `health.check` (fleet status), `thread.list`,
|
||||
`thread.view`, `dm.read`, `chat.messages`.
|
||||
|
||||
New `box-relay.sh` client commands (this change):
|
||||
|
||||
```bash
|
||||
box unread [<agent>] # fleet unread/activity counts
|
||||
box dm log [<limit=20>] [--agent <agent>]
|
||||
```
|
||||
|
||||
New `box-ctl.py` backend verbs (this change; also callable over the board's
|
||||
existing SSH bridge until the board speaks REST):
|
||||
|
||||
```bash
|
||||
box-ctl.py unread [--agent <agent>]
|
||||
box-ctl.py dm-log [limit] [--agent <agent>] # back-compat: bare [limit] unchanged
|
||||
```
|
||||
|
||||
Also fixed: `box lookup unread` / `muse unread` previously always failed with
|
||||
"Unknown lookup target 'unread'" (`_lookup_unreads` was never wired into
|
||||
`cmd_lookup`); it now works and supports `--json`.
|
||||
|
||||
## 3. VM board REST (to implement on the VM)
|
||||
|
||||
Base: `https://box.muse-dev.online/api/box`. All endpoints require
|
||||
agent-signature auth (§4) or the existing `ops_session` cookie (humans).
|
||||
|
||||
```text
|
||||
GET /api/box/fleet exists today; keep behavior
|
||||
GET /api/box/threads?agent=X backend: box-ctl.py thread-list --agent X (pass JSON through)
|
||||
GET /api/box/unread?agent=X backend: box-ctl.py unread [--agent X]
|
||||
GET /api/box/dm/log?limit=N&agent=X
|
||||
backend: box-ctl.py dm-log [N] [--agent X]
|
||||
```
|
||||
|
||||
### Agent scoping (server-enforced)
|
||||
|
||||
- Verified identity `operator-X` or `X` (X in `muse,pip,646,opm,def,dev`)
|
||||
may only read slices for X. The board MUST pass `--agent X` to box-ctl and
|
||||
MUST NOT accept a different `agent=` query value from that identity.
|
||||
- `ops_session` (human PIN login) may omit `agent=` and read the full fleet.
|
||||
- Unknown/expired signatures → `401`. Authenticated but out-of-scope → `403`.
|
||||
|
||||
### Response schemas (bl verbs pass through unchanged)
|
||||
|
||||
`GET /api/box/unread`:
|
||||
|
||||
```json
|
||||
{"ok": true, "nodes": [
|
||||
{"node": "muse", "unread": 2, "approval_pending": false,
|
||||
"title": "muse (2)", "thread": "abc123-uuid-or-null"}
|
||||
]}
|
||||
```
|
||||
|
||||
`GET /api/box/dm/log` (agent filter matches entries from OR to the agent):
|
||||
|
||||
```json
|
||||
{"ok": true,
|
||||
"entries": [{"type": "sent", "id": "bdf7beb6", "agent": "opm",
|
||||
"to": "pip", "target": "pip tasks",
|
||||
"ts": "2026-10-06T05:56:01.328629+00:00"}],
|
||||
"dms": ["... same array, legacy key ..."]}
|
||||
```
|
||||
|
||||
`GET /api/box/fleet`: existing `{"ok": true, "fleet": [...]}` shape, unchanged.
|
||||
|
||||
Errors follow `docs/BOX-API-DESIGN-DMS.md` §3.1 (`{"ok": false, "code", "error"}`).
|
||||
|
||||
## 4. Agent-signature auth for the REST endpoints
|
||||
|
||||
Same identity primitive as signed DMs and `exec-constrained.py`; a signature
|
||||
is not a secret, so agents can sign without handling credentials.
|
||||
|
||||
1. Client builds the canonical string (LF-separated, no trailing newline):
|
||||
|
||||
```text
|
||||
{METHOD}\n{PATH}\n{SORTED_QUERY}\n{TS}\n{NONCE}
|
||||
```
|
||||
|
||||
- `METHOD`: `GET`; `PATH`: e.g. `/api/box/dm/log`; `SORTED_QUERY`: raw
|
||||
query string sorted by key (`agent=opm&limit=5`), empty string when none.
|
||||
- `TS`: unix epoch seconds; `NONCE`: 16–128 hex chars, single use.
|
||||
2. Client signs it: `ssh-keygen -Y sign -f <key> -n box-api`.
|
||||
3. Client sends headers (armor is base64-encoded to stay header-safe):
|
||||
|
||||
```text
|
||||
X-Box-Identity: operator-646
|
||||
X-Box-Timestamp: 1728...
|
||||
X-Box-Nonce: <hex>
|
||||
X-Box-Signature: <base64 of the -----BEGIN SSH SIGNATURE----- armor>
|
||||
```
|
||||
|
||||
4. Server recomputes the canonical string from the received request, base64-
|
||||
decodes the signature, and runs `ssh-keygen -Y verify -f allowed_signers
|
||||
-I <identity> -n box-api -s <sigfile>` with the canonical string on stdin.
|
||||
Accept only if: verify exit 0, `|now-TS| ≤ 300`, nonce unseen (cache ≥600s).
|
||||
Signers file is synced from bl `dm-signers/allowed_signers`.
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
TS=$(date +%s); NONCE=$(python3 -c "import secrets; print(secrets.token_hex(16))")
|
||||
CANON=$(printf 'GET\n/api/box/dm/log\nagent=opm&limit=5\n%s\n%s' "$TS" "$NONCE")
|
||||
SIG=$(printf '%s' "$CANON" | ssh-keygen -Y sign -f ~/.ssh/id_frontdoor -n box-api \
|
||||
| base64 -w0)
|
||||
curl -s 'https://box.muse-dev.online/api/box/dm/log?agent=opm&limit=5' \
|
||||
-H "X-Box-Identity: operator-646" -H "X-Box-Timestamp: $TS" \
|
||||
-H "X-Box-Nonce: $NONCE" -H "X-Box-Signature: $SIG"
|
||||
```
|
||||
|
||||
## 5. Rollout notes
|
||||
|
||||
- `exec-constrained.py` reads `OPS` at startup: restart the service after
|
||||
deploying for `fleet.unread` / `dm.log` to appear in `GET /ops`.
|
||||
- `box-relay.sh` is served from bl (`GET /box`); agents re-fetch to get
|
||||
`unread` / `dm log`.
|
||||
- Until the VM board implements §3, agents use the named-ops path (§2),
|
||||
which needs no SSH today.
|
||||
- Non-goals: write endpoints (`dm.send` etc. stay on the ops path for now),
|
||||
PIN/human flows (unchanged), secret handling (no secrets cross the wire).
|
||||
@@ -0,0 +1,92 @@
|
||||
# Box Approvals over HTTPS (No SSH)
|
||||
|
||||
> **Box is the main surface.** All operator work goes through Box (box.muse-dev.online). The web UI, `box` CLI, and agents share the same API endpoints. No UI-only powers.
|
||||
|
||||
**Date:** 2026-10-06
|
||||
**Status:** implemented on bl (`exec-constrained.py` + `box-relay.sh`;
|
||||
`box-ctl.py` verbs pre-existed, plus fast node validation and
|
||||
`quality-validate` branches; `approvals.py` untouched)
|
||||
**Scope:** approval visibility (check) + governed decisions (deny, auto,
|
||||
one-shot allow). Persistent/forced allow (`--always`/`--force`) stays
|
||||
SSH-only.
|
||||
|
||||
## 1. Why
|
||||
|
||||
Agents blocked on browser approvals needed SSH to see fleet approval
|
||||
state, deny a bad prompt, auto-resolve trusted prompts, or allow a
|
||||
known-good one. All of this now rides the agent HTTPS path
|
||||
(`https://exec.muse-dev.online/exec`, signature or Bearer [REDACTED], named-op
|
||||
allowlist, audit log).
|
||||
|
||||
## 2. New ops
|
||||
|
||||
| Op | Args | Backend | Access |
|
||||
|---|---|---|---|
|
||||
| `approval.check` | `{node?}` (default fleet) | `box-ctl.py approval-check` | read-only, in `DEFAULT_PERMS` |
|
||||
| `approval.deny` | `{node!, message!, allow_main_chat?}` | `box-ctl.py approval-deny` | known-identities-only |
|
||||
| `approval.auto` | `{node?}` (default fleet) | `box-ctl.py approval-auto` | known-identities-only |
|
||||
| `approval.allow` | `{node!, message!, allow_main_chat?}` | `box-ctl.py approval-allow` (one-shot) | known-identities-only |
|
||||
|
||||
New `box-relay.sh` client commands:
|
||||
|
||||
```bash
|
||||
box approvals check [node]
|
||||
box approvals allow <node> --message <text> [--allow-main-chat]
|
||||
box approvals deny <node> --message <text> [--allow-main-chat]
|
||||
box approvals auto [node]
|
||||
```
|
||||
|
||||
Raw op calls (signature auth, no token):
|
||||
|
||||
```bash
|
||||
exec-sign.sh approval.check '{}'
|
||||
exec-sign.sh approval.check '{"node": "646"}'
|
||||
exec-sign.sh approval.allow '{"node": "646", "message": "trusted deploy script"}'
|
||||
exec-sign.sh approval.auto '{"node": "opm"}'
|
||||
```
|
||||
|
||||
## 3. What the decisions do
|
||||
|
||||
- `approval.allow` clicks Allow **once** on the node's active prompt.
|
||||
There is deliberately no remote `--always` (persistent site allow)
|
||||
or `--force`.
|
||||
- `approval.deny` clicks Deny on the node's active prompt.
|
||||
- `approval.auto` scans (fleet or one node) and allows only TRUSTED
|
||||
non-key prompts. Key/passkey approvals are never auto-approved;
|
||||
they need an explicit allow/deny, which notifies the waiting agent.
|
||||
- All three are audited with identity + op + node.
|
||||
|
||||
## 4. Safety notes (same posture as existing ops)
|
||||
|
||||
- **Attribution is mandatory.** The allow/deny flows DM the waiting
|
||||
agent, historically with an `[operator]` prefix. A remote caller is
|
||||
an agent, not the operator — so the exec layer **requires** a
|
||||
non-empty `message` (≤2000 chars, no controls) on both allow and
|
||||
deny. (`box-ctl.py` still permits omitting `--message` for SSH
|
||||
callers; the HTTPS layer is the narrower gate.)
|
||||
- **One-shot only.** The allow argv never carries `--always` or
|
||||
`--force`; the validators reject those keys. Persistence stays an
|
||||
SSH-side decision.
|
||||
- **Sidechat-first.** `allow_main_chat` defaults to false; Main Chat
|
||||
delivery needs the explicit flag, same as `notify`/`dm.ack`.
|
||||
- **Fixed argv, validated values.** Nodes must be fleet members (fast
|
||||
`BAD_NODE` before any CDP probe — `approval-check`/`approval-auto`
|
||||
gained the same node check allow/deny already had); unknown arg
|
||||
keys rejected.
|
||||
- **Timeouts.** Check 180s (fleet CDP scan), auto 300s (scan plus one
|
||||
click per trusted prompt), allow/deny 120s.
|
||||
- **Retry-safe reads.** `approval-check` / `approval-list` joined
|
||||
`IDEMPOTENT_ACTIONS`; all six approval verbs have
|
||||
`quality-validate` dry-run branches.
|
||||
|
||||
## 5. Rollout notes
|
||||
|
||||
- Restart `exec-constrained.py` after deploy for the 4 new ops to
|
||||
appear in `GET /ops` (repo total becomes 83).
|
||||
- `box-relay.sh` is served from bl (`GET /box`); agents re-fetch to
|
||||
get the `approvals` group.
|
||||
- While fleet browsers crash-loop, decision ops fail honestly
|
||||
(`APPROVAL_FAILED` / CDP errors) instead of hanging; check still
|
||||
reports per-node state including `UNREACHABLE`.
|
||||
- Non-goals: deletes, `main-loop` enable/disable, policy writes,
|
||||
swarm kill/prune — future expansions, same pattern.
|
||||
@@ -0,0 +1,80 @@
|
||||
# Box Dev + Comms over HTTPS (No SSH)
|
||||
|
||||
> **Box is the main surface.** All operator work goes through Box (box.muse-dev.online). The web UI, `box` CLI, and agents share the same API endpoints. No UI-only powers.
|
||||
|
||||
**Date:** 2026-10-06
|
||||
**Status:** implemented on bl (`exec-constrained.py` + `box-ctl.py` + `box-relay.sh`)
|
||||
**Scope:** git visibility, test runs, notify, work-order acks. Read-only lookups
|
||||
live in `docs/BOX-API-READ-HTTPS.md`.
|
||||
|
||||
## 1. Why
|
||||
|
||||
Agents developing the box must inspect the tree, run the suite, nudge peers,
|
||||
and acknowledge work orders without SSH. All of this now rides the existing
|
||||
agent HTTPS path (`https://exec.muse-dev.online/exec`, signature or bearer
|
||||
auth, named-op allowlist, audit log) — no new trust model.
|
||||
|
||||
## 2. New ops
|
||||
|
||||
| Op | Args | Backend | Write? | Who |
|
||||
|---|---|---|---|---|
|
||||
| `git.status` | `{}` | `box-ctl.py git-status` | no | any valid signer |
|
||||
| `git.diff` | `{path?, stat?}` | `box-ctl.py git-diff [--stat] [--path p]` | no | any valid signer |
|
||||
| `git.log` | `{limit?, path?}` (1..50, default 10) | `box-ctl.py git-log` | no | any valid signer |
|
||||
| `tests.run` | `{test?, filter?}` (`tests.<module>` or full suite; `filter` is unittest `-k`) | `box-ctl.py tests-run [module] [--filter p]` | yes (executes) | known identities only |
|
||||
| `notify.send` | `{agent, message≤1000, sidechat?, sender?}` | `box-ctl.py notify ...` | yes (sends DM) | known identities only |
|
||||
| `dm.ack` | `{id, to, sender!, sidechat?, allow_main_chat?}` | `box-ctl.py ack ...` | yes (sends DM) | known identities only |
|
||||
|
||||
New `box-ctl.py` verbs: `git-status`, `git-diff`, `git-log`, `tests-run`,
|
||||
`ack` (all in `USAGE`, `quality-validate`, and — for the git reads —
|
||||
`IDEMPOTENT_ACTIONS`).
|
||||
|
||||
New `box-relay.sh` client commands:
|
||||
|
||||
```bash
|
||||
box git status
|
||||
box git diff [--stat] [--path <path>] # path also accepted positionally
|
||||
box git log [<limit=10>] [--path <path>]
|
||||
box tests run [tests.<module>] [--filter <pattern>] # full suite (~2-3 min) when omitted; filter is -k
|
||||
box notify <agent> [--sidechat <n>] [--sender <a>] <message...>
|
||||
box dm ack <id> --to <agent> --sender <agent> [--sidechat <name>]
|
||||
```
|
||||
|
||||
Raw op call (signature auth, no token):
|
||||
|
||||
```bash
|
||||
exec-sign.sh git.log '{"limit": 5, "path": "bin/dm.py"}'
|
||||
exec-sign.sh tests.run '{"test": "tests.test_box_dev_https"}'
|
||||
exec-sign.sh dm.ack '{"id": "bdf7beb6", "to": "pip", "sender": "opm"}'
|
||||
```
|
||||
|
||||
## 3. Safety notes (same posture as existing ops)
|
||||
|
||||
- **Fixed argv, validated values.** Clients influence only whitelisted argument
|
||||
values. Git paths must be repo-relative without `..` (plus symlink-escape
|
||||
check in box-ctl); test modules must match `^tests\.[a-z0-9_]+$` and exist;
|
||||
ack ids must be 6–64 hex; notify messages ≤1000 chars.
|
||||
- **Caps.** `git diff` output capped at 64KB, status at 200 entries, test
|
||||
output at 32KB tail; every capped response carries `truncated: true`.
|
||||
- **Sidechat-first.** `notify.send` and `dm.ack` default to the recipient's
|
||||
sidechat and never touch Main Chat unless `allow_main_chat` is set —
|
||||
mirroring `box-ctl.py notify` and the WO dispatcher.
|
||||
- **Attribution.** `sender` is caller-asserted (validated ∈ fleet agents),
|
||||
same as the existing `dm.send` op; the HTTPS identity is recorded
|
||||
separately in the exec audit log. `dm.ack` requires an explicit sender —
|
||||
no silent default.
|
||||
- **tests.run executes repo code** (whatever is in `tests/`), so it is
|
||||
`side_effecting`, excluded from the read-only default permission subset,
|
||||
and capped at a 600s timeout. Test failures report as
|
||||
`{"ok": false, "returncode", "output"}` — the op itself succeeded.
|
||||
- New `box-ctl.py` fail codes `GIT_ERROR` / `TESTS_ERROR` are registered in
|
||||
`KNOWN_ERROR_CODES`, so `quality-check` stays at its baseline.
|
||||
|
||||
## 4. Rollout notes
|
||||
|
||||
- Restart `exec-constrained.py` after deploy for the new ops to appear in
|
||||
`GET /ops`.
|
||||
- `box-relay.sh` is served from bl (`GET /box`); agents re-fetch to get
|
||||
`git` / `tests` / `notify` / `dm ack`.
|
||||
- Non-goals: git commit/push, service restarts for box itself, live
|
||||
streaming tails — future expansions, same pattern.
|
||||
@@ -0,0 +1,88 @@
|
||||
# Box Job Lifecycle over HTTPS (No SSH)
|
||||
|
||||
> **Box is the main surface.** All operator work goes through Box (box.muse-dev.online). The web UI, `box` CLI, and agents share the same API endpoints. No UI-only powers.
|
||||
|
||||
**Date:** 2026-10-06
|
||||
**Status:** implemented on bl (`exec-constrained.py` + `box-relay.sh`;
|
||||
`box-ctl.py` verbs pre-existed)
|
||||
**Scope:** safe job-lifecycle mutations. Deletes are deliberately NOT
|
||||
exposed (`job-delete`, `timer-delete` stay SSH/operator-only).
|
||||
|
||||
## 1. Why
|
||||
|
||||
Agents own scheduled automation but could only run jobs (`cron.run`) or read
|
||||
them (`box.exec`). Creating, updating, triggering, chaining, previewing, and
|
||||
pausing jobs needed SSH. All of this now rides the agent HTTPS path
|
||||
(`https://exec.muse-dev.online/exec`, signature or bearer auth, named-op
|
||||
allowlist, audit log).
|
||||
|
||||
## 2. New ops
|
||||
|
||||
| Op | Args | Backend | Write? | Who |
|
||||
|---|---|---|---|---|
|
||||
| `job.put` | `{name, definition}` | `box-ctl.py job-put` (definition on stdin) | yes (writes + commits) | known identities only |
|
||||
| `job.trigger` | `{name}` | `box-ctl.py job-trigger` | yes (dispatches now) | known identities only |
|
||||
| `job.chain` | `{from, to, on_failure?}` | `box-ctl.py job-chain` | yes (writes + commits) | known identities only |
|
||||
| `job.next` | `{job_id, success?}` | `box-ctl.py job-next` | no (dry-run) | any valid signer |
|
||||
| `cron.timer_stop` | `{name}` | `box-ctl.py timer-stop` | yes (systemd) | known identities only |
|
||||
| `cron.timer_disable` | `{name}` | `box-ctl.py timer-disable` | yes (systemd) | known identities only |
|
||||
|
||||
New `box-relay.sh` client commands:
|
||||
|
||||
```bash
|
||||
box cron put <name> '<json-definition>' # create or update (see §3)
|
||||
box cron trigger <name> # dispatch now (audited JSON)
|
||||
box cron chain <from> <to> [--on-failure] # wire chain_next
|
||||
box cron next <job-id> [--success|--fail] # dry-run: what dispatches next
|
||||
box timer stop <name> # pause schedule (keeps unit)
|
||||
box timer disable <name> # pause schedule (disables unit)
|
||||
```
|
||||
|
||||
Raw op call (signature auth, no token):
|
||||
|
||||
```bash
|
||||
exec-sign.sh job.next '{"job_id": "heartbeat-20200101-000000-deadbeef"}'
|
||||
exec-sign.sh job.chain '{"from": "ops-audit-step2", "to": "ops-audit-step3"}'
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- `job.trigger` vs existing `job.run`: `job.run` shells straight to
|
||||
`job-dispatch.py` and relays raw output; `job.trigger` goes through
|
||||
`box-ctl.py` (existence check, 300s bound, audit trail, JSON contract).
|
||||
Prefer `job.trigger` for agent-driven dispatches.
|
||||
- `job.next` job ids look like `<name>-YYYYMMDD-HHMMSS-<8hex>`; with no
|
||||
`success` flag the box infers it from the last recorded result.
|
||||
|
||||
## 3. Job definition shape (`job.put`)
|
||||
|
||||
The full schema is enforced by `box-ctl.py validate_job` (single copy);
|
||||
required fields: `name` (must match the argv name), `schedule` (`manual`
|
||||
or convertible cron), `agent` (fleet member), `prompt_template` (1–4000
|
||||
chars, no protocol literals, known `{placeholders}` only). `timeout`
|
||||
60–3600s, `on_failure` policy, optional `chain_next` (must exist) and
|
||||
`sidechat` / `dm_target` routing. `box-ctl.py` writes `jobs/<name>.json`
|
||||
and commits (`Add/Update job <name> via box-ctl`).
|
||||
|
||||
## 4. Safety notes (same posture as existing ops)
|
||||
|
||||
- **Fixed argv, validated values.** Job names match
|
||||
`^[a-z0-9][a-z0-9-]{0,63}$`; trigger/chain/timer ops require the job
|
||||
file to exist; chain rejects self-links and cycles; timer ops require
|
||||
the unit to exist. All checks run before any side effect.
|
||||
- **Stdin plumbing.** `job.put` is the first op to pipe a request body to
|
||||
`box-ctl.py` stdin (the raw definition, not the envelope); the routing
|
||||
lives in one helper (`_stdin_body`) covered by unit tests.
|
||||
- **No deletes, no kills.** `job-delete` / `timer-delete` are reachable
|
||||
only over the operator SSH path, by explicit scope decision.
|
||||
- **Audited.** Every execution records identity + op + args hash;
|
||||
`box-ctl.py` additionally audits each mutation with its target.
|
||||
|
||||
## 5. Rollout notes
|
||||
|
||||
- Restart `exec-constrained.py` after deploy for the new ops to appear in
|
||||
`GET /ops`.
|
||||
- `box-relay.sh` is served from bl (`GET /box`); agents re-fetch to get
|
||||
the `cron put|trigger|chain|next` and `timer stop|disable` commands.
|
||||
- Non-goals: deletes, `job-result` ingestion, `strat`/`loop` writes,
|
||||
approvals — future expansions, same pattern.
|
||||
@@ -0,0 +1,85 @@
|
||||
# Box Loop + Strategy Writes over HTTPS (No SSH)
|
||||
|
||||
> **Box is the main surface.** All operator work goes through Box (box.muse-dev.online). The web UI, `box` CLI, and agents share the same API endpoints. No UI-only powers.
|
||||
|
||||
**Date:** 2026-10-06
|
||||
**Status:** implemented on bl (`exec-constrained.py` + `box-relay.sh`;
|
||||
`box-ctl.py` verbs pre-existed, plus a vars-name validator fix)
|
||||
**Scope:** safe loop/strategy/variable mutations. Reads (`loop-status`,
|
||||
`loop-health`, `loop-breaks`, `strat-get`, `vars-get`, ...) already ride
|
||||
`box.exec` or typed read ops and are unchanged here.
|
||||
|
||||
## 1. Why
|
||||
|
||||
Agents watching loop health could see breaks but needed SSH to remediate
|
||||
them, resolve stale followups, tune strategy overrides, or undo variable
|
||||
changes. All of this now rides the agent HTTPS path
|
||||
(`https://exec.muse-dev.online/exec`, signature or bearer auth, named-op
|
||||
allowlist, audit log).
|
||||
|
||||
## 2. New ops (all known-identities-only, none in the read-only subset)
|
||||
|
||||
| Op | Args | Backend |
|
||||
|---|---|---|
|
||||
| `loop.remediate` | `{dry_run?}` (default false) | `box-ctl.py loop-remediate [--dry-run]` |
|
||||
| `loop.resolve` | `{dm_id, note?}` | `box-ctl.py loop-resolve` |
|
||||
| `strat.set` | `{type!, subtype?, agent?, track?, priority?, timeout_s?, nudges?, escalate?}` | `box-ctl.py strat-set` (payload as argv JSON) |
|
||||
| `strat.reset` | `{type!, subtype?, agent?}` | `box-ctl.py strat-reset` |
|
||||
| `vars.reset` | `{name}` | `box-ctl.py vars-reset` (restore default) |
|
||||
| `vars.rollback` | `{name, revision?}` (int step or timestamp) | `box-ctl.py vars-rollback` |
|
||||
|
||||
New `box-relay.sh` client commands:
|
||||
|
||||
```bash
|
||||
box loop remediate [--dry-run]
|
||||
box loop resolve <dm_id> [note...]
|
||||
box strat set <type> [--subtype S] [--agent A] [--track true|false] [--priority p] [--timeout N] [--nudges N] [--escalate E]
|
||||
box strat reset <type> [subtype] [--agent <agent>]
|
||||
box vars reset <name>
|
||||
box vars rollback <name> [revision]
|
||||
```
|
||||
|
||||
Raw op call (signature auth, no token):
|
||||
|
||||
```bash
|
||||
exec-sign.sh loop.remediate '{"dry_run": true}'
|
||||
exec-sign.sh strat.set '{"type": "job", "priority": "important", "nudges": 3}'
|
||||
```
|
||||
|
||||
## 3. What remediate does (non-dry)
|
||||
|
||||
`loop.remediate` delegates to `gravity.remediate_breaks`: resolves
|
||||
followups already answered in logs, re-arms expired followups with nudges
|
||||
remaining (runs the followup sweeper once), auto-allows TRUSTED (non-key)
|
||||
browser approvals, and on hard breaks appends a `hard_break_alert` to
|
||||
job-log plus one DM to opm. Use `{"dry_run": true}` first to preview the
|
||||
`remediated` / `escalated` lists with zero side effects.
|
||||
|
||||
## 4. Safety notes (same posture as existing ops)
|
||||
|
||||
- **Fixed argv, validated values.** Strategy types are a strict enum
|
||||
(`wake|job|siphon|manual|health|heartbeat`) — the backend silently maps
|
||||
typos to MANUAL, so the op rejects them instead. Priorities are a
|
||||
strict enum; timeouts/nudges must be integers (backend clamps);
|
||||
dm ids must be 6–64 hex; variable names mirror the engine identifier
|
||||
rule. All checks run before any side effect.
|
||||
- **Stdin-free.** Unlike `job.put`, `strat.set` passes its JSON payload
|
||||
as an argv token (`strat-set <type> [JSON]`), so no new stdin plumbing
|
||||
was needed.
|
||||
- **Vars-name validator fix.** `quality-validate` for all five vars
|
||||
verbs used the job-name regex (`^[a-z0-9-]{1,64}$`), rejecting every
|
||||
real variable name (`max_nudge_count`, ...). They now share
|
||||
`qv_var_name` (`^[A-Za-z0-9_.-]{1,64}$`), mirroring the
|
||||
`exec-constrained.py` rule.
|
||||
- **Audited.** Every execution records identity + op + args hash;
|
||||
`box-ctl.py` additionally audits each mutation with its target.
|
||||
|
||||
## 5. Rollout notes
|
||||
|
||||
- Restart `exec-constrained.py` after deploy for the new ops to appear in
|
||||
`GET /ops`.
|
||||
- `box-relay.sh` is served from bl (`GET /box`); agents re-fetch to get
|
||||
the `loop` / `strat` groups and `vars reset|rollback`.
|
||||
- Non-goals: approvals, deletes, `main-loop` enable/disable — future
|
||||
expansions, same pattern. (md drive files shipped separately; see
|
||||
BOX-MD-HTTPS.md.)
|
||||
@@ -0,0 +1,109 @@
|
||||
# Box Md Drive Files over HTTPS (No SSH)
|
||||
|
||||
> **Box is the main surface.** All operator work goes through Box (box.muse-dev.online). The web UI, `box` CLI, and agents share the same API endpoints. No UI-only powers.
|
||||
|
||||
**Date:** 2026-10-06
|
||||
**Status:** implemented on bl (`exec-constrained.py` + `box-relay.sh`;
|
||||
`box-ctl.py` verbs pre-existed, plus traversal hardening, output caps,
|
||||
`--stdin` content plumbing, and hyphenated amend/append/pull aliases)
|
||||
**Scope:** md reads (audit/list/read/diff) + governed writes
|
||||
(amend/append/pull/inject-drive/sync-all). Raw container writes
|
||||
(`md-write`) stay SSH-only by design.
|
||||
|
||||
## 1. Why
|
||||
|
||||
Agents shaping fleet behavior could see drive scores but needed SSH to
|
||||
read an agent's `SOUL.md`, diff it against the canonical template, or
|
||||
push updated operator files. All of this now rides the agent HTTPS path
|
||||
(`https://exec.muse-dev.online/exec`, signature or Bearer [REDACTED], named-op
|
||||
allowlist, audit log).
|
||||
|
||||
## 2. New ops
|
||||
|
||||
Reads (all `side_effecting: false`, all in `DEFAULT_PERMS`):
|
||||
|
||||
| Op | Args | Backend |
|
||||
|---|---|---|
|
||||
| `md.audit` | `{accounts?}` (default all) | `box-ctl.py md-audit [accounts...]` |
|
||||
| `md.list` | `{account!, path?}` | `box-ctl.py md-list` (capped, see §4) |
|
||||
| `md.read` | `{account!, filename!}` | `box-ctl.py md-read` (capped, see §4) |
|
||||
| `md.diff` | `{account!, filename!}` (shared template only) | `box-ctl.py md-diff` (capped, see §4) |
|
||||
|
||||
Governed writes (all known-identities-only, none in the read-only subset):
|
||||
|
||||
| Op | Args | Backend |
|
||||
|---|---|---|
|
||||
| `md.pull` | `{account!, filename!}` (shared template only) | `box-ctl.py md-pull` |
|
||||
| `md.inject_drive` | `{account!}` | `box-ctl.py md-inject-drive` |
|
||||
| `md.sync_all` | `{}` | `box-ctl.py md-sync-all` |
|
||||
| `md.amend` | `{filename!, content!, author?, reason?}` | `box-ctl.py md-amend --stdin` (content on stdin) |
|
||||
| `md.append` | `{filename!, text!, author?, section?}` | `box-ctl.py md-append --stdin` (text on stdin) |
|
||||
|
||||
New `box-relay.sh` client commands:
|
||||
|
||||
```bash
|
||||
box md audit [accounts...]
|
||||
box md list <account> [path]
|
||||
box md read <account> <filename>
|
||||
box md diff <account> <filename>
|
||||
box md pull <account> <filename>
|
||||
box md inject-drive <account>
|
||||
box md sync-all
|
||||
box md amend <filename> (--content <text>|--file <path>) [--author <name>] [--reason <why>]
|
||||
box md append <filename> (--content <text>|--file <path>) [--author <name>] [--section <header>]
|
||||
```
|
||||
|
||||
Raw op calls (signature auth, no token):
|
||||
|
||||
```bash
|
||||
exec-sign.sh md.audit '{"accounts": ["646", "opm"]}'
|
||||
exec-sign.sh md.read '{"account": "646", "filename": "SOUL.md"}'
|
||||
exec-sign.sh md.diff '{"account": "pip", "filename": "HEARTBEAT.md"}'
|
||||
exec-sign.sh md.append '{"filename": "AGENTS.md", "text": "lesson ...", "author": "646"}'
|
||||
```
|
||||
|
||||
## 3. What the governed writes do
|
||||
|
||||
- `md.amend` rewrites a `shared/operators/` template after the
|
||||
drive-safety checks (HEARTBEAT checklist not gutted,
|
||||
PROACTIVE_PREFERENCES not blanked, SOUL not reverted to stock),
|
||||
then git-commits it. Full-file content rides stdin (up to 256KB).
|
||||
- `md.append` appends a timestamped, attributed note (optional section)
|
||||
via the same validated + committed path (up to 64KB).
|
||||
- `md.pull` / `md.inject_drive` / `md.sync_all` push canonical
|
||||
templates *out* to containers; no agent-supplied content crosses.
|
||||
Injection always overwrites (AGENTS.md preserves remote `## Lessons`).
|
||||
|
||||
## 4. Safety notes (same posture as existing ops)
|
||||
|
||||
- **Traversal hardening (single-copy in `agent_md.py`).** Account,
|
||||
filename, and list-path validation now lives in `agent_md.py`
|
||||
(`MDValidationError`, raised before any gateway call or write);
|
||||
`box-ctl.py` maps it to `BAD_NAME`, and exec ops + quality-validate
|
||||
mirror the same shapes. Previously `md-read 646 ../x` reached the
|
||||
gateway and `md amend ../../x` could escape `shared/operators/`.
|
||||
Template flows (diff/amend/append/pull) additionally require one of
|
||||
the 8 known template names.
|
||||
- **Fixed argv, validated values.** Unknown arg keys rejected; author /
|
||||
reason / section are control-char-free with length caps; amend
|
||||
content must be non-empty.
|
||||
- **Caps with `truncated` flags.** Reads cap at 64KB, diffs at 64KB,
|
||||
listings at 200 entries — same convention as git/tests verbs.
|
||||
- **No raw `md-write` op.** Arbitrary content-to-container stays
|
||||
SSH-only; remote writes go through the validated template flows.
|
||||
- **Timeouts.** Audit 300s, sync-all 600s, single-file ops 120s.
|
||||
- **Audited.** Every execution records identity + op; `box-ctl.py`
|
||||
additionally audits each verb with its target.
|
||||
- **Retry-safe reads.** `md-audit` / `md-list` / `md-read` / `md-diff`
|
||||
joined `IDEMPOTENT_ACTIONS`; all ten md verbs have
|
||||
`quality-validate` dry-run branches.
|
||||
|
||||
## 5. Rollout notes
|
||||
|
||||
- Restart `exec-constrained.py` after deploy for the 9 new ops to
|
||||
appear in `GET /ops` (repo total becomes 79).
|
||||
- `box-relay.sh` is served from bl (`GET /box`); agents re-fetch to
|
||||
get the `md` group.
|
||||
- Non-goals: deletes, `main-loop` enable/disable, policy writes,
|
||||
swarm kill/prune — future expansions, same pattern. (Approvals
|
||||
shipped separately; see BOX-APPROVALS-HTTPS.md.)
|
||||
@@ -0,0 +1,103 @@
|
||||
# Muse-Choices Deny/Escalate Policy — DECISION RECORD (Final)
|
||||
|
||||
Topic: add deny/escalate decisions to the `muse-choices` auto-approve daemon
|
||||
(`bin/muse_choice_watcher.py`), which today only approves (top choice per
|
||||
prompt kind). Interviewed 2026-10-06/07 per grill contract; accepted
|
||||
verbatim below, which flipped this record from Draft to Final.
|
||||
|
||||
## Standing constraints (settled by user)
|
||||
|
||||
- All prompts must resolve: no stuck states are acceptable in any outcome.
|
||||
- The full-auto top-choice flow must always exist as a path.
|
||||
- Model review of choices is a FUTURE layer. Deferred out of this interview.
|
||||
|
||||
## Settled decisions
|
||||
|
||||
- D0 (helper form): a checked-in repo rules file informs decisions.
|
||||
Source: user's structured answerquared 2026-10-06 ("Repo rules file
|
||||
(Recommended)" for "which helper should inform approve/deny/hold
|
||||
decisions"). Rationale recorded at selection time: deterministic,
|
||||
versioned, sub-second, unit-testable; no agent round-trip latency.
|
||||
|
||||
## Settled during interview
|
||||
|
||||
- D0b (approve path needs no helper): straightforward prompts resolve
|
||||
locally with top-choice keys (the five matcher kinds, already
|
||||
implemented and live). Helpers (D0 rules file) govern deny/hold
|
||||
judgments only. Source: user direction 2026-10-06 ("the watcher
|
||||
itself should be able to input 1"; "always have the flow for full
|
||||
auto just top choice").
|
||||
- D1 (rule match dimensions): command pattern first, plus kind and
|
||||
text pattern. Source: user selected option 1, 2026-10-06.
|
||||
Rationale: the observed risk lives in the `$ command` of approval
|
||||
dialogs; kind/text add precision around it.
|
||||
- D2 (deny mechanics): deny exists only for permission kinds --
|
||||
`muse-approval` dialogs receive `2` + Enter, `y/n` prompts receive
|
||||
`n` + Enter. Question kinds (interview, letter, numbered) always
|
||||
resolve top-choice and are never denied. Source: user selected
|
||||
option 1, 2026-10-06. Rationale: deny is only meaningful where a
|
||||
permission is refused; questions stay total.
|
||||
- D3 (hold mechanics): hold leaves the dialog untouched, suppresses
|
||||
auto-answer, raises a HELD entry in `box muse-choices status` plus
|
||||
an audit record; the operator resolves via a box command, otherwise
|
||||
a SHORT window expires back to top-choice approve. Source: user
|
||||
selected option 1 with "short window", 2026-10-06. Exact duration
|
||||
proposed below (2 minutes, tunable); accepted or amended with the
|
||||
scope text in D5.
|
||||
|
||||
- D4 (unmatched default): approve top-choice, exactly today's
|
||||
behavior. Source: user selected option 1, 2026-10-06. Rationale:
|
||||
follows from the standing constraints; rules carve out only
|
||||
deny/hold exceptions, so an empty rules file changes nothing.
|
||||
|
||||
## Open questions (unresolved)
|
||||
|
||||
None. All interview questions resolved and the scope accepted.
|
||||
|
||||
## Scope contract (ACCEPTED)
|
||||
|
||||
Artifact boundary, IN:
|
||||
|
||||
- `docs/MUSE-CHOICES-POLICY.md`: this record (Draft -> Final on acceptance).
|
||||
- New `muse-choices-rules.json` at repo root (beside
|
||||
`keepalive-config.json`): the checked-in deny/hold rules.
|
||||
- `bin/muse_choice_watcher.py`: rule evaluation, deny/hold paths, HELD
|
||||
state with short-window expiry, resolve plumbing.
|
||||
- `bin/super-cli.py`: `box muse-choices resolve` command + HELD display
|
||||
in status.
|
||||
- `tests/test_muse_choice_watcher.py`: rule eval, per-kind deny keys,
|
||||
hold/suppress/expiry, resolve flow.
|
||||
|
||||
Artifact boundary, OUT (rejected or deferred, each needs its own
|
||||
interview to re-enter):
|
||||
|
||||
- Model review of choices (deferred future stage).
|
||||
- Peer-agent consultation (rejected in D0).
|
||||
- New matcher shapes (matcher suite's lane).
|
||||
- `box runtime` work (adjacent lane, untouched).
|
||||
- Timer cadence / daemon supervision changes.
|
||||
|
||||
Done means (all observable):
|
||||
|
||||
- [ ] This record marked Final with the acceptance quoted.
|
||||
- [ ] Rules file loads; empty rules == today's behavior exactly.
|
||||
- [ ] Deny sends `2`+Enter / `n`+Enter per D2: unit tests + one live
|
||||
scratch proof per permission kind.
|
||||
- [ ] Hold suppresses + shows HELD + resolves via box + expires to
|
||||
approve: unit tests + one live scratch proof of hold and one of
|
||||
expiry.
|
||||
- [ ] Audit records for deny/hold/resolve/expire in `box-ctl.jsonl`.
|
||||
- [ ] Full suite green; fleet reloaded; desired state left as found.
|
||||
|
||||
Acceptance (quoted verbatim, chat, 2026-10-07T00:19:57Z): "ACCEPT".
|
||||
Accepted as written, including the 2-minute tunable hold window. Per the
|
||||
grill scope contract, later work outside the IN boundary needs explicit
|
||||
owner approval or its own follow-up interview; "go" authorizes only this
|
||||
boundary. No owning issue exists in this workflow, so this record is the
|
||||
lane-coordination evidence.
|
||||
|
||||
## Non-goals (accepted with the scope)
|
||||
|
||||
- Model-based review of choices (deferred future layer).
|
||||
- Peer-agent consultation over sidechat (rejected in favor of D0).
|
||||
- Changes to approval matching shapes (covered by the matcher test suite).
|
||||
@@ -0,0 +1,81 @@
|
||||
# Supervision Scope Contract
|
||||
|
||||
Status: **Draft** — decisions below are unsettled until marked otherwise.
|
||||
Only explicit user acceptance moves this document (or any decision) to Final.
|
||||
|
||||
## Goal
|
||||
|
||||
Every fleet node stays alive and truthfully reported: browsers supervised,
|
||||
relays supervised, dead nodes recovered or loudly paged, and `box` status
|
||||
honest from any shell (including PID/net-blind sandboxed shells).
|
||||
|
||||
## Non-goals (proposed)
|
||||
|
||||
- Agent lifecycle/onboarding stages (provision, auth, OTP, invite redeem).
|
||||
- Work completion (job dispatch, followups, harvester, completion auditor).
|
||||
- Loop-health scoring and drive repair.
|
||||
|
||||
## Supervisors (observed, all installed 2026-10-06)
|
||||
|
||||
| Supervisor | Cadence | Coverage | Decides |
|
||||
|---|---|---|---|
|
||||
| chromebox-watchdog@\<node\>.timer ×6 | 2 min | all registry nodes (def/dev timers installed 18:32Z) | browser+egress per node; tunnel restart, chrome relaunch |
|
||||
| cdp-relay-watchdog.timer | 5 min | registry-driven (`watched_nodes()`) | relay veth IP + connectivity; relay restart |
|
||||
| agent-health.timer (user) | 5 min | registry-driven | API check per node; kill+restart with 2-strike rule + circuit breaker (3 futile → open 30 min) |
|
||||
| ensure-node-supervision.sh | on node-up / `--all` | new + drifted nodes | NODES.md row + chromebox timer install |
|
||||
| host_evidence fallback | on `box` read | registry (relay) + installed timers (browser) | effective status when live probes are blind |
|
||||
|
||||
## Decisions
|
||||
|
||||
(D1..D7 below — all UNRESOLVED unless marked.)
|
||||
|
||||
### D1. Contract boundary: which supervisors are in scope — SETTLED (recommended accepted)
|
||||
|
||||
IN: chromebox-watchdog ×6, cdp-relay-watchdog, agent-health + circuit
|
||||
breaker, ensure-node-supervision feed, host_evidence fallback.
|
||||
OUT: onboarding pipeline lifecycle, completion auditor, loop-health
|
||||
(each keeps its own owner and interviews separately).
|
||||
|
||||
### D2. Kill-path precedence (chromebox-watchdog vs agent-health) — UNRESOLVED
|
||||
|
||||
Both can kill a browser today; only time guards (<2 min) de-conflict them.
|
||||
|
||||
### D3. Egress-down fall-through (relaunch chrome after failed tunnel restart?) — UNRESOLVED
|
||||
|
||||
Observed 19:12Z: tunnel restart failed, watchdog relaunched chrome 3×
|
||||
anyway (one FAILED page). Browser was never the problem.
|
||||
|
||||
### D4. Circuit-breaker thresholds (3 futile / 30 min cooldown) — UNRESOLVED
|
||||
|
||||
Current values unvalidated against real recurrence intervals.
|
||||
|
||||
### D5. Coverage source of truth — UNRESOLVED
|
||||
|
||||
Registry-only vs registry+installed-timers for browser verdicts.
|
||||
|
||||
### D6. Concurrent-edit protocol for shared supervision files — UNRESOLVED
|
||||
|
||||
Two agents editing super-cli.py / watchdogs / runbook; one clobber
|
||||
(18:03Z) and one unattributed commit (c9143a5) already occurred.
|
||||
|
||||
### D7. Done means — UNRESOLVED
|
||||
|
||||
Proposed checklist: timers on all 6 firing silent; relay/agent-health
|
||||
loops registry-driven with tests; ensure hook live; this doc Final.
|
||||
|
||||
## Risks
|
||||
|
||||
- Egress-down pages read as browser failures (D3).
|
||||
- Uncommitted supervisor work can be clobbered by a concurrent editor (D6).
|
||||
- c9143a5 contains unattributed peer hunks (host_evidence `_covered_nodes`,
|
||||
fleet-status test updates) — needs an amend-or-leave decision.
|
||||
|
||||
## Validation
|
||||
|
||||
- `box fleet status` truthful from blind shells (live-verified 6/6 ACTIVE).
|
||||
- Focused suites green (supervision, fleet, agent-health, watchdog coverage).
|
||||
- Timer firing proven via journal, not config presence.
|
||||
|
||||
## Unresolved items
|
||||
|
||||
D2–D7 unresolved. D1 settled.
|
||||
Reference in New Issue
Block a user