box/docs/DIGEST-PROTOCOL.md

# Digest Protocol Runbook

> **Box is the main surface.** All operator work goes through Box (box.muse-dev.online). The web UI, `box` CLI, and agents share the same API endpoints. No UI-only powers.

How the main loop sends digests and how agents close them.

**Status:** spec (2026-10-04). Implemented in `self_main_loop.py` + `response-harvester.py`;
no new machinery — the protocol wires digests into the existing `dm.py --expect-reply`
/ `followups.json` / harvester response path.

---

## 1. Why this protocol exists

Main-loop digests (`[main-loop] N new in <agent> main chat: ...`) were being sent
with no response contract: no stable ID, no declared timeout, no nudge/escalation
policy, and no way to distinguish "seen and closed" from "ignored". Across 43
digests there were zero RESULT replies and zero measurable outcomes — not because
agents ignored them, but because nothing defined how to answer.

This protocol closes that wiring gap without inventing a new system:
- digest composer → `dm.py send --expect-reply` (followup record, timeout/nudges
  declared at send time)
- agent replies in-band with a verb: `[ACK|CLAIM|RESULT|DECLINE|NO-ACTION <id>]`
- `response-harvester.py` matches by `job_id` and records the outcome in
  `followups.json`
- the sweeper nudges/escalates only unresolved followups, per the declared policy

## 2. Digest classes

Not every digest deserves a reply demand (notification fatigue; and the standing
rule that the human only sees real decisions and escalations). The composer
already marks `[?]` (question) and `[!]` (operator-needed / urgent).

| Class | Marker | Response required? | Followup record? | Measured by |
|---|---|---|---|---|
| **ACTIONABLE** | carries `[?]` or `[!]` | **yes** | yes | delivered → acked → closed |
| **INFORMATIONAL** | neither | no | no | delivered only |

Informational digests are sent exactly as today. They are **excluded from every
reply-rate metric** — an informational digest is measured as *delivered*, full stop.
Judging informational FYI traffic by answer rate is structurally misleading.

## 3. Digest ID

Every ACTIONABLE digest gets a stable ID:

```
ml-<agent>-<YYYYMMDD-HHMMSS>
```

Example: `ml-646-20261004-213000`.

The ID is embedded visibly in the digest as `[JOB ml-...]`. This marker survives
`extract_trailing_tags` (only known canonical keys are stripped) and is picked up
by `dm_send`'s job_id regex (`\[JOB\s+([A-Za-z0-9_-]+)\]`) anywhere in the message,
so it lands in `tags["job_id"]` → the followup record. **No new dm.py flags are
needed for the ID.**

Replies match on this ID, so attribution games with `[from:X]` can never break
resolution.

## 4. Reply verbs

Printed in the contract footer of every ACTIONABLE digest (Section 7).

| Verb | Syntax | Meaning | Followup effect |
|---|---|---|---|
| `ACK` | `[ACK <id>]` | Seen, noted | status `acknowledged` — stops nudges, **not closed** |
| `CLAIM` | `[CLAIM <id>]` | I'm handling it | status `acknowledged` — stops nudges, **not closed** |
| `RESULT` | `[RESULT <id>] <result text>` | Done, result attached | status `resolved`, outcome `RESULT` — **closed** |
| `DECLINE` | `[DECLINE <id>]` | Won't act (reason optional) | status `resolved`, outcome `DECLINE` — **closed** |
| `NO-ACTION` | `[NO-ACTION <id>]` | Reviewed, nothing needed | status `resolved`, outcome `NO-ACTION` — **closed** |

Semantics:
- `ACK` vs `CLAIM`: both acknowledge. `CLAIM` signals ownership; `ACK` is just
  "seen". Use `CLAIM` when you're picking up the work so the human knows who's on it.
- `NO-ACTION` is a real answer, not a dodge. It means "I reviewed this and there
  is genuinely nothing to do." It closes the digest so it stops being counted as open.
- `DECLINE` should carry a reason when one exists (`[DECLINE ml-646-...] blocked:
  chromebox flapping`), but a bare DECLINE is still a valid close.
- `RESULT` closes only when the result text is present. An empty RESULT is a malformed
  reply — the harvester treats it as unrecognized.

Nudge interaction: `ACK`/`CLAIM` suppress further nudges but leave the followup open
until a closing verb arrives. This matches the real workflow — "I'm on it" shouldn't
trigger nags, but the loop isn't closed until there's a result.

## 5. Timeout policy

Declared at send time (standing rule: policy is part of the send, not decided later).

| Digest class | Timeout | Nudges |
|---|---|---|
| normal (marked `[?]`) | 60 min | 2 |
| urgent (marked `[!]`) | 30 min | 2 |

Implemented as `--reply-timeout {3600|1800} --reply-nudges 2` on the `dm.py send`
invocation for ACTIONABLE digests. The timeout and nudge count are stored in the
followup record; the sweeper honors them.

## 6. Nudge / escalation policy

- The sweeper nudges only followups with `status == "pending"`. Anything
  `acknowledged` or `resolved` is never re-nudged (ACKed digests getting nagged
  trains agents to ignore the protocol — verify this when wiring the sweeper).
- After the declared nudges are exhausted and the timeout passes, the followup is
  **escalated**, not retried forever:
  - **Phase 1:** the sweeper's existing escalation path (DM to the escalate target
    declared at send time via `--reply-escalate`).
  - **Phase 2:** route to the registered `main-loop brain` sidechat for triage.
    The brain is registered in `job-sidechats.json` as `main-loop brain` and is
    name-resolvable, so no hardcoded UUIDs are needed.
- `stale` = still `pending` past timeout + nudges → escalated.

Digest routing must use sidechat **names** (`646 tasks`, `main-loop brain`) via the
existing name resolution — never hardcoded thread UUIDs (see the stale-UUID lesson
in AGENTS.md).

Note: the harvester only monitors threads registered in `job-sidechats.json`.
`opm/heartbeat` is not registered (digests there resolve via fuzzy search), so
digest replies there would never resolve. **Route opm's digests to the registered
`main-loop brain` sidechat instead of `heartbeat`** — cleaner separation
(brain = triage workspace, heartbeat = heartbeats) and zero new registrations.

## 7. In-band contract footer

The digest budget is 600 chars, so the footer is compact. Actionable digests trim
message previews from 5 to 3 to fit it:

```
[main-loop] [JOB ml-646-20261004-213000] 3 new in 646 main chat — reply needed:
- human: <preview> [?]
Reply: [ACK id] seen | [CLAIM id] mine | [RESULT id] done | [DECLINE id] | [NO-ACTION id]. Report back here. Box dispatches the next step; do not DM the next agent directly.
```

The footer is the primary contract. This doc is the reference.

## 8. Example exchange

```
# opm → 646 tasks (ACTIONABLE, --expect-reply --reply-timeout 3600 --reply-nudges 2)

[from:opm] [id:a6579a8d] [main-loop] [JOB ml-646-20261004-213000] 3 new in 646 main chat — reply needed:
- human: should we archive the stale pipe-7d2896 thread? [?]
- opm: propose yes, it's superseded by the brain channel [?]
Reply: [ACK id] seen | [CLAIM id] mine | [RESULT id] done | [DECLINE id] | [NO-ACTION id]. Report back here. Box dispatches the next step; do not DM the next agent directly.

# 646 picks it up

[from:646] [CLAIM ml-646-20261004-213000] reviewing now

→ followup status: acknowledged (nudges suppressed, loop still open)

# 646 finishes

[from:646] [RESULT ml-646-20261004-213000] archived pipe-7d2896 via muse-cli; brain channel is canonical

→ followup status: resolved, outcome: RESULT (loop closed)
```

A shorter close:

```
# pip sees an informational-with-a-question, reviews, nothing to do

[from:pip] [NO-ACTION ml-pip-20261004-214500] reviewed — nothing needed, timer stagger already handles it

→ followup status: resolved, outcome: NO-ACTION (loop closed)
```

## 9. Metrics

Source of truth: `followups.json` (`outcome` field, written by the harvester).

| Metric | Definition |
|---|---|
| **delivered** | `dm.py` reported `verified:true` with placement confirmed for the digest message (see dm-log) |
| **acked** | outcome ∈ {`ACK`, `CLAIM`} |
| **closed** | outcome ∈ {`RESULT`, `DECLINE`, `NO-ACTION`} |
| **stale** | still `pending` past timeout + nudges → escalated |
| **closure_rate** | `closed / actionable_delivered` |

Rules:
- Informational digests are excluded from every rate. They are measured as
  delivered, full stop.
- Actionable digests are measured as closed. A digest that is ACKed but never
  resolved counts as open until a closing verb arrives.
- `delivered` requires the placement confirmation, not just the `verified:true`
  flag (see the AGENTS.md placement-blindness lesson).

Reporting: `self_main_loop.py` `status` exposes `digest_health()` counts
(delivered/acked/closed/stale), surfaced through the existing Box
`main-loop/status` API. No new endpoint needed in phase 1.

## 10. Implementation notes

- `dm.py` — **no changes.** `--expect-reply`, job_id extraction from `[JOB …]`,
  followup registration, and placement verification already exist and are tested.
- `response-harvester.py` — extend the reply regex to verb-aware matching:
  `\[(ACK|CLAIM|RESULT|DECLINE|NO-ACTION)\s+([A-Za-z0-9_-]+)\]`; on match, resolve
  by job_id and record `outcome`. `ACK`/`CLAIM` → `acknowledged`;
  `RESULT`/`DECLINE`/`NO-ACTION` → `resolved` + outcome.
- `self_main_loop.py` — `compose_digest` classifies actionable vs informational;
  actionable embeds `[JOB ml-…]` + footer; `send_prompt` passes
  `--expect-reply --reply-timeout {3600|1800} --reply-nudges 2` for actionable only.
- Sweeper — verify it nudges only `status == "pending"` (no code change expected).

All changes follow review-then-commit. No new daemons, no new state files beyond
the existing `followups.json`.

---

## 11. Recursive workflow discipline (box -> agent -> box)

Success = `box -> agent -> box -> agent -> box` (recursive). The failure mode is
`box -> agent : agent` — work handed agent-to-agent in chat threads, with nothing
ever returning to box. The tooling (`box job result|status|next|chain`) makes the
recursive path the easy path; this section makes it the default:

- **Reply in-thread.** Post `[RESULT <id>] <outcome>` back in the digest thread
  (this thread). The harvester watches it; `box job result <job-id>` records it
  into `job-log.jsonl`.
- **Box chains the next step.** A `[RESULT]` on a chained job fires
  `trigger_chain_next()` (or `box job next` to dry-run, `box job chain` to wire);
  box dispatches the next step to the next agent. The agent never needs to know
  who goes next.
- **Never DM the next agent directly.** Handing work off in a chat thread
  bypasses box orchestration: no followup record, no chain state, no
  `job status` visibility. If you catch yourself composing a DM to hand off
  work, stop and post the `[RESULT]` instead — box routes onward.
- **The footer says it.** Every ACTIONABLE digest now closes with:
  *"Report back here. Box dispatches the next step; do not DM the next agent
  directly."* — the contract is in-band, not tribal knowledge.

*Companion docs: `DM-SPEC.md` (DM format), `THREAD-BOOKKEEPING.md` (pin/archive),
`WARP-EGRESS-FIX.md` (partition handling).*