box/docs/BOX-API-SPEC.md

# BOX-API-SPEC.md: Agentic Timer Management Interface

## Overview

The box page (`https://box.muse-dev.online/`) provides an agentic interface
for scheduling and managing systemd timers on bl (the main compute host).
Agents and humans use the same REST API; the web UI is a thin client over it.

**Principle:** Timers are jobs. The box API is the control plane for the
JOB system (see JOB-SPEC.md). Agents can list, create, and manage timers
without SSH access to bl.

## Authentication

Same ops auth as the box page:
- **Bearer token**: `Authorization: Bearer <operator-token>` (for agents)
- **PIN session**: Cookie from `/api/box/login` (for humans via UI)
- **Agent signatures**: Box-signed requests (for automated agents)

All endpoints require ops-level auth. Anonymous requests get 403.
Public users see only the PIN login page, never timer data.

## Base URL

```
https://box.muse-dev.online/api/box
```

## Endpoints

### List Timers

```
GET /api/box/timers
```

Returns all systemd user timers on bl.

**Response (200):**
```json
{
  "timers": [
    {
      "name": "job-board-watch.timer",
      "active": true,
      "enabled": true,
      "last_run": "2026-10-04T02:30:00Z",
      "next_run": "2026-10-04T02:35:00Z",
      "job_name": "board-watch"
    }
  ]
}
```

### Get Timer Status

```
GET /api/box/timers/{name}
```

**Response (200):**
```json
{
  "name": "job-board-watch.timer",
  "active": true,
  "enabled": true,
  "last_run": "2026-10-04T02:30:00Z",
  "next_run": "2026-10-04T02:35:00Z",
  "last_result": "success",
  "job_definition": { ... }  // parsed YAML
}
```

**Errors:**
- `404`: Timer not found

### Create Timer (and Job)

```
POST /api/box/timers
Content-Type: application/yaml
```

Body is a Job YAML definition (see JOB-SPEC.md).

**Example body:**
```yaml
name: board-watch
schedule: "*/5 * * * *"
agent: muse
prompt_template: "Check board..."
timeout: 300
```

**Response (201):**
```json
{
  "name": "job-board-watch.timer",
  "job_name": "board-watch",
  "created": true
}
```

**Errors:**
- `400`: Invalid YAML or missing required fields
- `409`: Timer already exists

**Backend actions:**
1. Write `/home/super/Projects/NetVM/jobs/<name>.yaml`
2. Generate systemd `.service` and `.timer` units
3. `systemctl --user daemon-reload`
4. `systemctl --user enable --now job-<name>.timer`

### Control Timer

```
POST /api/box/timers/{name}/start
POST /api/box/timers/{name}/stop
POST /api/box/timers/{name}/enable
POST /api/box/timers/{name}/disable
POST /api/box/timers/{name}/restart
```

**Response (200):**
```json
{
  "name": "job-board-watch.timer",
  "action": "start",
  "success": true
}
```

### Delete Timer

```
DELETE /api/box/timers/{name}
```

**Response (200):**
```json
{
  "name": "job-board-watch.timer",
  "deleted": true
}
```

**Backend actions:**
1. `systemctl --user stop job-<name>.timer`
2. `systemctl --user disable job-<name>.timer`
3. Remove unit files
4. `systemctl --user daemon-reload`
5. Optionally keep or delete the job YAML (query param `?keep_yaml=true`)

### List Jobs

```
GET /api/box/jobs
```

Returns job definitions (YAML files in `/home/super/Projects/NetVM/jobs/`).

**Response (200):**
```json
{
  "jobs": [
    {
      "name": "board-watch",
      "schedule": "*/5 * * * *",
      "agent": "muse",
      "has_timer": true,
      "timer_active": true
    }
  ]
}
```

### Get Job Definition

```
GET /api/box/jobs/{name}
```

**Response (200):**
```json
{
  "name": "board-watch",
  "definition": { ... }  // parsed YAML
}
```

### Trigger Job Now

```
POST /api/box/jobs/{name}/trigger
```

Manually trigger a job immediately (outside its schedule). Useful for testing.

**Response (200):**
```json
{
  "job_name": "board-watch",
  "job_id": "board-watch-20261004-023500-x1y2z3",
  "triggered": true
}
```

## Backend Implementation

The board server (on VM, port 8090) proxies to bl via SSH.

**SSH path:** VM → bl via tailnet (`super@100.123.153.75`), using VM's SSH key.
The board server runs as `super` on VM, which has passwordless SSH to bl.

**Command mapping:**

| API | SSH command on bl |
|-----|-------------------|
| List timers | `systemctl --user list-timers --no-pager --output=json` |
| Get status | `systemctl --user status job-{name}.timer --no-pager` |
| Start | `systemctl --user start job-{name}.timer` |
| Stop | `systemctl --user stop job-{name}.timer` |
| Enable | `systemctl --user enable job-{name}.timer` |
| Disable | `systemctl --user disable job-{name}.timer` |
| Create | Write YAML + units, `daemon-reload`, `enable --now` |

**Security:**
- The board server validates job YAML before writing (schema check)
- Timer names are sanitized: `^[a-z0-9-]+$` only
- No shell injection: use `systemctl` with fixed args, never shell-expand user input
- SSH uses a restricted key (only `systemctl --user` commands allowed via `authorized_keys` command restriction)
- All actions logged to `/srv/box/audit.jsonl`

## Rate Limiting

Use shared rate limiter (see `bin/rate_limiter.py` on bl):
- Max 10 timer operations per minute per agent
- Prevents accidental timer spam

## Error Handling

All errors return JSON:
```json
{
  "error": "Timer not found",
  "code": "NOT_FOUND"
}
```

**Codes:**
- `NOT_FOUND` (404)
- `INVALID_YAML` (400)
- `ALREADY_EXISTS` (409)
- `SSH_ERROR` (502) — bl unreachable
- `FORBIDDEN` (403) — insufficient auth

## Agentic Usage Examples

**Python (using requests):**
```python
import requests

TOKEN = "opm-bearer-token-here"
BASE = "https://box.muse-dev.online/api/box"
H = {"Authorization": f"Bearer {TOKEN}"}

# List timers
r = requests.get(f"{BASE}/timers", headers=H)
print(r.json())

# Create job
job_yaml = open("myjob.yaml").read()
r = requests.post(f"{BASE}/timers", headers={**H, "Content-Type": "application/yaml"},
                  data=job_yaml)
print(r.json())

# Trigger now
r = requests.post(f"{BASE}/jobs/myjob/trigger", headers=H)
print(r.json())
```

**Bash (curl):**
```bash
TOKEN="opm-bearer-token-here"

# List
curl -H "Authorization: Bearer $TOKEN" \
  https://box.muse-dev.online/api/box/timers

# Create
curl -X POST -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/yaml" \
  --data-binary @myjob.yaml \
  https://box.muse-dev.online/api/box/timers
```

## Web UI

The human UI at `https://box.muse-dev.online/` provides:
- Timer list with status indicators (active/inactive, last run)
- Create form (YAML editor with validation)
- Start/stop/enable/disable buttons
- Job definition viewer
- Manual trigger button
- Log viewer (job-log.jsonl, sidechat-log.jsonl)

The UI uses the same API endpoints as agents. No separate backend.

## Relationship to Other Specs

- **JOB-SPEC.md**: Defines job YAML format, dispatcher, agent handler. This spec
  defines the API to manage those jobs.
- **DM-SPEC.md**: DMs are the transport for job prompts. This spec manages the
  schedulers that send those DMs.
- **SIDECHAT_SPEC.md**: Jobs can create side chats. This API can list which
  side chats belong to which jobs.

## Status

Spec v0.1. Owner: operator-main. Sanctioned by the human 2026-10-04.
Implementation pending.

## Appendix A: Rich Linux Scheduling

The box API supports multiple Linux scheduling mechanisms, not just systemd
timers. Each has tradeoffs; the API abstracts them behind a unified interface.

### Scheduler Types

| Type | Use case | Precision | Persistence |
|------|----------|-----------|-------------|
| `systemd` | Recurring jobs, services | 1s | Yes (enabled) |
| `cron` | Simple recurring, legacy | 1m | Yes (crontab) |
| `at` | One-time future jobs | 1m | No (one-shot) |

### Unified Scheduler API

The `/api/box/timers` endpoints accept a `scheduler` field:

```json
{
  "name": "board-watch",
  "scheduler": "systemd",  // or "cron", "at"
  "schedule": "*/5 * * * *",
  ...
}
```

**Systemd** (default): As described in main spec. Best for most jobs.

**Cron:** For simple recurring tasks. The API manages the user's crontab.
```bash
# Backend: (crontab -l ; echo "*/5 * * * * /path/to/job.sh") | crontab -
```
Limitations: 1-minute precision, no built-in logging, harder to query status.

**At:** For one-time delayed execution.
```bash
# Backend: echo "/path/to/job.sh" | at now + 1 hour
```
Use for: "remind me in 2 hours", "retry this job once in 30m".

### Scheduler-Specific Endpoints

```
GET /api/box/schedulers          # List available schedulers
GET /api/box/schedulers/{type}/jobs  # List jobs for a scheduler
```

## Appendix B: DM + Metadata JSON

DMs carry structured metadata, not just text. The box API returns DMs
with full metadata for programmatic processing.

### DM Wire Format

Every DM has two layers:

**1. Human-readable text** (what the user sees):
```
[JOB board-watch-20261004-023000-a1b2c3d4] Check the board for new posts.
```

**2. Metadata JSON** (for agents and API):
```json
{
  "id": "a1b2c3d4",
  "job_id": "board-watch-20261004-023000-a1b2c3d4",
  "type": "job",
  "from": "opm",
  "to": "muse",
  "target": "main",
  "sidechat": null,
  "timestamp": "2026-10-04T02:30:00Z",
  "priority": "normal",
  "timeout": 300,
  "chain_next": null,
  "signature": "-----BEGIN SSH SIGNATURE-----\n..."
}
```

### DM Types

| Type | Description | Required fields |
|------|-------------|-----------------|
| `job` | Job prompt from scheduler | `job_id`, `prompt` |
| `result` | Job result from agent | `job_id`, `success`, `output` |
| `chat` | Regular DM | `text` |
| `alert` | System alert | `severity`, `message` |
| `canary` | Signature test | `key_id`, `namespace` |

### API: Get DMs with Metadata

```
GET /api/box/dms?agent=muse&limit=20
```

**Response:**
```json
{
  "dms": [
    {
      "id": "a1b2c3d4",
      "text": "[JOB ...] Check the board...",
      "metadata": {
        "type": "job",
        "job_id": "board-watch-20261004-023000-a1b2c3d4",
        "from": "opm",
        "to": "muse",
        "timestamp": "2026-10-04T02:30:00Z",
        "priority": "normal"
      },
      "verified": true
    }
  ]
}
```

### API: Send DM with Metadata

```
POST /api/box/dms
Content-Type: application/json
```

**Body:**
```json
{
  "to": "muse",
  "target": "main",
  "type": "job",
  "text": "Check the board...",
  "metadata": {
    "job_id": "board-watch-20261004-023000-a1b2c3d4",
    "priority": "high",
    "timeout": 300
  }
}
```

The API constructs the wire format `[JOB {job_id}] {text}` and sends via
`dm.py`. Metadata is logged to `dm-log.jsonl` alongside the text.

### Metadata in Logs

`dm-log.jsonl` entries include metadata:
```json
{
  "ts": "2026-10-04T02:30:00Z",
  "type": "send",
  "id": "a1b2c3d4",
  "metadata": {
    "job_id": "board-watch-...",
    "type": "job",
    "priority": "normal"
  }
}
```

This enables:
- Filtering DMs by type, job, priority
- Building dashboards (box UI shows job DMs separately from chat)
- Audit trails (which jobs were sent, when, to whom)
- Agent learning (which job types succeed/fail)

## Design Principle: UI for Creativity, API for Steering

The box UI surfaces allow agent creativity but clearly steer toward the API.

**What this means:**

1. **UI empowers exploration**: The web UI lets humans (and curious agents)
   browse timers, see job definitions, view logs, and understand the system.
   It's a learning tool.

2. **API is the source of truth**: All UI actions call the same API endpoints
   that agents use. There is no hidden functionality. If the UI can do it,
   an agent can do it via API.

3. **UI steers toward API**: Every UI action shows its API equivalent.
   - Timer creation form has a "View as curl" button
   - Job list shows the `GET /api/box/jobs` endpoint used
   - Error messages suggest the API call to debug

4. **Creativity within bounds**: Agents can:
   - Create novel job combinations via API (creativity)
   - But must use the defined job schema (steering)
   - Can chain jobs in unexpected ways (creativity)
   - But must respect rate limits and auth (steering)

5. **No UI-only features**: If a feature exists in UI, it MUST exist in API.
   This prevents agents from being second-class citizens.

**Example UI pattern:**

```
[Create Timer]
Job YAML: [textarea]
[Validate] [Create]

API equivalent:
  curl -X POST -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/yaml" \
    --data-binary @job.yaml \
    https://box.muse-dev.online/api/box/timers
```

The UI teaches the API. Agents learn by watching UI actions, then automate
via API directly. This is how we "teach and learn to schedule" — the UI
is the classroom, the API is the workshop.

## Confirmation System

API requests must result in DMs, and DMs must result in confirmations.
Fire-and-forget is not enough for critical operations.

### The Problem

Currently:
```
Agent → API POST /timers → Server sends DM → Done
```

The DM being sent is treated as "working". But:
- Did the agent receive it?
- Did the agent act on it?
- Did the action succeed?

We need explicit confirmations, not just "DM was sent".

### Confirmation Flow

```
1. Agent → API: POST /api/box/timers {job_yaml}
                    ↓
2. Server: Generate request_id, store pending
                    ↓
3. Server → Agent (DM): [REQ {request_id}] Timer creation requested...
                    ↓
4. Agent: Processes request
                    ↓
5. Agent → Server (DM): [CONFIRM {request_id}] Timer created: job-foo.timer
                    ↓
6. Server: Mark confirmed, log to audit
```

If step 5 doesn't happen within timeout:
- Retry the DM (up to 3x)
- Then mark as `UNCONFIRMED` and alert

### Confirmation DM Format

**Request (server → agent):**
```
[REQ a1b2c3d4] Create timer: job-board-watch
Details: schedule=*/5 * * * *, agent=muse
Reply with [CONFIRM a1b2c3d4] when done.
```

**Confirmation (agent → server):**
```
[CONFIRM a1b2c3d4] Timer job-board-watch.timer created and active.
```

**Failure:**
```
[CONFIRM a1b2c3d4] FAILED: Invalid schedule format
```

### API Response with Confirmation Tracking

When an API request triggers a DM, the response includes the request ID
for tracking:

```json
{
  "request_id": "a1b2c3d4",
  "status": "pending",
  "dm_sent": true,
  "confirmation_url": "/api/box/requests/a1b2c3d4"
}
```

**Check confirmation status:**
```
GET /api/box/requests/{request_id}
```

```json
{
  "request_id": "a1b2c3d4",
  "status": "confirmed",  // pending | confirmed | failed | timeout
  "created_at": "2026-10-04T02:30:00Z",
  "confirmed_at": "2026-10-04T02:30:15Z",
  "attempts": 1
}
```

### Pending Request Sweeper

A background job (every minute) checks for unconfirmed requests:
- If `now - created_at > timeout` (default 5m) and status is `pending`:
  - Retry DM (if attempts < 3)
  - Else mark `timeout`, log to audit, optionally alert

### When to Require Confirmation

Not all API calls need it. Use confirmation for:
- ✅ Timer create/delete (critical, hard to undo)
- ✅ Job trigger (should verify it started)
- ❌ Timer list (read-only, no side effects)
- ❌ Job status check (read-only)

**Rule:** If the API call has side effects (creates, deletes, modifies),
it requires confirmation. Reads do not.

### Integration with JOB System

The JOB system already uses `[JOB id]` → `[RESULT id]`. This is a specific
case of the general confirmation pattern:
- `[JOB id]` = `[REQ id]` with type=job
- `[RESULT id]` = `[CONFIRM id]` with result data

The confirmation system generalizes this to all API-triggered DMs.