box/docs/RETENTION-ROTATIONS-P2.md

Retention Piece 2 — Job Definition Archival & Pruning

Owner: operator-646 Branch: dev/operator-646/retention-rotations-p1 Date: 2026-10-07

Extends the fleet retention framework from Piece 1 (RETENTION-ROTATIONS-P1.md) to prune and archive retired manual job definitions from jobs/ into jobs/archive/.

Problem

The root jobs/ directory accumulated over 235 job definition files, primarily composed of retired auto-work-* exploratory batches and one-off test pipelines. Every invocation of job-scheduler.py (glob.glob("jobs/*.json")) and CLI tools was parsing all 235 files, causing unnecessary filesystem overhead and clutter.

Policy Implemented

Domain	Eligibility Criteria	Action	Target Layout
jobs/*.json	schedule == "manual" or empty, AND not protected	git mv archive	jobs/archive/<name>.json (plain JSON, preserves history)

Dynamic Safety Guardrails

A job is never archived automatically if any of the following hold:

1. Systemd User Units: Registered as an ExecStart target in ~/.config/systemd/user/job-*.service (e.g. work-finder, heartbeat, muse-auditor, opm-swarm-harvest, autonomy-pulse-*).
2. Active Pipelines: Associated with a running, dispatched, or pending run in pipelines.json (e.g. ops-audit-*, pipe-demo-*).
3. Chained Dependencies: Linked via chain_next, on_success, or on_failure from any active protected job.
4. Core Fleet Baseline: Hardcoded protective baseline: heartbeat, refine-system, canary-test.

Tooling & Architecture

• bin/retention-archive-jobs.py: Standalone CLI driver supporting scan, archive, unarchive, and list with --dry-run, --force, and --json.
• bin/retention-run-rotations.sh: Hourly rotation driver now runs retention-archive-jobs.py scan as the 4th phase of the retention pipeline.
• bin/box-ctl.py:
  • job-list: Excludes archived jobs by default; includes them when passed --archived / -a.
  • job-archive <name>: Archives a job via retention-archive-jobs.py.
  • job-unarchive <name>: Restores an archived job to active jobs/.
• bin/super-cli.py: Exposes box job archive <name>, box job unarchive <name>, and box job list --archived.
• bin/job-dispatch.py: Guards against direct dispatch of archived jobs; fails fast with an explicit unarchive prompt.

Operations & Verification

Run on-demand scan:

bin/retention-archive-jobs.py scan [--dry-run]

Manual archive / unarchive:

box job archive <name>
box job unarchive <name>

List archived jobs:

box job list --archived

Execute full retention cycle:

bin/retention-run-rotations.sh

First Run Results (2026-10-07)

• Total jobs scanned: 235
• Dynamically protected jobs: 13
• Eligible retired jobs identified: 17
• Jobs archived to jobs/archive/: 646-opm-watch, 646-pip-sync, 646-sidechat-task, auto-work-queue-f02, auto-work-queue-f06, auto-work-queue-f10, auto-work-queue-f14, auto-work-queue-f18, auto-work-xop-e02, auto-work-xop-e06, auto-work-xop-e10, auto-work-xop-e14, auto-work-xop-e18, mainloop-p1-pilot, mainloop-p2-noswitcher, mainloop-p3-bridge, mainloop-p4-steady.
• Committed in git as: fff5556 (chore(retention): archive retired jobs [...]).
• Active jobs remaining: 218.
• Unit tests: 10 tests in tests/test_retention_archive_jobs.py (all green).