# Retention Piece 2 — Job Definition Archival & Pruning

Owner: operator-646
Branch: `dev/operator-646/retention-rotations-p1`
Date: 2026-10-07

Extends the fleet retention framework from Piece 1 ([RETENTION-ROTATIONS-P1.md](file:///home/super/Projects/NetVM/docs/RETENTION-ROTATIONS-P1.md)) to prune and archive retired manual job definitions from `jobs/` into `jobs/archive/`.

## Problem

The root `jobs/` directory accumulated over 235 job definition files, primarily composed of retired `auto-work-*` exploratory batches and one-off test pipelines. Every invocation of `job-scheduler.py` (`glob.glob("jobs/*.json")`) and CLI tools was parsing all 235 files, causing unnecessary filesystem overhead and clutter.

## Policy Implemented

| Domain | Eligibility Criteria | Action | Target Layout |
|:---|:---|:---|:---|
| `jobs/*.json` | `schedule == "manual"` or empty, AND not protected | `git mv` archive | `jobs/archive/<name>.json` (plain JSON, preserves history) |

### Dynamic Safety Guardrails

A job is **never** archived automatically if any of the following hold:
1. **Systemd User Units**: Registered as an `ExecStart` target in `~/.config/systemd/user/job-*.service` (e.g. `work-finder`, `heartbeat`, `muse-auditor`, `opm-swarm-harvest`, `autonomy-pulse-*`).
2. **Active Pipelines**: Associated with a running, dispatched, or pending run in `pipelines.json` (e.g. `ops-audit-*`, `pipe-demo-*`).
3. **Chained Dependencies**: Linked via `chain_next`, `on_success`, or `on_failure` from any active protected job.
4. **Core Fleet Baseline**: Hardcoded protective baseline: `heartbeat`, `refine-system`, `canary-test`.

## Tooling & Architecture

- `bin/retention-archive-jobs.py`: Standalone CLI driver supporting `scan`, `archive`, `unarchive`, and `list` with `--dry-run`, `--force`, and `--json`.
- `bin/retention-run-rotations.sh`: Hourly rotation driver now runs `retention-archive-jobs.py scan` as the 4th phase of the retention pipeline.
- `bin/box-ctl.py`:
  - `job-list`: Excludes archived jobs by default; includes them when passed `--archived` / `-a`.
  - `job-archive <name>`: Archives a job via `retention-archive-jobs.py`.
  - `job-unarchive <name>`: Restores an archived job to active `jobs/`.
- `bin/super-cli.py`: Exposes `box job archive <name>`, `box job unarchive <name>`, and `box job list --archived`.
- `bin/job-dispatch.py`: Guards against direct dispatch of archived jobs; fails fast with an explicit unarchive prompt.

## Operations & Verification

Run on-demand scan:
```bash
bin/retention-archive-jobs.py scan [--dry-run]
```

Manual archive / unarchive:
```bash
box job archive <name>
box job unarchive <name>
```

List archived jobs:
```bash
box job list --archived
```

Execute full retention cycle:
```bash
bin/retention-run-rotations.sh
```

## First Run Results (2026-10-07)

- Total jobs scanned: 235
- Dynamically protected jobs: 13
- Eligible retired jobs identified: 17
- Jobs archived to `jobs/archive/`:
  `646-opm-watch`, `646-pip-sync`, `646-sidechat-task`, `auto-work-queue-f02`, `auto-work-queue-f06`, `auto-work-queue-f10`, `auto-work-queue-f14`, `auto-work-queue-f18`, `auto-work-xop-e02`, `auto-work-xop-e06`, `auto-work-xop-e10`, `auto-work-xop-e14`, `auto-work-xop-e18`, `mainloop-p1-pilot`, `mainloop-p2-noswitcher`, `mainloop-p3-bridge`, `mainloop-p4-steady`.
- Committed in git as: `fff5556` (`chore(retention): archive retired jobs [...]`).
- Active jobs remaining: 218.
- Unit tests: 10 tests in `tests/test_retention_archive_jobs.py` (all green).