feat: setup-fed watchdog supervision for all registry nodes

Close the def/dev supervision gap at the source: every node brought
up gets watched, and every supervisor enumerates the registry.

- bin/ensure-node-supervision.sh (new, idempotent): appends the
  NODES.md row (netvm-names port, honors CDP_PORT_OVERRIDE so it
  never fights provision's picker) and installs/enables
  chromebox-watchdog-<node>.timer. --all heals drift (registry +
  /etc/netvm identities). Template verified byte-identical to the
  installed def unit.
- netvm-node-up.sh: calls ensure (non-fatal) at the end. Provision
  and the onboarding pipeline reach it transitively.
- relay-health-check.sh, cdp-latency-check.sh: registry-driven
  watched_nodes() + LIB_ONLY guards (were hardcoded 4 nodes).
- tests/test_node_supervision.py (6): row add/idempotent/override,
  timer render, node-up wiring, both watched_nodes().
- CHROMEBOX-RUNBOOK.md: setup-fed supervision section.

Pairs with the registry-driven relay/chromebox watchdogs: new rows
are picked up on the next run with no per-node code edits.
This commit is contained in:
Muse Sidechat
2026-10-06 19:28:29 +00:00
parent c9143a558b
commit 66c8900a58
6 changed files with 249 additions and 3 deletions
+19 -1
View File
@@ -17,7 +17,25 @@ source "$BIN_DIR/netvm-names.sh"
TIMEOUT_S=10
for node in muse pip 646 opm; do
# Registry-driven node list (was hardcoded 4 nodes; def/dev had no
# cdp-latency coverage — 2026-10-06).
watched_nodes() {
"$BIN_DIR/netvm-registry.py" 2>/dev/null | cut -d: -f1
}
# Allow sourcing for tests without running checks.
if [ "${CDP_LATENCY_CHECK_LIB_ONLY:-}" = "1" ]; then
return 0 2>/dev/null || exit 0
fi
NODES="$(watched_nodes)"
if [ -z "$NODES" ]; then
echo "node registry empty/unreadable" >&2
exit 1
fi
# shellcheck disable=SC2086 (intended word splitting: one node per word)
for node in $NODES; do
netvm_names "$node"
url="http://${PEER_IP}:${CDP_PORT}/json/version"
probe=$(curl -s -m "$TIMEOUT_S" -o /dev/null -w "%{time_total} %{http_code}" "$url" 2>/dev/null)