super/box - box - Box Build Platform (Tea)

super/box

Fork 0

Commit Graph

Author	SHA1	Message	Date
operator-main	a32670731f	chromebox-watchdog: fix kill loop on slow cold starts Two fixes: (1) retry health check 4x with 15s gaps after relaunch instead of single 25s check; (2) skip kill if browser launched <2min ago (probably still starting). Prevents watchdog from killing a working-but-slow browser. Session: sidechat/chromebox-ops	2026-10-04 13:17:13 +00:00
operator-main	343920e0c3	Watchdog upgrades: stage-specific logging + new CDP relay watchdog chromebox-watchdog.sh: HEALTH_FAIL_REASON pinpoints which health stage failed (no process / CDP unreachable / no Chat page); chromium stdout redirected to per-profile chromebox-<profile>.log; 10MB log rotation (one generation); Chat title match relaxed to .*Chat. bin/cdp-relay-watchdog.sh (new): keeps per-node CDP relays alive. Two-stage check: (1) host veth IP assigned (fail-loud, no auto-fix — veth recreation touches WireGuard/iptables), (2) relay connectivity via curl to veth IP:port (never trust pidfiles — observed stale 2026-10-04). Restarts dead/misrouted relays in-netns. Runs via systemd timer every 5min. Pattern mirrors chromebox-watchdog.sh. Session: sidechat/chromebox-ops	2026-10-04 12:40:56 +00:00
operator-main	f19aff9b79	Fix supervisor SIGKILL loop killing restarted browsers (opm/pip) Root cause: agent-health.service runs Type=oneshot with the default KillMode=control-group. restart_browser() spawned the replacement chromium with nohup under the service, so systemd SIGKILLed it the moment the service exited. Every 5-min tick: FAIL -> restart -> RECOVERED -> SIGKILL at teardown. opm and pip were permanently dark and dm.py read masked it (empty output, exit 0). Fix: launch replacements via systemd-run --user --scope (backgrounded) so the browser lives in a transient scope outside the service cgroup and survives teardown. setsid does NOT escape either. Note: systemd-run --scope waits for the scope even with --no-block (verified 2026-10-03), hence the backgrounding. Same fix in chromebox-watchdog.sh (timers currently off). dm.py: dm_read() now uses run_full() and prints a WARNING to stderr with rc + last error line instead of failing silently on empty reads. Trailers: Session: sidechat/opm-blind-fix	2026-10-04 01:16:59 +00:00

Author

SHA1

Message

Date

operator-main

a32670731f

chromebox-watchdog: fix kill loop on slow cold starts

Two fixes: (1) retry health check 4x with 15s gaps after relaunch instead of single 25s check; (2) skip kill if browser launched <2min ago (probably still starting). Prevents watchdog from killing a working-but-slow browser.

Session: sidechat/chromebox-ops

2026-10-04 13:17:13 +00:00

operator-main

343920e0c3

Watchdog upgrades: stage-specific logging + new CDP relay watchdog

chromebox-watchdog.sh: HEALTH_FAIL_REASON pinpoints which health stage failed (no process / CDP unreachable / no Chat page); chromium stdout redirected to per-profile chromebox-<profile>.log; 10MB log rotation (one generation); Chat title match relaxed to .*Chat.

bin/cdp-relay-watchdog.sh (new): keeps per-node CDP relays alive. Two-stage check: (1) host veth IP assigned (fail-loud, no auto-fix — veth recreation touches WireGuard/iptables), (2) relay connectivity via curl to veth IP:port (never trust pidfiles — observed stale 2026-10-04). Restarts dead/misrouted relays in-netns. Runs via systemd timer every 5min. Pattern mirrors chromebox-watchdog.sh.

Session: sidechat/chromebox-ops

2026-10-04 12:40:56 +00:00

operator-main

f19aff9b79

Fix supervisor SIGKILL loop killing restarted browsers (opm/pip)

Root cause: agent-health.service runs Type=oneshot with the default
KillMode=control-group. restart_browser() spawned the replacement
chromium with nohup under the service, so systemd SIGKILLed it the
moment the service exited. Every 5-min tick: FAIL -> restart ->
RECOVERED -> SIGKILL at teardown. opm and pip were permanently dark
and dm.py read masked it (empty output, exit 0).

Fix: launch replacements via systemd-run --user --scope (backgrounded)
so the browser lives in a transient scope outside the service cgroup
and survives teardown. setsid does NOT escape either. Note:
systemd-run --scope waits for the scope even with --no-block
(verified 2026-10-03), hence the backgrounding. Same fix in
chromebox-watchdog.sh (timers currently off).

dm.py: dm_read() now uses run_full() and prints a WARNING to stderr
with rc + last error line instead of failing silently on empty reads.

Trailers: Session: sidechat/opm-blind-fix

2026-10-04 01:16:59 +00:00

3 Commits