3 Commits

Author SHA1 Message Date
opm ce1b8def77 fix(approvals): harden target regex against scheme truncation
The target-extraction regex stopped at ':' and '/', so a dialog
containing a full URL (e.g. 'http://verify.muse-dev.online/api/...')
was captured as just the scheme ('http'). Add an optional
'(?:https?://)?' prefix so the hostname is captured instead.

This is defense-in-depth: the 2026-10-11 bare-scheme dialogs render
'http' directly from the app (closed source), but if the app resumes
rendering full URLs, the extractor will now yield the hostname rather
than the scheme.

Verified against 5 cases: bare 'http' unchanged, full http/https URLs
now yield hostnames, IPs and single-label hosts unchanged.
2026-10-11 20:06:32 -04:00
super 78d22bd950 Merge pull request #212 from dev/opm/211-container-ssh-recovery
feat(ssh): add node SSH dial-in verification script

Fixes #211
2026-10-09 22:43:09 +00:00
opm c0113c1ebf feat(ssh): add node SSH dial-in verification script
Adds bin/verify-node-ssh.sh: checks reverse-tunnel listeners and SSH
auth for each fleet node port from the VM. Distinguishes dark nodes
(no listener) from auth failures (authorized_keys perms/keys).

Verification 2026-10-09:
- muse/2225, 646/2226, pip/2227, muse-main/2224, opm/2228: LISTEN
- def/2229, dev/2230: DARK (no reverse tunnel)
- All listening nodes reject VM super key (expected: nodes authorize
  per-operator/id_frontdoor keys, not the VM super key)

Fixes #211
2026-10-09 21:56:08 +00:00
2 changed files with 61 additions and 1 deletions
+1 -1
View File
@@ -735,7 +735,7 @@ def inspect_node_approvals(node: str) -> dict:
ip = None
target = None
m_t = re.search(
r"(?:connection to|share information with|contact|connect to)\s+([A-Za-z0-9][A-Za-z0-9.-]*[A-Za-z0-9])",
r"(?:connection to|share information with|contact|connect to)\s+(?:https?://)?([A-Za-z0-9][A-Za-z0-9.-]*[A-Za-z0-9])",
card_text,
)
if m_t:
+60
View File
@@ -0,0 +1,60 @@
#!/bin/bash
# verify-node-ssh.sh — verify container SSH dial-in readiness across fleet nodes.
# Checks from the VM: reverse-tunnel listeners + SSH auth for each node port.
#
# Port map (docs/OPERATOR-DRIVE-RUNBOOK.md):
# muse-main 2224 | muse 2225 | 646 2226 | pip 2227 | opm 2228 | def 2229 | dev 2230
#
# What it checks per node:
# 1. Reverse-tunnel listener on 127.0.0.1:<port> (dark node = no listener)
# 2. SSH dial-in with BatchMode (auth failure = authorized_keys perms/key issue)
#
# Common root causes (see #211):
# - sshd requires non-group-writable authorized_keys (must be 600)
# - stale /run/nologin blocks logins
# - missing id_frontdoor keys on dark nodes
#
# Usage: run on the VM (super@34.139.37.135), or via:
# ssh-vm.sh "bash -s" < verify-node-ssh.sh
set -u
# node:port pairs to check
NODES="muse:2225 646:2226 pip:2227 def:2229 dev:2230 muse-main:2224 opm:2228"
fail=0
for pair in $NODES; do
node="${pair%%:*}"
port="${pair##*:}"
# 1. listener check
if ss -tln 2>/dev/null | grep -q "127.0.0.1:${port} "; then
listener="LISTEN"
else
listener="DARK (no listener)"
fi
# 2. auth check (only if listening)
if [ "$listener" = "LISTEN" ]; then
out=$(timeout 15 ssh -o StrictHostKeyChecking=no -o BatchMode=yes \
-o ConnectTimeout=10 -p "$port" hatch@127.0.0.1 'echo OK' 2>&1)
case "$out" in
OK) auth="OK" ;;
*"Permission denied"*) auth="AUTH-FAIL (check authorized_keys perms/keys)" ;;
*"Connection refused"*) auth="REFUSED (tunnel died after listen check)" ;;
*) auth="OTHER: $(echo "$out" | head -1 | cut -c1-60)" ;;
esac
else
auth="SKIP"
fi
printf '%-10s port %-5s listener: %-22s auth: %s\n' "$node" "$port" "$listener" "$auth"
[ "$listener" = "DARK (no listener)" ] && fail=1
case "$auth" in AUTH-FAIL*) fail=1 ;; esac
done
if [ "$fail" -eq 0 ]; then
echo "ALL NODES REACHABLE"
else
echo "ISSUES FOUND (see above)"
fi
exit "$fail"