Hybrid Headless Gateway & Chromebox Architecture: `muse-cli` Adaptation

Overview

NetVM previously automated agent interaction (thread reading, message sends, job execution) solely via headless Chromebox containers over Chrome DevTools Protocol (CDP) and DOM manipulation.

While browser automation is indispensable for interactive UI flows (initial authentication, OTP submission, age verification, visual inspection), routine agent messaging and thread polling over DOM mutation suffered from high latency and UI placement flakiness (placement_failed, NOCOMPOSE, NOINPUT).

To resolve this, we adapted muse-cli into the NetVM box (super-cli.py) ecosystem as a fast, headless gateway transport connecting directly over WebSockets (wss://gateway.muse.ai/v1/noise) via encrypted Noise protocol frames (Noise_XX_25519_AESGCM_SHA256).

1. Strict Cloudflare Identity & Egress Isolation

A core architectural invariant of NetVM is that no fleet node shares an unisolated egress network identity or ambient host IP. Every command executed via the CLI maintains strict network and session separation.

Network Namespace Boundary

Each node possesses a dedicated Linux network namespace (warp-<node>) containing its own WireGuard interface (wb-<tag>), routing all traffic through its assigned Cloudflare WARP client identity:

warp-muse (Interface: wb-4016c3db)
warp-pip (Interface: wb-fed5038b)
warp-646 (Interface: wb-ed0b853b)
warp-opm (Interface: wb-54353f9c)

Execution Wrapper: `bin/muse-cli-node`

The executable wrapper bin/muse-cli-node bridges CLI requests into the agent's isolated namespace:

bin/muse-cli-node <node> <subcommand> [args...]

Network Egress: Dispatches execution through bin/netvm-exec.sh, ensuring all WebSocket and HTTP requests originate from the node's specific WARP interface.
Session Cookie Partitioning: Uses isolated cookie stores located in ~/.config/muse-cli/<node>/cookies.txt (permissions 0600).
Auto-Healing Cookie Refresh: On receiving an AuthError (or 401 Unauthorized), muse-cli-node intercepts the error, runs bin/refresh-node-cookies.py to export fresh session cookies directly from the node's local Chromium instance via CDP inside its netns, and transparently retries the command once.

2. Hybrid Modality Architecture

NetVM leverages both modalities according to their operational strengths:

Task / Domain	Primary Modality	Fallback / Complementary Modality
Routine Thread Listing	Gateway (`muse_hybrid.get_threads`)	DOM CDP (`box-chat.py thread-list`)
Thread History Reading	Gateway (`muse_hybrid.get_history`)	DOM CDP (`box-chat.py thread-messages`)
High-Frequency Job/DM Dispatch	Gateway (`muse-cli send`)	DOM CDP (`muse-chat-api.py send`)
Live Event Streaming	Gateway (`muse-cli watch`)	Siphon log polling (`siphon-bl.py`)
Initial Login & OTP Entry	DOM CDP (`bin/muse-signin.py`)	N/A (Requires interactive DOM inputs)
Visual Screencast & Audit	DOM CDP (`Page.startScreencast`)	N/A

3. Programmatic & CLI Interfaces

Python Interface: `bin/muse_hybrid.py`

Provides clean programmatic access for scripts and background daemons:

import muse_hybrid

# Fetch thread listing (sub-second)
threads, err = muse_hybrid.get_threads("pip")

# Fetch message history
msgs, err = muse_hybrid.get_history("pip", thread_id="4466d0c1-7961-4cf3-b99d-1ab7c38484c2", limit=5)

# Fast send
res, err = muse_hybrid.send_message("pip", "Hello from gateway", thread_id="4466d0c1-...")

CLI Interface: `super-cli.py` (`box`)

Direct Gateway Passthrough: Operators can execute any muse-cli command under a specific node's identity:
```
box muse pip status
box muse 646 threads
box muse opm watch
```
Accelerated Thread Commands: box thread list <agent> and box thread view <agent> <thread_id> automatically query muse_hybrid first. If the gateway encounters an issue, they gracefully fall back to the existing box-chat.py CDP path.

4. Verification & Health Audit

To verify the hybrid gateway across all fleet nodes:

# Verify status per node
for node in muse pip 646 opm; do
  echo "=== $node ==="
  box muse $node status | jq -r '.identity.name'
done

# Verify thread view via hybrid path
box thread view pip 4466d0c1-7961-4cf3-b99d-1ab7c38484c2 --limit 3

4.6 KiB Raw Blame History

Hybrid Headless Gateway & Chromebox Architecture: muse-cli Adaptation