operator
97 lines
4.8 KiB
Markdown
97 lines
4.8 KiB
Markdown
# Hybrid Headless Gateway & Chromebox Architecture: `muse-cli` Adaptation
|
|
|
|
> **Box is the main surface.** All operator work goes through Box (box.muse-dev.online). The web UI, `box` CLI, and agents share the same API endpoints. No UI-only powers.
|
|
|
|
## Overview
|
|
|
|
NetVM previously automated agent interaction (thread reading, message sends, job execution) solely via headless Chromebox containers over Chrome DevTools Protocol (CDP) and DOM manipulation.
|
|
|
|
While browser automation is indispensable for interactive UI flows (initial authentication, OTP submission, age verification, visual inspection), routine agent messaging and thread polling over DOM mutation suffered from high latency and UI placement flakiness (`placement_failed`, `NOCOMPOSE`, `NOINPUT`).
|
|
|
|
To resolve this, we adapted **`muse-cli`** into the NetVM `box` (`super-cli.py`) ecosystem as a **fast, headless gateway transport** connecting directly over WebSockets (`wss://gateway.muse.ai/v1/noise`) via encrypted Noise protocol frames (`Noise_XX_25519_AESGCM_SHA256`).
|
|
|
|
---
|
|
|
|
## 1. Strict Cloudflare Identity & Egress Isolation
|
|
|
|
A core architectural invariant of NetVM is that **no fleet node shares an unisolated egress network identity or ambient host IP**. Every command executed via the CLI maintains strict network and session separation.
|
|
|
|
### Network Namespace Boundary
|
|
Each node possesses a dedicated Linux network namespace (`warp-<node>`) containing its own WireGuard interface (`wb-<tag>`), routing all traffic through its assigned Cloudflare WARP client identity:
|
|
- `warp-muse` (Interface: `wb-4016c3db`)
|
|
- `warp-pip` (Interface: `wb-fed5038b`)
|
|
- `warp-646` (Interface: `wb-ed0b853b`)
|
|
- `warp-opm` (Interface: `wb-54353f9c`)
|
|
|
|
### Execution Wrapper: `bin/muse-cli-node`
|
|
The executable wrapper [`bin/muse-cli-node`](file:///home/super/Projects/NetVM/bin/muse-cli-node) bridges CLI requests into the agent's isolated namespace:
|
|
```bash
|
|
bin/muse-cli-node <node> <subcommand> [args...]
|
|
```
|
|
1. **Network Egress**: Dispatches execution through [`bin/netvm-exec.sh`](file:///home/super/Projects/NetVM/bin/netvm-exec.sh), ensuring all WebSocket and HTTP requests originate from the node's specific WARP interface.
|
|
2. **Session Cookie Partitioning**: Uses isolated cookie stores located in `~/.config/muse-cli/<node>/cookies.txt` (permissions `0600`).
|
|
3. **Auto-Healing Cookie Refresh**: On receiving an `AuthError` (or 401 Unauthorized), `muse-cli-node` intercepts the error, runs [`bin/refresh-node-cookies.py`](file:///home/super/Projects/NetVM/bin/refresh-node-cookies.py) to export fresh session cookies directly from the node's local Chromium instance via CDP inside its netns, and transparently retries the command once.
|
|
|
|
---
|
|
|
|
## 2. Hybrid Modality Architecture
|
|
|
|
NetVM leverages both modalities according to their operational strengths:
|
|
|
|
| Task / Domain | Primary Modality | Fallback / Complementary Modality |
|
|
| :--- | :--- | :--- |
|
|
| **Routine Thread Listing** | Gateway (`muse_hybrid.get_threads`) | DOM CDP (`box-chat.py thread-list`) |
|
|
| **Thread History Reading** | Gateway (`muse_hybrid.get_history`) | DOM CDP (`box-chat.py thread-messages`) |
|
|
| **High-Frequency Job/DM Dispatch** | Gateway (`muse-cli send`) | DOM CDP (`muse-chat-api.py send`) |
|
|
| **Live Event Streaming** | Gateway (`muse-cli watch`) | Siphon log polling (`siphon-bl.py`) |
|
|
| **Initial Login & OTP Entry** | DOM CDP ([`bin/muse-signin.py`](file:///home/super/Projects/NetVM/bin/muse-signin.py)) | N/A (Requires interactive DOM inputs) |
|
|
| **Visual Screencast & Audit** | DOM CDP (`Page.startScreencast`) | N/A |
|
|
|
|
---
|
|
|
|
## 3. Programmatic & CLI Interfaces
|
|
|
|
### Python Interface: `bin/muse_hybrid.py`
|
|
Provides clean programmatic access for scripts and background daemons:
|
|
```python
|
|
import muse_hybrid
|
|
|
|
# Fetch thread listing (sub-second)
|
|
threads, err = muse_hybrid.get_threads("pip")
|
|
|
|
# Fetch message history
|
|
msgs, err = muse_hybrid.get_history("pip", thread_id="4466d0c1-7961-4cf3-b99d-1ab7c38484c2", limit=5)
|
|
|
|
# Fast send
|
|
res, err = muse_hybrid.send_message("pip", "Hello from gateway", thread_id="4466d0c1-...")
|
|
```
|
|
|
|
### CLI Interface: `super-cli.py` (`box`)
|
|
|
|
1. **Direct Gateway Passthrough**:
|
|
Operators can execute any `muse-cli` command under a specific node's identity:
|
|
```bash
|
|
box muse pip status
|
|
box muse 646 threads
|
|
box muse opm watch
|
|
```
|
|
|
|
2. **Accelerated Thread Commands**:
|
|
`box thread list <agent>` and `box thread view <agent> <thread_id>` automatically query `muse_hybrid` first. If the gateway encounters an issue, they gracefully fall back to the existing `box-chat.py` CDP path.
|
|
|
|
---
|
|
|
|
## 4. Verification & Health Audit
|
|
|
|
To verify the hybrid gateway across all fleet nodes:
|
|
```bash
|
|
# Verify status per node
|
|
for node in muse pip 646 opm; do
|
|
echo "=== $node ==="
|
|
box muse $node status | jq -r '.identity.name'
|
|
done
|
|
|
|
# Verify thread view via hybrid path
|
|
box thread view pip 4466d0c1-7961-4cf3-b99d-1ab7c38484c2 --limit 3
|
|
```
|