99 lines
4.4 KiB
Markdown
99 lines
4.4 KiB
Markdown
# NetVM
|
|
|
|
Fleet networking layer. Every node gets a stable network identity; every
|
|
byte of automation traffic is attributable, consistent, and boring — the
|
|
way good citizens look to the rest of the internet.
|
|
|
|
## Scope
|
|
|
|
1. **Per-node egress identity** — one Warp-backed egress per chrome-box /
|
|
node, in its own network namespace. Stable egress IP per node, explicit
|
|
mapping, isolated blast radius.
|
|
2. **Tailscale fabric** — node addressing, operator SSH, human reachability.
|
|
3. **Human-handoff links** — the reachable path a human uses to complete
|
|
CAPTCHA/2FA on a live browser session (original scope, kept).
|
|
|
|
NetVM owns networking. chrome-box owns browsers and consumes network from
|
|
NetVM. email-alert owns notifications. The Account(s) hub owns identity
|
|
records. Secrets and identities are the human's — agents manage structure
|
|
and lifecycle only, never credentials.
|
|
|
|
## Why per-node egress
|
|
|
|
- **Anti-fraud, not evasion.** Providers flag IP-hopping as suspicious. A
|
|
node that always egresses from the same place looks like what it is: a
|
|
legitimate machine. Standardizing on Warp is the good-citizen move.
|
|
- **Blast-radius isolation.** One IP reputation problem affects one node,
|
|
never the fleet. Nodes deliberately do not all share one egress.
|
|
- **DevOps standardization.** Every node is provisioned the same way:
|
|
netns + WireGuard-via-Warp + topology entry. No snowflakes.
|
|
|
|
Nodes may share an egress identity deliberately (documented in NODES.md);
|
|
sharing is a designed topology choice, never an accident.
|
|
|
|
## Architecture
|
|
|
|
chrome-box profile (client X)
|
|
-> launched inside netns warp-clientX (NetVM provides the launcher)
|
|
-> wg interface (Warp WireGuard params, human-generated config)
|
|
-> Cloudflare edge, stable colo (egress IP in NODES.md)
|
|
-> internet
|
|
|
|
operator / human
|
|
-> Tailscale tailnet (100.x) (management + Waypipe handoff)
|
|
-> node tail IP
|
|
|
|
Warp runs at the network layer, so the browser needs no proxy config —
|
|
the whole namespace egresses through Warp.
|
|
|
|
### Pattern decision
|
|
|
|
Two ways to get per-node Warp egress were considered:
|
|
|
|
- **warp-cli proxy mode** (mode proxy + SOCKS5 per node): simpler, but
|
|
warp-cli talks to a single system daemon (warp-svc) — running N daemons
|
|
on one host is unverified and fights the service model.
|
|
- **WireGuard in netns (chosen)**: Warp is WireGuard under the hood. One
|
|
wg interface per node namespace, config generated once by the human
|
|
(wgcf or equivalent), no daemon, no D-Bus, fully scriptable. One
|
|
interface per chrome-box, each independently up/down-able.
|
|
|
|
Upgrade path if pool IPs prove too fluid: Cloudflare Zero Trust dedicated
|
|
egress (true static IPs, paid).
|
|
|
|
## Provisioning a node
|
|
|
|
Identity creation is the human's job; lifecycle is scriptable:
|
|
|
|
1. Human: run `bin/netvm-new-identity.sh <node>` ON the node — it
|
|
registers the Warp identity and installs /etc/netvm/<node>.conf
|
|
(root-owned, 0600). This file is a credential — agents never create,
|
|
read, or copy it, and the script is excluded from the operator sudoers
|
|
allowlist.
|
|
2. sudo bin/netvm-node-up.sh <node> — creates netns warp-<node>, raises
|
|
the wg interface inside it, verifies egress, prints the result.
|
|
3. chrome-box launches the client's Chromium inside that netns.
|
|
|
|
Tear down: sudo bin/netvm-node-down.sh <node>.
|
|
|
|
## Files
|
|
|
|
- bin/netvm-verify.sh — prerequisite checks (ip, wg, netns, tailscale…).
|
|
- bin/netvm-node-up.sh <node> — bring up a node's egress.
|
|
- bin/netvm-node-down.sh <node> — tear a node's egress down.
|
|
- bin/netvm-topology.sh — print the live topology table.
|
|
- `bin/netvm-provision-edge.sh` — prepare an edge device (sudoers allowlist, deps, /etc/netvm); run on the node, once.
|
|
- `bin/netvm-fleet.sh` — operator fleet control over the tailnet (topology/up/down/ssh per node).
|
|
- NODES.md — the registry: node -> netns -> Warp identity -> egress IP.
|
|
|
|
## Verification checklist
|
|
|
|
- [ ] netvm-verify.sh passes on the target host.
|
|
- [ ] Per-node wg interface comes up inside its netns; egress IP differs
|
|
per node (or matches the designed sharing in NODES.md).
|
|
- [ ] Egress IP per node stable over 7 days (same colo pool).
|
|
- [ ] Chromium launched in the netns reaches the internet via the node's
|
|
egress (ip netns exec warp-<node> curl ifconfig.me).
|
|
- [ ] Waypipe handoff to a node over the tailnet shows a live browser.
|
|
- [ ] One node's egress killed -> other nodes unaffected (blast radius).
|