diff --git a/docs/DOM-PAGE-STRUCTURE.md b/docs/DOM-PAGE-STRUCTURE.md new file mode 100644 index 0000000..4a6a0e0 --- /dev/null +++ b/docs/DOM-PAGE-STRUCTURE.md @@ -0,0 +1,538 @@ +# muse.ai DOM Page Structure Reference + +**Generated:** 2026-10-04 via live CDP inspection (opm browser, warp-opm netns, CDP 9440) +**Purpose:** Reference for future automation development. All selectors verified against live DOM. + +--- + +## 1. Page States / Modes + +The muse.ai SPA has distinct DOM states. **Detect state before acting** — selectors valid in one state are absent in another. + +### 1.1 Main Chat + +| Property | Value | +|---|---| +| URL | `https://muse.ai/` | +| Title | `Chat — ` (e.g., `Chat — operator-main`) | +| Switcher `[data-testid="hatch-chat-switcher-trigger"]` | ✅ Present | +| Compose `[data-testid="hatch-chat-compose"]` | ❌ Absent (panel closed) | +| Panel content `[data-testid="hatch-side-chats-panel-content"]` | ❌ Absent (panel closed) | +| Panel shell `[data-testid="hatch-side-chats-panel-shell"]` | ✅ Present (empty/collapsed) | +| Message input `[aria-label="Message"]` | ✅ Present, visible | + +**Detection:** +```javascript +// Reliable main-chat detection +const isMainChat = + window.location.href === 'https://muse.ai/' && + document.title.startsWith('Chat —') && + !!document.querySelector('[data-testid="hatch-chat-switcher-trigger"]'); +``` + +### 1.2 Panel Open (Chat Navigation Panel Visible) + +Reached by clicking the switcher on main chat. This is the **required precondition** for sidechat creation. + +| Property | Value | +|---|---| +| URL | `https://muse.ai/` (unchanged) | +| Title | `Chat — ` (unchanged) | +| Switcher | ✅ Present | +| Compose `[data-testid="hatch-chat-compose"]` | ✅ Present **and visible** | +| Panel content | ✅ Present | +| Thread rows `[data-testid="hatch-thread-row"]` | ✅ Present (26 rows observed) | + +**Detection:** +```javascript +// Panel is open and ready for sidechat creation +const isPanelOpen = + !!document.querySelector('[data-testid="hatch-side-chats-panel-content"]') && + !!document.querySelector('[data-testid="hatch-chat-compose"]'); + +// Verify compose button is actually clickable +const composeBtn = document.querySelector('[data-testid="hatch-chat-compose"]'); +const isClickable = composeBtn && composeBtn.getBoundingClientRect().width > 0; +``` + +**Reliable open sequence:** +```javascript +// 1. Ensure on main chat (Ctrl+J via CDP Input.dispatchKeyEvent) +// 2. Check if panel already open +if (!document.querySelector('[data-testid="hatch-chat-compose"]')) { + // 3. Click switcher to open panel + document.querySelector('[data-testid="hatch-chat-switcher-trigger"]').click(); + // 4. Wait 2s, re-check for compose button +} +``` + +### 1.3 `/thread/new` (Fresh Sidechat / Poisoned State) + +**This is the "poisoned" state.** After creating a sidechat, the browser lands here with a **stripped DOM** that lacks the switcher and compose button. The next sidechat create will fail unless you navigate back to main first. + +| Property | Value | +|---|---| +| URL | `https://muse.ai/thread/new` | +| Title | `Muse` (just "Muse", **not** "Chat — ...") | +| Switcher | ❌ **Absent** | +| Compose | ❌ **Absent** | +| Panel shell | ✅ Present but stripped (no panel-content) | +| Message input | ✅ Present | +| Button count | ~45 (vs ~21 on partially-loaded states) | + +**Detection:** +```javascript +const isThreadNew = + window.location.href.includes('/thread/new') || + (document.title === 'Muse' && + !document.querySelector('[data-testid="hatch-chat-switcher-trigger"]')); +``` + +**Recovery:** Send Ctrl+J via CDP to return to main chat, then open panel via switcher click. + +**Distinct testid set** (no chat-switcher testids, dock-rail based): +``` +hatch-dock-rail, hatch-dock-rail-divider, hatch-nav-chat, +hatch-dock-chat-notification-badge, hatch-nav-search, +hatch-nav-system-item-*, app-download-dock-mark, hatch-dock-more, +hatch-side-chats-open-drag-handle, hatch-side-chats-panel-shell, +hatch-chat-nav-fade, hatch-chat-leading-controls, +hatch-status-panel-sliding-surface, hatch-composer-placeholder-overlay, +hatch-browser-task-banner-host +``` + +### 1.4 Sidechat Thread (`/thread/`) + +A real sidechat with a UUID. DOM structure is similar to main chat (switcher present, panel available). Thread rows in the panel show the sidechat title. + +**Note:** After sending the first message in a `/thread/new` chat, the URL transitions to `/thread/`. Poll `window.location.href` after send to capture the real thread ID. + +### 1.5 Landing Page (Not Logged In) + +| Property | Value | +|---|---| +| Title | `muse.ai` (just the domain) | +| Indicators | Body contains "Log in" button, no chat UI | + +**Detection:** +```javascript +const isLanding = document.title === 'muse.ai'; +// or +const isLoggedOut = document.body.innerText.includes('Log in') && + !document.querySelector('[data-testid="hatch-chat-switcher-trigger"]'); +``` + +### 1.6 State Summary Table + +| State | URL | Title | Switcher | Compose | Panel Content | +|---|---|---|---|---|---| +| Main chat | `muse.ai/` | `Chat — ...` | ✅ | ❌ | ❌ | +| Panel open | `muse.ai/` | `Chat — ...` | ✅ | ✅ | ✅ | +| `/thread/new` | `muse.ai/thread/new` | `Muse` | ❌ | ❌ | ❌ | +| `/thread/` | `muse.ai/thread/` | `Chat — ...` | ✅ | ❌ | ❌ | +| Landing | `muse.ai/` | `muse.ai` | ❌ | ❌ | ❌ | + +--- + +## 2. Key Selectors + +### 2.1 Chat Switcher (Panel Toggle) + +```javascript +document.querySelector('[data-testid="hatch-chat-switcher-trigger"]') +``` + +| Attribute | Value | +|---|---| +| Tag | `BUTTON` | +| `data-testid` | `hatch-chat-switcher-trigger` | +| Inner text | `Chats\nOpen chat and side chats` (or `Chats\nUnread chats\nOpen chat and side chats` when unreads exist) | +| `aria-expanded` | Not used (always `null`) | +| Class (partial) | `bg-fill-blur-thick shadow-blur-elevation-01 text-body-medium` | + +**Parent chain** (3-4 levels): +``` +BUTTON[data-testid="hatch-chat-switcher-trigger"] +└── DIV[data-testid="hatch-chat-leading-controls"] (cls: pointer-events-auto absolute start-3) + └── DIV (cls: pointer-events-none absolute inset-x-0) + └── DIV (cls: absolute inset-0 overflow-hidden) + └── DIV (cls: relative flex min-h-0) +``` + +**Visibility:** Present on main chat and `/thread/`. **Absent** on `/thread/new` and landing page. + +**Note:** The old detection `document.body.innerText.includes('Side chats')` is **invalid** — it matches the switcher's "Open chat and side chats" text and DM content mentioning "Side chats". Use the exact header check or `hatch-side-chats-panel-content` presence instead. + +### 2.2 Compose Button (New Side Chat `+`) + +```javascript +document.querySelector('[data-testid="hatch-chat-compose"]') +``` + +| Attribute | Value | +|---|---| +| Tag | `BUTTON` | +| `data-testid` | `hatch-chat-compose` | +| `aria-label` | `New side chat` | +| `title` | `New side chat` | +| Inner text | *(empty — SVG icon only)* | +| Class (partial) | `inline-flex shrink-0 cursor-pointer` | + +**Parent chain:** +``` +BUTTON[data-testid="hatch-chat-compose"] +└── SPAN (cls: contents) + └── DIV (cls: group/nav-section flex w-full) + └── SECTION (cls: flex flex-col) + └── DIV (cls: flex flex-col relative) +``` + +**Visibility conditions:** +- ✅ Present and visible **only when panel is open** +- ❌ Absent when panel is closed (even on main chat) +- ❌ Absent on `/thread/new` (stripped DOM) +- ❌ Absent on landing page + +**Critical:** This button contains only an SVG icon. Do NOT search by text `"+"` — it will never match. + +### 2.3 Sidechat Panel Shell + +```javascript +document.querySelector('[data-testid="hatch-side-chats-panel-shell"]') +``` + +Container for the entire chat navigation panel. Present in most states but may be empty/collapsed. + +**Children when panel is open:** +``` +hatch-side-chats-panel-sliding-surface +hatch-side-chats-panel-content +hatch-chat-search-toolbar +hatch-chat-search-field +hatch-side-chats-options +hatch-thread-row (× N) +hatch-chat-compose +hatch-side-chat-unread-indicator +``` + +### 2.4 Sidechat Panel Content + +```javascript +document.querySelector('[data-testid="hatch-side-chats-panel-content"]') +``` + +**This is the reliable "panel is open" indicator.** When present, the compose button should also be present. + +### 2.5 Thread Rows (Chat List Items) + +```javascript +document.querySelectorAll('[data-testid="hatch-thread-row"]') +``` + +| Attribute | Value | +|---|---| +| Tag | `DIV` | +| `data-testid` | `hatch-thread-row` | +| `role` | `button` | +| `tabindex` | `0` | +| Active row | `aria-current="page"` | +| Class (partial) | `group/nav-row relative flex h-9 w-full shrink-0 items-center` | + +**Not links** — they are `div[role="button"]` with click handlers. No `href`. To navigate, click the row. + +**Text format:** `\n<unread-indicator>\n<timestamp>` (e.g., `"Main chat\n\njust now"`) + +**Exact header check** (reliable "Side chats" panel header detection): +```javascript +// Find leaf element with exact text "Side chats" +const header = [...document.querySelectorAll('*')].find(e => + e.children.length === 0 && (e.innerText || '').trim() === 'Side chats' +); +// Returns SPAN element when panel is open, null otherwise +``` + +### 2.6 Message Input (Composer) + +```javascript +document.querySelector('[aria-label="Message"]') +``` + +| Attribute | Value | +|---|---| +| Tag | `TEXTAREA` | +| `aria-label` | `Message` | +| Class (partial) | `text-text-primary block max-h-48` | + +**Parent chain:** +``` +TEXTAREA[aria-label="Message"] +└── DIV (cls: relative flex min-h-8) + └── DIV (cls: flex shrink-0 flex-col) + └── DIV (cls: backdrop-blur-elevation-01 bg-fill-blur-thick shadow-blur-elevation-01) +``` + +### 2.7 "Main Chat" Navigation Button + +To navigate to main chat from a sidechat, find the button with **exact** text `"Main chat"`: + +```javascript +const mainBtn = [...document.querySelectorAll('button, [role="button"]')] + .find(e => (e.innerText || '').trim() === 'Main chat'); +``` + +**Must use exact match** (`=== 'Main chat'`), not substring — thread rows may contain "Main chat" as part of longer text. + +**Alternative:** Ctrl+J via CDP `Input.dispatchKeyEvent` (proven reliable for main-chat navigation): +```javascript +// Via CDP websocket: +{"id": 30, "method": "Input.dispatchKeyEvent", + "params": {"type": "keyDown", "key": "j", "code": "KeyJ", "ctrlKey": True, "modifiers": 2}} +{"id": 31, "method": "Input.dispatchKeyEvent", + "params": {"type": "keyUp", "key": "j", "code": "KeyJ", "ctrlKey": True, "modifiers": 2}} +``` + +--- + +## 3. Approval / Permission Dialogs + +### 3.1 How `check_approvals` Works + +Location: `/home/super/Projects/NetVM/bin/muse-chat-api.py`, `check_approvals(ws)` + +**Detection logic:** +1. Scans `document.body.innerText` for `"Allow"` + `"to share"` pattern +2. Finds elements containing both strings with text length < 500 chars +3. Fallback: looks for buttons with text containing `allow`/`deny`/`block` (if ≥2 found and no dialogs yet) + +**Critical filter** (added 2026-10-04 to fix false positives): +```python +# Only block for IP-based permission dialogs. +# Dialogs without IPs are likely false positives (chat content, UI text). +ips = re.findall(r'\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b', dialog_text) +if not ips: + continue # Skip — don't block +``` + +**Trusted IPs** (auto-approve "Allow"): +```python +TRUSTED_IPS = { + "34.139.37.135", # VM (gateway) + "100.123.153.75", # bl (main compute) + "100.81.31.9", # VM tailnet +} +``` + +**Auto-approve action:** Clicks button with text `"allow once"` or exact `"allow"`. + +**Exit code 2** with `APPROVAL_NEEDED` if untrusted IP dialog is found. + +### 3.2 Dialog Selectors + +No `[role="dialog"]` or `[role="alertdialog"]` elements were observed in the current DOM state. Permission dialogs appear as overlay elements when triggered (e.g., by download attempts, screen sharing requests). + +**Patterns to watch for:** +```javascript +// Dialog containers (when present) +'[role="dialog"]' +'[role="alertdialog"]' +'[data-testid*="dialog"]' +'[data-testid*="approval"]' +'[data-testid*="permission"]' + +// Permission buttons (when dialog is active) +'button' // with text: /allow|deny|block/i +``` + +**Known false positive:** Task/checklist buttons in chat content may contain "Confirm" in their text (e.g., "Confirm 03:45 heartbeat in main thread"). The IP filter prevents these from blocking automation. + +### 3.3 React DOM Selectors + +Per AGENTS.md (2026-10-03): *"The Muse platform is a React app. DOM selectors for dialogs exist identically in headful and headless environments."* The `check_approvals` selectors work in both. + +--- + +## 4. Overlays and Interaction Blockers + +### 4.1 Observed Overlays + +One fixed-position overlay with `z-index: 26` was observed: +```javascript +{ + tag: "DIV", + id: null, + testid: null, + role: null, + z: "26", + cls: "absolute start-0 top-0" +} +``` + +### 4.2 Potential Blockers + +| Element | Impact | +|---|---| +| Permission dialogs | Block sends until approved/dismissed | +| `hatch-browser-task-banner-host` | Browser task notification banner | +| `hatch-composer-placeholder-overlay` | Composer placeholder (not a blocker) | +| `hatch-status-panel-sliding-surface` | Status panel (may overlay content) | + +**Detection:** +```javascript +const blockers = [...document.querySelectorAll('*')].filter(e => { + const cs = getComputedStyle(e); + const r = e.getBoundingClientRect(); + return (cs.position === 'fixed' || cs.position === 'absolute') && + r.width > 100 && r.height > 100 && + parseInt(cs.zIndex || '0') > 10; +}); +``` + +--- + +## 5. URL Patterns + +| URL Pattern | Meaning | DOM State | +|---|---|---| +| `https://muse.ai/` | Main chat | Full DOM, switcher present | +| `https://muse.ai/thread/new` | New/unsent sidechat | **Stripped DOM**, no switcher/compose | +| `https://muse.ai/thread/<uuid>` | Existing sidechat | Full DOM, switcher present | + +**UUID format:** Standard UUID v4 (e.g., `550e8400-e29b-41d4-a716-446655440000`). + +**Navigation notes:** +- After `sidechat create`, URL goes to `/thread/new` (placeholder) +- After first message send, URL transitions to `/thread/<uuid>` (real ID) +- Poll `window.location.href` after send to capture the real thread ID +- `/thread/new` **poisons** subsequent creates — always navigate to main first + +--- + +## 6. Full `data-testid` Inventory + +57 unique `data-testid` values observed on main chat with panel open: + +### Chat Core +``` +hatch-chat-switcher-trigger # Panel toggle button +hatch-chat-switcher-icon-knockout +hatch-chat-switcher-navigation-icon +hatch-chat-compose # New side chat (+) button +hatch-chat-leading-controls # Switcher container +hatch-chat-nav-fade +hatch-chat-search-field # Panel search input +hatch-chat-search-toolbar +``` + +### Sidechat Panel +``` +hatch-side-chats-panel-shell +hatch-side-chats-panel-sliding-surface +hatch-side-chats-panel-content +hatch-side-chats-options +hatch-side-chat-unread-indicator +hatch-thread-row # Individual chat rows (× N) +active-side-chat-load-more-sentinel +``` + +### Navigation Dock +``` +hatch-dock-rail +hatch-dock-more +hatch-nav-chat +hatch-nav-search +hatch-nav-system-item-66656564 # (hex-encoded IDs) +hatch-nav-system-item-676f616c73 +hatch-nav-system-item-6964656173 +hatch-nav-system-item-6c696272617279 +hatch-panel-resize-divider-line +``` + +### Status/Other +``` +hatch-status-panel-close +hatch-status-panel-close-drag-handle +hatch-status-panel-close-drag-handle-divider-line +hatch-status-panel-close-toolbar +hatch-status-panel-sliding-surface +hatch-composer-placeholder-overlay +hatch-browser-task-banner-host +hatch-invite-friends-button +app-download-dock-mark +``` + +--- + +## 7. Reliable Automation Sequences + +### 7.1 Sidechat Creation (Robust) + +```python +# 1. Navigate to main chat (recover from any state) +send_ctrl_j() # CDP Input.dispatchKeyEvent +time.sleep(3) + +# 2. Verify we're on main chat +assert window.location.href == 'https://muse.ai/' +assert document.querySelector('[data-testid="hatch-chat-switcher-trigger"]') + +# 3. Open panel if compose button not present +if not document.querySelector('[data-testid="hatch-chat-compose"]'): + document.querySelector('[data-testid="hatch-chat-switcher-trigger"]').click() + time.sleep(2) + +# 4. Verify compose button is present AND visible +btn = document.querySelector('[data-testid="hatch-chat-compose"]') +assert btn and btn.getBoundingClientRect().width > 0 + +# 5. Click to create +btn.click() + +# 6. Poll for /thread/ URL (up to 15s) +# 7. Send message directly (browser is on the new sidechat) +# 8. Poll URL for /thread/<uuid> to capture real thread ID +``` + +### 7.2 State Detection (One-Liner) + +```javascript +(() => { + const url = window.location.href; + const title = document.title; + const hasSwitcher = !!document.querySelector('[data-testid="hatch-chat-switcher-trigger"]'); + const hasCompose = !!document.querySelector('[data-testid="hatch-chat-compose"]'); + const hasPanelContent = !!document.querySelector('[data-testid="hatch-side-chats-panel-content"]'); + + if (title === 'muse.ai') return 'LANDING'; + if (url.includes('/thread/new')) return 'THREAD_NEW'; + if (url.includes('/thread/')) return hasSwitcher ? 'SIDECHAT' : 'THREAD_NEW_STRIPPED'; + if (url === 'https://muse.ai/') { + if (!hasSwitcher) return 'UNKNOWN_NO_SWITCHER'; + if (hasCompose && hasPanelContent) return 'MAIN_PANEL_OPEN'; + return 'MAIN_PANEL_CLOSED'; + } + return 'UNKNOWN'; +})() +``` + +--- + +## 8. Body Landmarks (Top-Level Structure) + +```html +<body> + <div> <!-- React root (has_react: true) --> + <!-- App content --> + </div> + <iframe class="hidden"> × 4 <!-- Hidden utility iframes --> + <script id="hatch-early-gateway-bootstrap"> + <div class="min-w-0 flex group/sidebar-wrapper min-h-svh"> <!-- Sidebar wrapper --> + <section aria-label="Notifications alt+T"> + <script> × N <!-- App scripts --> +</body> +``` + +--- + +## Appendix: Change Log + +- **2026-10-04:** Initial mapping via live CDP inspection. Documented 5 page states, 57 testids, approval dialog logic, and the `/thread/new` poisoning behavior.