super/box
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
6a54320d69113b4f9aca460a170e81672e300624
box/docs/DOM-PAGE-STRUCTURE.md
T

21 KiB
Raw Blame History

muse.ai DOM Page Structure Reference

Box is the main surface. All operator work goes through Box (box.muse-dev.online). The web UI, box CLI, and agents share the same API endpoints. No UI-only powers.

Generated: 2026-10-04 via live CDP inspection (opm browser, warp-opm netns, CDP 9440) Purpose: Reference for future automation development. All selectors verified against live DOM.

1. Page States / Modes

The muse.ai SPA has distinct DOM states. Detect state before acting — selectors valid in one state are absent in another.

1.1 Main Chat

Property Value
URL https://muse.ai/
Title Chat — <agent-name> (e.g., Chat — operator-main)
Switcher [data-testid="hatch-chat-switcher-trigger"] ✅ Present
Compose [data-testid="hatch-chat-compose"] ❌ Absent (panel closed)
Panel content [data-testid="hatch-side-chats-panel-content"] ❌ Absent (panel closed)
Panel shell [data-testid="hatch-side-chats-panel-shell"] ✅ Present (empty/collapsed)
Message input [aria-label="Message"] ✅ Present, visible

Detection:

// Reliable main-chat detection
const isMainChat =
  window.location.href === 'https://muse.ai/' &&
  document.title.startsWith('Chat —') &&
  !!document.querySelector('[data-testid="hatch-chat-switcher-trigger"]');

1.2 Panel Open (Chat Navigation Panel Visible)

Reached by clicking the switcher on main chat. This is the required precondition for sidechat creation.

Property Value
URL https://muse.ai/ (unchanged)
Title Chat — <agent-name> (unchanged)
Switcher ✅ Present
Compose [data-testid="hatch-chat-compose"] ✅ Present and visible
Panel content ✅ Present
Thread rows [data-testid="hatch-thread-row"] ✅ Present (26 rows observed)

Detection:

// Panel is open and ready for sidechat creation
const isPanelOpen =
  !!document.querySelector('[data-testid="hatch-side-chats-panel-content"]') &&
  !!document.querySelector('[data-testid="hatch-chat-compose"]');

// Verify compose button is actually clickable
const composeBtn = document.querySelector('[data-testid="hatch-chat-compose"]');
const isClickable = composeBtn && composeBtn.getBoundingClientRect().width > 0;

Reliable open sequence:

// 1. Ensure on main chat (Ctrl+J via CDP Input.dispatchKeyEvent)
// 2. Check if panel already open
if (!document.querySelector('[data-testid="hatch-chat-compose"]')) {
  // 3. Click switcher to open panel
  document.querySelector('[data-testid="hatch-chat-switcher-trigger"]').click();
  // 4. Wait 2s, re-check for compose button
}

1.3 /thread/new (Fresh Sidechat / Poisoned State)

This is the "poisoned" state. After creating a sidechat, the browser lands here with a stripped DOM that lacks the switcher and compose button. The next sidechat create will fail unless you navigate back to main first.

Property Value
URL https://muse.ai/thread/new
Title Muse (just "Muse", not "Chat — ...")
Switcher ❌ Absent
Compose ❌ Absent
Panel shell ✅ Present but stripped (no panel-content)
Message input ✅ Present
Button count ~45 (vs ~21 on partially-loaded states)

Detection:

const isThreadNew =
  window.location.href.includes('/thread/new') ||
  (document.title === 'Muse' &&
   !document.querySelector('[data-testid="hatch-chat-switcher-trigger"]'));

Recovery: Click button[aria-label="Back to main chat"] (present on /thread/new and on regular /thread/<uuid> pages — verified on all fleet nodes 2026-10-04), or click [data-testid="hatch-nav-chat"], then open the panel via the switcher. Do NOT use Ctrl+J here: stripped states lack keyboard focus, so the keystroke goes nowhere (supersedes earlier Ctrl+J guidance; see DOM-INDEX §5).

Distinct testid set (no chat-switcher testids, dock-rail based):

hatch-dock-rail, hatch-dock-rail-divider, hatch-nav-chat,
hatch-dock-chat-notification-badge, hatch-nav-search,
hatch-nav-system-item-*, app-download-dock-mark, hatch-dock-more,
hatch-side-chats-open-drag-handle, hatch-side-chats-panel-shell,
hatch-chat-nav-fade, hatch-chat-leading-controls,
hatch-status-panel-sliding-surface, hatch-composer-placeholder-overlay,
hatch-browser-task-banner-host

1.4 Sidechat Thread (/thread/<uuid>)

A real sidechat with a UUID. DOM structure is similar to main chat (switcher present, panel available). Thread rows in the panel show the sidechat title.

Note: After sending the first message in a /thread/new chat, the URL transitions to /thread/<uuid>. Poll window.location.href after send to capture the real thread ID.

1.5 Landing Page (Not Logged In)

Property Value
Title muse.ai (just the domain)
Indicators Body contains "Log in" button, no chat UI

Detection:

const isLanding = document.title === 'muse.ai';
// or
const isLoggedOut = document.body.innerText.includes('Log in') &&
  !document.querySelector('[data-testid="hatch-chat-switcher-trigger"]');

1.6 State Summary Table

State URL Title Switcher Compose Panel Content
Main chat muse.ai/ Chat — ... ✅ ❌ ❌
Panel open muse.ai/ Chat — ... ✅ ✅ ✅
/thread/new muse.ai/thread/new Muse ❌ ❌ ❌
/thread/<uuid> muse.ai/thread/<uuid> Chat — ... ✅ ❌ ❌
Landing muse.ai/ muse.ai ❌ ❌ ❌

2. Key Selectors

2.1 Chat Switcher (Panel Toggle)

document.querySelector('[data-testid="hatch-chat-switcher-trigger"]')
Attribute Value
Tag BUTTON
data-testid hatch-chat-switcher-trigger
Inner text The current thread's title (e.g. 646 pip cross-operator coordination chat\nOpen chat and side chats) — NOT the literal Chats. Never match the switcher by text.
aria-expanded Not used (always null)
Class (partial) bg-fill-blur-thick shadow-blur-elevation-01 text-body-medium

Parent chain (3-4 levels):

BUTTON[data-testid="hatch-chat-switcher-trigger"]
└── DIV[data-testid="hatch-chat-leading-controls"]  (cls: pointer-events-auto absolute start-3)
    └── DIV  (cls: pointer-events-none absolute inset-x-0)
        └── DIV  (cls: absolute inset-0 overflow-hidden)
            └── DIV  (cls: relative flex min-h-0)

Visibility: Present on main chat and /thread/<uuid>. Absent on /thread/new and landing page.

Note: The old detection document.body.innerText.includes('Side chats') is invalid — it matches the switcher's "Open chat and side chats" text and DM content mentioning "Side chats". Use the exact header check or hatch-side-chats-panel-content presence instead.

2.2 Compose Button (New Side Chat +)

document.querySelector('[data-testid="hatch-chat-compose"]')
Attribute Value
Tag BUTTON
data-testid hatch-chat-compose
aria-label New side chat
title New side chat
Inner text (empty — SVG icon only)
Class (partial) inline-flex shrink-0 cursor-pointer

Parent chain:

BUTTON[data-testid="hatch-chat-compose"]
└── SPAN  (cls: contents)
    └── DIV  (cls: group/nav-section flex w-full)
        └── SECTION  (cls: flex flex-col)
            └── DIV  (cls: flex flex-col relative)

Visibility conditions:

  • ✅ Present and visible only when panel is open
  • ❌ Absent when panel is closed (even on main chat)
  • ❌ Absent on /thread/new (stripped DOM)
  • ❌ Absent on landing page

Critical: This button contains only an SVG icon. Do NOT search by text "+" — it will never match.

2.3 Sidechat Panel Shell

document.querySelector('[data-testid="hatch-side-chats-panel-shell"]')

Container for the entire chat navigation panel. Present in most states but may be empty/collapsed.

Children when panel is open:

hatch-side-chats-panel-sliding-surface
hatch-side-chats-panel-content
hatch-chat-search-toolbar
hatch-chat-search-field
hatch-side-chats-options
hatch-thread-row (× N)
hatch-chat-compose
hatch-side-chat-unread-indicator

2.4 Sidechat Panel Content

document.querySelector('[data-testid="hatch-side-chats-panel-content"]')

This is the reliable "panel is open" indicator. When present, the compose button should also be present.

2.5 Thread Rows (Chat List Items)

document.querySelectorAll('[data-testid="hatch-thread-row"]')
Attribute Value
Tag DIV
data-testid hatch-thread-row
role button
tabindex 0
Active row aria-current="page"
Class (partial) group/nav-row relative flex h-9 w-full shrink-0 items-center

Not links — they are div[role="button"] with click handlers. No href. To navigate, click the row.

Text format: <title>\n<unread-indicator>\n<timestamp> (e.g., "Main chat\n\njust now")

Exact header check (reliable "Side chats" panel header detection):

// Find leaf element with exact text "Side chats"
const header = [...document.querySelectorAll('*')].find(e =>
  e.children.length === 0 && (e.innerText || '').trim() === 'Side chats'
);
// Returns SPAN element when panel is open, null otherwise

2.6 Message Input (Composer)

document.querySelector('[aria-label="Message"]')
Attribute Value
Tag TEXTAREA
aria-label Message
Class (partial) text-text-primary block max-h-48

Parent chain:

TEXTAREA[aria-label="Message"]
└── DIV  (cls: relative flex min-h-8)
    └── DIV  (cls: flex shrink-0 flex-col)
        └── DIV  (cls: backdrop-blur-elevation-01 bg-fill-blur-thick shadow-blur-elevation-01)

2.7 "Main Chat" Navigation

Primary (verified 2026-10-04 on muse/pip/opm): the button with aria-label="Back to main chat" — present on /thread/<uuid> pages and on /thread/new, even with the panel closed:

const backBtn = [...document.querySelectorAll('button')]
  .find(b => b.getAttribute('aria-label') === 'Back to main chat');
backBtn?.click();

Panel-open alternative: when the sidechat panel is open, find the element with exact text "Main chat":

const mainBtn = [...document.querySelectorAll('button, [role="button"]')]
  .find(e => (e.innerText || '').trim() === 'Main chat');

Must use exact match (=== 'Main chat'), not substring — thread rows may contain "Main chat" as part of longer text. Note: with the panel closed on a thread page there is no exact-text "Main chat" element at all — use the aria-label button above.

Do not use Ctrl+J for this (superseded 2026-10-04): it needs keyboard focus, which stripped/partial states lack. Prefer DOM clicks (commit 72574ba).

3. Approval / Permission Dialogs

3.1 How check_approvals Works

Location: /home/super/Projects/NetVM/bin/muse-chat-api.py, check_approvals(ws)

Detection logic:

  1. Scans document.body.innerText for "Allow" + "to share" pattern
  2. Finds elements containing both strings with text length < 500 chars
  3. Fallback: looks for buttons with text containing allow/deny/block (if ≥2 found and no dialogs yet)

Critical filter (added 2026-10-04 to fix false positives):

# Only block for IP-based permission dialogs.
# Dialogs without IPs are likely false positives (chat content, UI text).
ips = re.findall(r'\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b', dialog_text)
if not ips:
    continue  # Skip — don't block

Trusted IPs (auto-approve "Allow"):

TRUSTED_IPS = {
    "34.139.37.135",   # VM (gateway)
    "100.123.153.75",  # bl (main compute)
    "100.81.31.9",     # VM tailnet
}

Auto-approve action: Clicks button with text "allow once" or exact "allow".

Exit code 2 with APPROVAL_NEEDED if untrusted IP dialog is found.

3.2 Dialog Selectors

No [role="dialog"] or [role="alertdialog"] elements were observed in the current DOM state. Permission dialogs appear as overlay elements when triggered (e.g., by download attempts, screen sharing requests).

Patterns to watch for:

// Dialog containers (when present)
'[role="dialog"]'
'[role="alertdialog"]'
'[data-testid*="dialog"]'
'[data-testid*="approval"]'
'[data-testid*="permission"]'

// Permission buttons (when dialog is active)
'button' // with text: /allow|deny|block/i

Known false positive: Task/checklist buttons in chat content may contain "Confirm" in their text (e.g., "Confirm 03:45 heartbeat in main thread"). The IP filter prevents these from blocking automation.

3.3 React DOM Selectors

Per AGENTS.md (2026-10-03): "The Muse platform is a React app. DOM selectors for dialogs exist identically in headful and headless environments." The check_approvals selectors work in both.

4. Overlays and Interaction Blockers

4.1 Observed Overlays

One fixed-position overlay with z-index: 26 was observed:

{
  tag: "DIV",
  id: null,
  testid: null,
  role: null,
  z: "26",
  cls: "absolute start-0 top-0"
}

4.2 Potential Blockers

Element Impact
Permission dialogs Block sends until approved/dismissed
hatch-browser-task-banner-host Browser task notification banner
hatch-composer-placeholder-overlay Composer placeholder (not a blocker)
hatch-status-panel-sliding-surface Status panel (may overlay content)

Detection:

const blockers = [...document.querySelectorAll('*')].filter(e => {
  const cs = getComputedStyle(e);
  const r = e.getBoundingClientRect();
  return (cs.position === 'fixed' || cs.position === 'absolute') &&
         r.width > 100 && r.height > 100 &&
         parseInt(cs.zIndex || '0') > 10;
});

5. URL Patterns

URL Pattern Meaning DOM State
https://muse.ai/ Main chat Full DOM, switcher present
https://muse.ai/thread/new New/unsent sidechat Stripped DOM, no switcher/compose
https://muse.ai/thread/<uuid> Existing sidechat Full DOM, switcher present

UUID format: Standard UUID v4 (e.g., 550e8400-e29b-41d4-a716-446655440000).

Navigation notes:

  • After sidechat create, URL goes to /thread/new (placeholder)
  • After first message send, URL transitions to /thread/<uuid> (real ID)
  • Poll window.location.href after send to capture the real thread ID
  • /thread/new poisons subsequent creates — always navigate to main first

6. Full data-testid Inventory

57 unique data-testid values observed on main chat with panel open:

Chat Core

hatch-chat-switcher-trigger      # Panel toggle button
hatch-chat-switcher-icon-knockout
hatch-chat-switcher-navigation-icon
hatch-chat-compose               # New side chat (+) button
hatch-chat-leading-controls      # Switcher container
hatch-chat-nav-fade
hatch-chat-search-field          # Panel search input
hatch-chat-search-toolbar

Sidechat Panel

hatch-side-chats-panel-shell
hatch-side-chats-panel-sliding-surface
hatch-side-chats-panel-content
hatch-side-chats-options
hatch-side-chat-unread-indicator
hatch-side-chats-open-drag-handle            # present when panel CLOSED (verified 2026-10-04)
hatch-side-chats-open-drag-handle-divider-line  # NEW 2026-10-04: divider for the above; absent when panel open
hatch-thread-row                 # Individual chat rows (× N)
active-side-chat-load-more-sentinel

Navigation Dock

hatch-dock-rail                              # DIV; children: dock-rail-divider (SPAN),
                                             # nav-chat, dock-chat-notification-badge,
                                             # nav-search, 4× nav-system-item-<hex>,
                                             # app-download-dock-mark, dock-more
hatch-dock-rail-divider                      # SPAN (not DIV)
hatch-dock-more
hatch-nav-chat                               # A, href="/", aria-label dynamic:
                                             # "Chat" | "Chat, N notification(s)" —
                                             # match testid, NEVER exact aria-label.
                                             # aria-current="page" when Chat section active.
                                             # Verified 2026-10-04 on muse/pip/opm.
hatch-nav-search
hatch-nav-system-item-66656564   # (hex-encoded IDs)
hatch-nav-system-item-676f616c73
hatch-nav-system-item-6964656173
hatch-nav-system-item-6c696272617279
hatch-panel-resize-divider-line

Status/Other

hatch-status-panel-close
hatch-status-panel-close-drag-handle
hatch-status-panel-close-drag-handle-divider-line
hatch-status-panel-close-toolbar
hatch-status-panel-sliding-surface
hatch-composer-placeholder-overlay
hatch-browser-task-banner-host
hatch-inline-approval-card                 # NEW 2026-10-04 (pip): DIV, no role —
                                           # browser-task approval card ("A task needs review" + Review button),
                                           # no IP in text. Structure TBD — see DOM-APPROVALS-SPEC.md (inspector 3/5).
hatch-invite-friends-button
app-download-dock-mark

7. Reliable Automation Sequences

7.1 Sidechat Creation (Robust)

# 1. Recover to a chat-active state (click-based; never Ctrl+J — see §2.7):
#    priority: compose (already ready) -> switcher -> hatch-nav-chat
#    (muse-chat-api.py cmd_sidechat_main clicks the exact-text "Main chat"
#    element; the aria-label "Back to main chat" button also works)
#    Poll up to ~20s for the compose button.
#    NOTE: plain JS .click() on the switcher does NOT reliably toggle the panel
#    (Radix control — trusted CDP Input.dispatchMouseEvent needed, verified
#    2026-10-04). Prefer the compose-button presence check over blind toggling.

# 2. Verify we're on main chat
assert window.location.href == 'https://muse.ai/'
assert document.querySelector('[data-testid="hatch-chat-switcher-trigger"]')

# 3. Open panel if compose button not present
if not document.querySelector('[data-testid="hatch-chat-compose"]'):
    document.querySelector('[data-testid="hatch-chat-switcher-trigger"]').click()
    time.sleep(2)

# 4. Verify compose button is present AND visible
btn = document.querySelector('[data-testid="hatch-chat-compose"]')
assert btn and btn.getBoundingClientRect().width > 0

# 5. Click to create
btn.click()

# 6. Poll for /thread/ URL (up to 15s)
# 7. Send message directly (browser is on the new sidechat)
# 8. Poll URL for /thread/<uuid> to capture real thread ID

7.2 State Detection (One-Liner)

(() => {
  const url = window.location.href;
  const title = document.title;
  const hasSwitcher = !!document.querySelector('[data-testid="hatch-chat-switcher-trigger"]');
  const hasCompose = !!document.querySelector('[data-testid="hatch-chat-compose"]');
  const hasPanelContent = !!document.querySelector('[data-testid="hatch-side-chats-panel-content"]');

  if (title === 'muse.ai') return 'LANDING';
  if (url.includes('/thread/new')) return 'THREAD_NEW';
  if (url.includes('/thread/')) return hasSwitcher ? 'SIDECHAT' : 'THREAD_NEW_STRIPPED';
  if (url === 'https://muse.ai/') {
    if (!hasSwitcher) return 'UNKNOWN_NO_SWITCHER';
    if (hasCompose && hasPanelContent) return 'MAIN_PANEL_OPEN';
    return 'MAIN_PANEL_CLOSED';
  }
  return 'UNKNOWN';
})()

8. Body Landmarks (Top-Level Structure)

<body>
  <div>                          <!-- React root (has_react: true) -->
    <!-- App content -->
  </div>
  <iframe class="hidden"> × 4     <!-- Hidden utility iframes -->
  <script id="hatch-early-gateway-bootstrap">
  <div class="min-w-0 flex group/sidebar-wrapper min-h-svh">  <!-- Sidebar wrapper -->
  <section aria-label="Notifications alt+T">
  <script> × N                   <!-- App scripts -->
</body>

Appendix: Change Log

  • 2026-10-04: Initial mapping via live CDP inspection. Documented 5 page states, 57 testids, approval dialog logic, and the /thread/new poisoning behavior.
  • 2026-10-04 (inspector 5/5 re-verification, muse/pip/opm via CDP):
    • hatch-nav-chat verified: A, href="/", dynamic aria-label (Chat / Chat, N notification(s)), aria-current="page".
    • Switcher inner text is the current thread title, not Chats — corrected §2.1 (never match switcher by text).
    • button[aria-label="Back to main chat"] is present on /thread/<uuid> pages too, not only /thread/new — §2.7 rewritten around it as the primary main-chat affordance; Ctrl+J guidance removed as superseded.
    • Plain JS .click() on the switcher does not reliably toggle the panel (Radix control needs trusted CDP pointer events) — noted in §7.1.
    • New testids: hatch-side-chats-open-drag-handle-divider-line (all nodes), hatch-inline-approval-card (pip; browser-task approval card, no IP — structure deferred to DOM-APPROVALS-SPEC.md).
    • Title gains an unread prefix: (1) Chat — <agent> (observed on pip).
    • Settled-state variance: shimmer spans 1 (not 2), role="status" 3–4 (not 4–7), buttons 46–57 — ranges, not fixed counts.
    • Fleet browsers are shared: another agent drove pip's browser mid-probe (URL/panel state changed between probes). Never assume a probe snapshot is still current — re-probe before acting.
Reference in New Issue View Git Blame Copy Permalink