Files

423 lines
16 KiB
Markdown
Raw Permalink Normal View History

# muse.ai Message DOM Map
> **Box is the main surface.** All operator work goes through Box (box.muse-dev.online). The web UI, `box` CLI, and agents share the same API endpoints. No UI-only powers.
Reference for automation against the muse.ai chat UI. Captured live via CDP
on the `opm` headless browser (viewport 780×493), 2026-10-04 ~03:45 UTC.
Verified states: main chat (`https://muse.ai/`) and empty new thread
(`https://muse.ai/thread/new`).
**Re-verified 2026-10-04 ~04:30 UTC** on all three fleet nodes (muse/CDP 9410,
pip/9420, opm/9440) against live main chats with real message history.
No per-node structural differences found. Corrections and additions from
that pass are inline below.
Related maps: sidechat panel selectors live in the sidechat work
(`hatch-chat-switcher-trigger`, `hatch-chat-compose`); this doc covers the
message composer, send button, message list, and chat activation.
---
## 1. Message input area
### Primary selector
```js
document.querySelector('textarea[aria-label="Message"]')
```
Fallback chain used by `muse-chat-api.py` (any one may match depending on render):
```js
document.querySelector('[contenteditable="true"]')
|| document.querySelector('textarea[placeholder*="Message"]')
|| document.querySelector('div[role="textbox"]')
```
### Observed attributes (main chat)
| Attribute | Value |
|---------------|--------------------------------------------------------------|
| tag | `TEXTAREA` |
| `aria-label` | `Message` |
| `placeholder` | `Message` |
| `rows` | `1` |
| `disabled` | `false` |
| `readOnly` | `false` |
| class | `text-text-primary block max-h-48 w-full resize-none bg-transparent` |
| visible | `offsetParent !== null` → `true` |
### Parent chain (4 levels)
```
textarea[aria-label="Message"]
└── div.relative.flex.min-h-8.min-w-0.items-center.ps-10 (also contains an <input> and a <div> sibling)
└── div.flex.shrink-0.flex-col.px-3.py-3
└── div[role="button"].backdrop-blur-elevation-01.bg-fill-blur-thick.shadow-blur-elevation-01.rounded-32 ← composer "pill"
└── div.min-w-0.flex-1
```
### Ready-for-typing detection
```js
const ta = document.querySelector('textarea[aria-label="Message"]');
const ready = !!ta && ta.offsetParent !== null && !ta.disabled && !ta.readOnly;
```
Empty-state signal: the placeholder overlay is visible when nothing is typed:
```js
document.querySelector('[data-testid="hatch-composer-placeholder-overlay"]')
// → <span> innerText "Message", visible when input is empty
```
### Typing into it (React-safe)
`execCommand('insertText')` is what the current tooling uses:
```js
const input = document.querySelector('textarea[aria-label="Message"]');
input.focus();
document.execCommand('insertText', false, message);
```
Alternative when React ignores execCommand — native setter + input event:
```js
const setter = Object.getOwnPropertyDescriptor(window.HTMLTextAreaElement.prototype, 'value').set;
setter.call(ta, text);
ta.dispatchEvent(new Event('input', { bubbles: true }));
```
Adjacent composer buttons (same pill, not the send button):
| `aria-label` | Purpose |
|---------------------|----------------|
| `Attach file` | file upload |
| `Dictate a message` | voice input |
---
## 2. Send button
### Selector
```js
[...document.querySelectorAll('button')]
.find(b => b.getAttribute('aria-label')?.toLowerCase().includes('send'))
// observed aria-label is exactly "Send"
```
- Icon: inline SVG, arrow-up glyph (path data starts `M11.45 2.32`).
- No `data-testid`, no `title`, no innerText — match on `aria-label` only.
### Enabled / disabled states
- Observed `disabled === false` in the DOM even with an empty composer; the
button is **rendered once text is typed** and may persist afterwards.
- Do not rely on `disabled` alone. Practical send flow (as implemented):
```js
const send = [...document.querySelectorAll('button')]
.find(b => b.getAttribute('aria-label')?.toLowerCase().includes('send'));
if (send) { send.click(); /* 'sent' */ }
else {
// fallback: Enter key on the input
input.dispatchEvent(new KeyboardEvent('keydown', { key: 'Enter', code: 'Enter', bubbles: true }));
}
```
### While the assistant is generating
The send button is replaced by the stop button:
```js
document.querySelector('[data-testid="hatch-composer-stop-button"]')
// present only while a response is streaming; absent when idle
```
---
## 3. Message list
### Scroll container
```js
document.getElementById('hatch-chat-scroll')
```
| Attribute | Value |
|-----------|--------------------------------------------------------------------|
| tag | `DIV` |
| id | `hatch-chat-scroll` |
| class | `flex flex-col overflow-hidden relative z-0 grow-1 shrink-1 min-h-0` |
### List element
```js
document.querySelector('div[role="log"]')
```
| Attribute | Value |
|--------------|----------------------------------------------------|
| tag | `DIV` |
| `role` | `log` |
| `aria-label` | `Chat messages` |
| class | `mx-auto w-full max-w-3xl min-w-0 @container flex` |
Parent chain of a message (bottom-up):
```
div[data-message-id] ← one per message, class "flex flex-col gap-2"
└── div[role="log"][aria-label="Chat messages"]
└── div
└── div
└── div
└── div#hatch-chat-scroll
```
### Individual message
Selector for all messages:
```js
document.querySelectorAll('div[data-message-id]')
```
**Structural correction (verified 2026-10-04, pip/muse/opm):** the interactive
`.group/msg` wrapper is *inside* `div[data-message-id]`, not a sibling
rendering. True per-message structure:
```
div[data-message-id] ← class "flex flex-col gap-2", one per message
├── div.group/msg ← interactive bubble + hover action rail
│ ├── div (action rail, absolute inset-y-0 end-full)
│ │ └── button[aria-label="More options"]
│ └── div.hatch-chat-groupable-bubble
│ ├── span.sr-only "You:" (own messages only, bubble-level)
│ └── p / div (visible body)
└── span.sr-only (direct child of the wrapper)
"User message: <full text>" / "Assistant message: <full text>"
```
The direct-child `span.sr-only` carries the **full accessible label** (author
prefix + entire message text, observed up to 2000+ chars). Own messages therefore
have *two* sr-only spans: `"User message: …"` on the wrapper and `"You:"` in the
bubble. Agent messages have only the wrapper-level `"Assistant message: …"` span.
There are still no `<article>` / `<time>` / `data-testid`-containing-"message"
elements, and no *visible* author name on either message type.
Author is encoded in the id prefix:
| Author | `data-message-id` format | `innerText` prefix |
|-----------|--------------------------------|--------------------------|
| assistant | `assistant-msg-<uuid>` | `Assistant message: ...` |
| user | `<uuid>` (bare, no prefix) | `User message: ...` |
The `User message: ...` prefix **is** observed (verified 2026-10-04 on pip's
main chat); id-prefix detection remains the most robust signal.
⚠️ `data-message-id` is **not messages-only**: non-message cards share the
attribute, e.g. `data-message-id="presentation-carrier:8917…"` (media/
presentation card, empty innerText). For real messages filter to
`/^[0-9a-f-]{36}$/` (user) or `/^assistant-msg-/` (assistant).
### Extracting text / author
```js
const msgs = [...document.querySelectorAll('[data-message-id]')].map(m => {
const id = m.getAttribute('data-message-id');
return {
id,
author: id.startsWith('assistant-msg') ? 'assistant' : 'user',
text: m.innerText
};
});
```
### Simple read-back (what `muse-chat-api.py messages` does)
```js
[...document.querySelectorAll('p')].slice(-n * 2).map(p => p.innerText.slice(0, width))
```
Works because message bodies render as `<p>` elements; the last `2n`
paragraphs cover roughly the last `n` messages. Crude but effective for
"did my text land?" verification.
### Virtualization (verified 2026-10-04)
The message list is **virtualized**: only viewport-near messages exist in the
DOM. Observed `[data-message-id]` counts on live main chats: 5, 33, 40, 110
across nodes and moments on the same threads. Consequences for automation:
- DOM readback is **windowed** — you only see what's rendered. Scroll
`#hatch-chat-scroll` to load older messages; re-query after scrolling.
- Never assert "message absent" from a single DOM snapshot; assert
"not in rendered window".
- `.group/msg` count always equals the rendered message count (every rendered
message gets the interactive wrapper + its `More options` rail button —
verified rails == groups on all three nodes).
### Date / time dividers
`div[role="log"]` children are not all messages. Date and time dividers are
direct children of the log, interspersed between `div[data-message-id]`:
```html
<!-- day boundary -->
<div class="pt-10 pb-4 text-center">
<span class="text-caption-1 text-text-secondary">Oct 3, 11:41 PM</span>
</div>
<!-- intra-day gap -->
<div class="py-4 text-center">
<span class="text-caption-1 text-text-secondary">2:38 AM</span>
</div>
```
- Day divider: `div.pt-10.pb-4.text-center`, text `Oct 3, 11:41 PM`.
- Time divider: `div.py-4.text-center`, text `2:38 AM` (bare 12h time).
- Zero `<time>` elements anywhere in the message DOM (verified on all nodes).
### Truncation
No truncation observed: a 2110-char message rendered in full, and no
"show more" / "expand" / "read more" buttons were found in any inspected
thread. Long messages are fully present in `innerText`.
### DMs vs chat messages
**No structural difference.** DMs arrive as ordinary user messages in main
chat; sender attribution is text-only, with two formats observed live:
- Legacy: `[from opm] [7fce46e0] <text>`
- Signed: `[from:operator-main] [id:fd553ef5] <text>` (+ `-----BEGIN SSH SIGNATURE-----` block)
Anything parsing DMs must match the text, not the markup. (Textual `[from:X]`
is unauthenticated — see DM trust notes elsewhere.)
### Unread signals
- `hatch-nav-chat` aria-label: `Chat, N notifications` (already documented).
- Page title prefix: `(1) Chat — <agent>` — the count also appears in
`document.title` (observed `(1)` on pip).
### Message-related testids seen in the wild
| testid | Meaning | Present when |
|---------------------------------------|--------------------------------------|--------------|
| `hatch-chat-typing-indicator` | assistant is typing/generating | generating only |
| `hatch-composer-stop-button` | stop-generation button | generating only |
| `hatch-composer-placeholder-overlay` | "Message" ghost text | composer empty |
| `hatch-chat-switcher-unread-indicator`| unread badge on the chat switcher | unread chats exist |
---
## 4. Chat activation button (speech-bubble / nav)
This is the dock-rail button that activates the Chat section. The user's
finding: sidechat create only works when the chat function is active, and
this button (not Ctrl+J) is the reliable activator.
### Selector
```js
document.querySelector('[data-testid="hatch-nav-chat"]')
```
### Observed attributes
| Attribute | Value |
|----------------|------------------------------------------------------------|
| tag | `A` (anchor, not button) |
| `data-testid` | `hatch-nav-chat` |
| `href` | `/` |
| `aria-label` | dynamic: `Chat` or `Chat, N notifications` (e.g. `Chat, 2 notifications`) |
| `aria-current` | `page` when Chat is the active section (absent otherwise) |
| visible | `true` (dock rail is always rendered) |
### Parent chain (4 levels)
```
a[data-testid="hatch-nav-chat"][aria-label="Chat, 2 notifications"][aria-current="page"]
└── div
└── div
└── div
└── div[data-testid="hatch-dock-rail"]
```
### States
| State | Signal |
|-------|--------|
| Chat section active | `aria-current="page"` present |
| Chat section inactive | `aria-current` absent (button still visible/clickable) |
| Unread messages | `aria-label` contains `N notifications` |
### Usage
```js
// Activate chat reliably (replaces Ctrl+J):
document.querySelector('[data-testid="hatch-nav-chat"]').click();
```
`cmd_sidechat_main` already falls back to this click when the "Main chat"
element lookup fails. Sibling nav buttons in the same rail:
`hatch-nav-search` (Search), plus `div[role="button"]` items for Feed, Ideas,
Goals, Library, and `hatch-dock-more` (Settings).
---
## 5. Loading / empty states
| State | How to detect |
|-------|---------------|
| New empty thread (`/thread/new`) | URL is `/thread/new`, `document.title === "Muse"`, zero `[data-message-id]` elements, composer present |
| Main chat loaded | URL is `/`, title is `Chat — <agent>`, `[data-message-id]` elements present |
| Assistant generating | `[data-testid="hatch-chat-typing-indicator"]` exists; send button replaced by `[data-testid="hatch-composer-stop-button"]` |
| Idle, ready to send | typing indicator absent, stop button absent, `textarea[aria-label="Message"]` visible and enabled |
| Composer empty | `[data-testid="hatch-composer-placeholder-overlay"]` visible with text `Message` |
| Side panel open (threads visible) | `[data-testid="hatch-chat-compose"]` ("New side chat") present |
### Page-title signals
| Title | Meaning |
|-------|---------|
| `Chat — <agent>` | logged in, chat UI loaded |
| `Muse` | partial load or `/thread/new` stripped state — treat as not-ready, navigate to `/` first |
---
## Quick-reference selector list
```js
// Composer
'textarea[aria-label="Message"]'
'[data-testid="hatch-composer-placeholder-overlay"]' // empty-state ghost text
// Send / stop
'button[aria-label="Send"]' // via aria-label includes 'send'
'[data-testid="hatch-composer-stop-button"]' // while generating
// Messages
'#hatch-chat-scroll' // scroll container
'div[role="log"][aria-label="Chat messages"]' // list
'[data-message-id]' // each message
'[data-message-id^="assistant-msg"]' // assistant messages only
// Chat activation
'[data-testid="hatch-nav-chat"]' // dock-rail Chat link
'[data-testid="hatch-chat-typing-indicator"]' // generating indicator
```
## Caveats
- Selectors observed on the `opm` account; class names are Tailwind-ish and
may shift with deploys — prefer `data-testid`, `aria-label`, `role`, and
`id` selectors over classes.
- `aria-label` on `hatch-nav-chat` embeds a live notification count; match
with `startsWith('Chat')` or the testid, never exact equality.
- During this capture another agent was driving the same browser; page URL
flipped between `/` and `/thread/new` between probes. Element identities
above were each verified at least once against the live DOM.