feat(pets): pop-out desktop overlay + notifications (#47938)
* fix(pets): map sprite rows by atlas shape, not a fixed order Petdex/Codex sheets are 8 cols x 9 rows (jump=4, failed=5, run=7, review=8); older Hermes sheets were 8 rows in our own order. We hardcoded the legacy order, so on a 9-row pet "celebrate" (jump) cropped the failed row — a finished turn rendered the sad pose. Derive the row taxonomy from the sheet's row count and ship the right list to the desktop canvas. * feat(desktop): pop-out pet overlay with bubble, composer, and mail Shift-click the floating pet to pop it into a transparent, always-on-top desktop window that stays visible while Hermes is minimized. It's a pure puppet of the in-app pet (no second gateway): the main renderer mirrors live state over IPC and the overlay renders the same sprite + bubble. - Speech bubble externalizes activity (working/thinking/your turn) and a gold star on finish; the star outlives the celebrate jump so the finish stays glanceable. - Drag anywhere (even off-window); position + in/out state persist. - Click opens a mini composer that sends to the most recent session. - Mail icon (only when a turn finished while you were away) raises the app on that thread and marks it read. - macOS: NSPanel + skipTransformProcessType so popping out never drops the app from cmd/alt-tab. Also fixes the completion beat: flashPetActivity now clears sibling beats, so a stale error can't make a clean finish render the failed pose. * docs(pets): document the desktop pop-out overlay * refactor(pets): drop done-star and align runtime mapping to spec rows Remove the completion star bubble now that the overlay mail icon carries "finished while away" signal. Keep completion feedback in the sprite animation and unread mail only. Tighten row mapping to the current Petdex 9-row taxonomy while preserving legacy compatibility: we now resolve state rows through aliases (wave/waving, jump/jumping, run/running), add a dedicated waiting state, and map awaiting-input to waiting instead of idle. This makes animation selection less awkward across mixed assets and keeps Hermes aligned with the live Petdex state viewer semantics. * fix(pets): bubbles are overlay-only, not on the in-window pet The speech bubble is the glance/notification surface for the popped-out overlay. In-window the app itself is the surface, so drop the bubble there.
This commit is contained in:
parent
25e78f129f
commit
365c23c554
@ -2,10 +2,10 @@
|
||||
|
||||
Petdex (https://github.com/crafter-station/petdex) is a public gallery of
|
||||
animated sprite "pets" for coding agents. Each pet is a ``pet.json`` plus a
|
||||
``spritesheet.{webp,png}`` — an 8-column × 9-row grid of 192×208 px frames
|
||||
where each *row* is an animation state (idle, wave, run, failed, review,
|
||||
jump, …). The official desktop only ever renders the idle row; reacting to
|
||||
real agent activity is the value Hermes adds here.
|
||||
``spritesheet.{webp,png}`` of 192×208 px cells. Current Codex/petdex sheets use
|
||||
an 8-column × 9-row atlas; older Hermes/petdex sheets used an 8-row atlas.
|
||||
Hermes infers the row taxonomy from the sheet and maps agent activity onto
|
||||
idle/run/review/failed/wave/jump.
|
||||
|
||||
This package is the **single source of truth** for the feature so the base
|
||||
CLI (Python) and TUI (Ink, via ``tui_gateway``) never duplicate the hard
|
||||
|
||||
@ -1,21 +1,19 @@
|
||||
"""Pet sprite geometry + animation-state taxonomy.
|
||||
|
||||
These values are *constants of the petdex format*, not per-pet data — the
|
||||
real ``pet.json`` only carries ``id``/``displayName``/``description``/
|
||||
``spritesheetPath``. The official petdex web app and desktop client both
|
||||
hardcode 192×208 frames, 6 frames per state, a 1100ms loop, and a 0.7 render
|
||||
scale; we match them so installed pets animate identically.
|
||||
These values are the common petdex/Codex pet geometry. The real ``pet.json``
|
||||
usually only carries ``id``/``displayName``/``description``/``spritesheetPath``;
|
||||
row taxonomy is inferred from the atlas shape so Hermes can render both legacy
|
||||
8-row sheets and current 9-row Codex sheets.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import Enum
|
||||
|
||||
# Frame geometry (pixels). Per the petdex package format a spritesheet is an
|
||||
# 8-row × 9-column grid of these frames (1728×1664 px): one state per row (see
|
||||
# ``STATE_ROWS``), frames stepping left→right across the 9 columns. We only
|
||||
# read ``FRAMES_PER_STATE`` (6) of each row; renderers derive the real column
|
||||
# count from the sheet width, so sheets with a different column count still work.
|
||||
# Frame geometry (pixels). Current Codex/petdex spritesheets are 8 columns x 9
|
||||
# rows (1536x1872), while older Hermes/petdex sheets used 9 columns x 8 rows
|
||||
# (1728x1664). Renderers derive both row taxonomy and real column count from the
|
||||
# concrete sheet, so either shape works.
|
||||
FRAME_W = 192
|
||||
FRAME_H = 208
|
||||
|
||||
@ -80,8 +78,9 @@ def resolve_cols(scale: float, unicode_cols: int = 0) -> int:
|
||||
class PetState(str, Enum):
|
||||
"""Animation state a pet can be shown in.
|
||||
|
||||
Values are the petdex spritesheet *row names*. Membership maps directly
|
||||
onto :data:`STATE_ROWS` (row index = position in that list).
|
||||
These are Hermes' activity state names. They are not always identical to the
|
||||
source atlas row names: Codex-format pets use rows like ``jumping`` /
|
||||
``running`` while the UI keeps the shorter ``jump`` / ``run`` names.
|
||||
"""
|
||||
|
||||
IDLE = "idle"
|
||||
@ -90,13 +89,12 @@ class PetState(str, Enum):
|
||||
FAILED = "failed"
|
||||
REVIEW = "review"
|
||||
JUMP = "jump"
|
||||
WAITING = "waiting"
|
||||
|
||||
|
||||
# Row order in the spritesheet (top → bottom). Index of a state name here is
|
||||
# the pixel row it occupies: ``row_y = STATE_ROWS.index(state) * FRAME_H``.
|
||||
# ``extra1``/``extra2`` are reserved petdex rows we don't drive yet but keep so
|
||||
# row math stays correct for sheets that include them.
|
||||
STATE_ROWS: list[str] = [
|
||||
# Legacy Hermes/petdex row order (top -> bottom) used by the older 8-row,
|
||||
# 9-column atlas shape.
|
||||
LEGACY_STATE_ROWS: list[str] = [
|
||||
PetState.IDLE.value,
|
||||
PetState.WAVE.value,
|
||||
PetState.RUN.value,
|
||||
@ -107,11 +105,63 @@ STATE_ROWS: list[str] = [
|
||||
"extra2",
|
||||
]
|
||||
|
||||
# Current Petdex row order (top -> bottom) used by 1536x1872 atlases:
|
||||
# 8 columns x 9 rows of 192x208 cells.
|
||||
CODEX_STATE_ROWS: list[str] = [
|
||||
PetState.IDLE.value,
|
||||
"running-right",
|
||||
"running-left",
|
||||
"waving",
|
||||
"jumping",
|
||||
PetState.FAILED.value,
|
||||
PetState.WAITING.value,
|
||||
"running",
|
||||
PetState.REVIEW.value,
|
||||
]
|
||||
|
||||
def state_row_index(state: "PetState | str") -> int:
|
||||
"""Return the spritesheet row index for *state* (clamped, never raises)."""
|
||||
# Default/fallback for callers without a sheet. Prefer the current 9-row Codex
|
||||
# format because generated pets and the public Codex pet contract use it.
|
||||
STATE_ROWS: list[str] = CODEX_STATE_ROWS
|
||||
|
||||
# Canonical Hermes activity names -> accepted row-name aliases in descending
|
||||
# preference. This keeps our internal state names stable (`wave`/`jump`/`run`)
|
||||
# while matching Petdex's current `waving`/`jumping`/`running` taxonomy.
|
||||
STATE_ALIASES: dict[str, tuple[str, ...]] = {
|
||||
PetState.IDLE.value: (PetState.IDLE.value,),
|
||||
PetState.WAVE.value: (PetState.WAVE.value, "waving"),
|
||||
PetState.JUMP.value: (PetState.JUMP.value, "jumping"),
|
||||
PetState.RUN.value: (PetState.RUN.value, "running"),
|
||||
PetState.FAILED.value: (PetState.FAILED.value,),
|
||||
PetState.REVIEW.value: (PetState.REVIEW.value,),
|
||||
PetState.WAITING.value: (PetState.WAITING.value,),
|
||||
}
|
||||
|
||||
|
||||
def state_aliases_for(state: "PetState | str") -> tuple[str, ...]:
|
||||
"""Return accepted row-name aliases for *state* (always non-empty)."""
|
||||
value = state.value if isinstance(state, PetState) else str(state)
|
||||
aliases = STATE_ALIASES.get(value)
|
||||
return aliases if aliases else (value,)
|
||||
|
||||
|
||||
def state_rows_for_grid(row_count: int | None) -> list[str]:
|
||||
"""Return the row taxonomy for a spritesheet with *row_count* rows."""
|
||||
try:
|
||||
return STATE_ROWS.index(value)
|
||||
except ValueError:
|
||||
return 0 # fall back to the idle row
|
||||
rows = int(row_count or 0)
|
||||
except (TypeError, ValueError):
|
||||
rows = 0
|
||||
|
||||
if rows >= len(CODEX_STATE_ROWS):
|
||||
return CODEX_STATE_ROWS
|
||||
return LEGACY_STATE_ROWS
|
||||
|
||||
|
||||
def state_row_index(state: "PetState | str", row_count: int | None = None) -> int:
|
||||
"""Return the spritesheet row index for *state* (clamped, never raises)."""
|
||||
rows = state_rows_for_grid(row_count)
|
||||
for name in state_aliases_for(state):
|
||||
try:
|
||||
return rows.index(name)
|
||||
except ValueError:
|
||||
continue
|
||||
return 0 # fall back to the idle row
|
||||
|
||||
@ -155,7 +155,8 @@ def _raw_frames(
|
||||
try:
|
||||
sheet = _open_sheet(Path(sheet_path))
|
||||
cols = max(1, sheet.width // frame_w)
|
||||
row = state_row_index(state_value)
|
||||
rows = max(1, sheet.height // frame_h)
|
||||
row = state_row_index(state_value, rows)
|
||||
top = row * frame_h
|
||||
# Clamp the row to the sheet (some pets ship fewer rows than the 8 the
|
||||
# taxonomy reserves).
|
||||
|
||||
@ -59,10 +59,8 @@ def derive_pet_state(
|
||||
4. ``tool_running`` → ``RUN`` (a tool is executing)
|
||||
5. ``reasoning`` → ``REVIEW`` (model is thinking / reading)
|
||||
6. ``busy`` → ``RUN`` (turn in flight, unspecified work)
|
||||
7. otherwise → ``IDLE`` (incl. ``awaiting_input``)
|
||||
|
||||
``awaiting_input`` is accepted for symmetry with the surfaces but maps to
|
||||
``IDLE`` — a pet waiting on the user should rest, not run.
|
||||
7. ``awaiting_input`` → ``WAITING`` (the agent is blocked on the user)
|
||||
8. otherwise → ``IDLE``
|
||||
"""
|
||||
if error:
|
||||
return PetState.FAILED
|
||||
@ -76,4 +74,6 @@ def derive_pet_state(
|
||||
return PetState.REVIEW
|
||||
if busy:
|
||||
return PetState.RUN
|
||||
if awaiting_input:
|
||||
return PetState.WAITING
|
||||
return PetState.IDLE
|
||||
|
||||
@ -5155,6 +5155,142 @@ function createNewSessionWindow() {
|
||||
return spawnSecondaryWindow({ newSession: true })
|
||||
}
|
||||
|
||||
// The pet overlay: a single transparent, frameless, always-on-top window that
|
||||
// hosts ONLY the floating mascot. Shift-clicking the in-window pet "pops it out"
|
||||
// here so it can leave the app's bounds and stay visible while Hermes is
|
||||
// minimized (Codex-style task-completion glance). It carries no gateway
|
||||
// connection of its own — the main renderer is the single source of truth and
|
||||
// pushes pet state over IPC (hermes:pet-overlay:state); the overlay just renders
|
||||
// it. Control flows back (pop-in, composer submit) via hermes:pet-overlay:control.
|
||||
let petOverlayWindow = null
|
||||
|
||||
function petOverlayUrl() {
|
||||
if (DEV_SERVER) {
|
||||
return `${DEV_SERVER.endsWith('/') ? DEV_SERVER.slice(0, -1) : DEV_SERVER}/?win=overlay#/`
|
||||
}
|
||||
|
||||
return `${pathToFileURL(resolveRendererIndex()).toString()}?win=overlay#/`
|
||||
}
|
||||
|
||||
function spawnPetOverlayWindow(bounds) {
|
||||
const win = new BrowserWindow({
|
||||
width: Math.max(80, Math.round(bounds?.width || 220)),
|
||||
height: Math.max(80, Math.round(bounds?.height || 220)),
|
||||
x: Number.isFinite(bounds?.x) ? Math.round(bounds.x) : undefined,
|
||||
y: Number.isFinite(bounds?.y) ? Math.round(bounds.y) : undefined,
|
||||
frame: false,
|
||||
transparent: true,
|
||||
resizable: false,
|
||||
movable: true,
|
||||
minimizable: false,
|
||||
maximizable: false,
|
||||
fullscreenable: false,
|
||||
// Windows/Linux need this so the helper window does not get its own
|
||||
// taskbar/alt-tab entry. On macOS, cmd-tab is app-level and this can make
|
||||
// the whole app look like it vanished when the only newly-created visible
|
||||
// window is a frameless overlay. Use NSPanel + Mission Control hiding below
|
||||
// instead, leaving the main Hermes app as the Dock/cmd-tab anchor.
|
||||
skipTaskbar: !IS_MAC,
|
||||
hasShadow: false,
|
||||
alwaysOnTop: true,
|
||||
// macOS panels are non-activating helper windows and can float over full
|
||||
// screen spaces without becoming the app's main switcher window.
|
||||
type: IS_MAC ? 'panel' : undefined,
|
||||
hiddenInMissionControl: IS_MAC,
|
||||
// Non-activating: the overlay must never become the app's key/main window,
|
||||
// or it (a frameless, taskbar-skipping panel) becomes the app's switcher
|
||||
// anchor and the Hermes icon drops out of cmd/alt-tab — especially when the
|
||||
// main window is minimized. We flip this on only while the composer needs
|
||||
// the keyboard (see hermes:pet-overlay:set-focusable).
|
||||
focusable: false,
|
||||
show: false,
|
||||
// Fully transparent — the renderer paints only the sprite + bubble.
|
||||
backgroundColor: '#00000000',
|
||||
webPreferences: {
|
||||
preload: path.join(__dirname, 'preload.cjs'),
|
||||
contextIsolation: true,
|
||||
sandbox: true,
|
||||
nodeIntegration: false,
|
||||
devTools: true,
|
||||
// Keep the sprite animating + bubble updating while the main window is
|
||||
// minimized/blurred — the whole point of the overlay.
|
||||
backgroundThrottling: false
|
||||
}
|
||||
})
|
||||
|
||||
// Float above other apps and follow the user across desktops so the pet is
|
||||
// always reachable. `floating` + `type: panel` is the macOS NSPanel path; the
|
||||
// more aggressive `screen-saver` level can interfere with normal app/window
|
||||
// switching semantics.
|
||||
win.setAlwaysOnTop(true, IS_MAC ? 'floating' : 'screen-saver')
|
||||
win.setHiddenInMissionControl?.(true)
|
||||
try {
|
||||
// Electron docs: macOS may transform process type on each
|
||||
// setVisibleOnAllWorkspaces() call unless skipTransformProcessType=true,
|
||||
// which briefly hides the Dock/cmd-tab presence. Keep Hermes in the normal
|
||||
// ForegroundApplication class so shift-clicking the pet never drops the app
|
||||
// out of app switchers.
|
||||
win.setVisibleOnAllWorkspaces(
|
||||
true,
|
||||
IS_MAC ? { visibleOnFullScreen: true, skipTransformProcessType: true } : undefined
|
||||
)
|
||||
} catch {
|
||||
// Not supported everywhere — best effort.
|
||||
}
|
||||
|
||||
wireCommonWindowHandlers(win)
|
||||
|
||||
win.once('ready-to-show', () => {
|
||||
if (!win.isDestroyed()) win.showInactive()
|
||||
})
|
||||
|
||||
win.on('closed', () => {
|
||||
if (petOverlayWindow === win) {
|
||||
petOverlayWindow = null
|
||||
}
|
||||
|
||||
// If the overlay went away on its own (e.g. ⌘W), tell the main renderer to
|
||||
// pop the pet back in so it doesn't stay hidden. Harmless echo when we're
|
||||
// the ones who closed it (popInPet already cleared the active flag).
|
||||
if (mainWindow && !mainWindow.isDestroyed()) {
|
||||
mainWindow.webContents.send('hermes:pet-overlay:control', { type: 'pop-in' })
|
||||
}
|
||||
})
|
||||
|
||||
win.loadURL(petOverlayUrl())
|
||||
|
||||
return win
|
||||
}
|
||||
|
||||
function openPetOverlay(bounds) {
|
||||
if (petOverlayWindow && !petOverlayWindow.isDestroyed()) {
|
||||
if (bounds) {
|
||||
petOverlayWindow.setBounds({
|
||||
x: Math.round(bounds.x),
|
||||
y: Math.round(bounds.y),
|
||||
width: Math.max(80, Math.round(bounds.width)),
|
||||
height: Math.max(80, Math.round(bounds.height))
|
||||
})
|
||||
}
|
||||
|
||||
petOverlayWindow.showInactive()
|
||||
|
||||
return petOverlayWindow
|
||||
}
|
||||
|
||||
petOverlayWindow = spawnPetOverlayWindow(bounds)
|
||||
|
||||
return petOverlayWindow
|
||||
}
|
||||
|
||||
function closePetOverlay() {
|
||||
if (petOverlayWindow && !petOverlayWindow.isDestroyed()) {
|
||||
petOverlayWindow.close()
|
||||
}
|
||||
|
||||
petOverlayWindow = null
|
||||
}
|
||||
|
||||
function createWindow() {
|
||||
const icon = getAppIconPath()
|
||||
mainWindow = new BrowserWindow({
|
||||
@ -5224,6 +5360,11 @@ function createWindow() {
|
||||
mainWindow.on('will-leave-full-screen', () => sendWindowStateChanged(false))
|
||||
mainWindow.on('leave-full-screen', () => sendWindowStateChanged(false))
|
||||
|
||||
// The overlay rides the main window — closing the app's primary window must
|
||||
// tear it down too (otherwise it strands as an orphan that blocks
|
||||
// window-all-closed from quitting on Windows/Linux).
|
||||
mainWindow.on('closed', () => closePetOverlay())
|
||||
|
||||
wireCommonWindowHandlers(mainWindow)
|
||||
|
||||
mainWindow.webContents.on('render-process-gone', (_event, details) => {
|
||||
@ -5344,6 +5485,102 @@ ipcMain.handle('hermes:window:openNewSession', async () => {
|
||||
|
||||
return { ok: true }
|
||||
})
|
||||
|
||||
// --- Pet overlay (pop-out mascot) -----------------------------------------
|
||||
// `request` is `{ bounds, screen }`. A fresh pop-out passes viewport-space
|
||||
// bounds (screen=false): convert to screen space by adding the main window's
|
||||
// content origin so the pet lands where it sat in-window. A remembered/dragged
|
||||
// spot passes screen-space bounds (screen=true) and is used as-is. We return the
|
||||
// resolved screen bounds so the renderer can persist exactly where it opened.
|
||||
ipcMain.handle('hermes:pet-overlay:open', async (_event, request) => {
|
||||
const bounds = request && request.bounds ? request.bounds : request
|
||||
const isScreen = Boolean(request && request.screen)
|
||||
let screenBounds = bounds
|
||||
|
||||
try {
|
||||
if (bounds && !isScreen && mainWindow && !mainWindow.isDestroyed()) {
|
||||
const content = mainWindow.getContentBounds()
|
||||
screenBounds = {
|
||||
x: content.x + (bounds.x || 0),
|
||||
y: content.y + (bounds.y || 0),
|
||||
width: bounds.width,
|
||||
height: bounds.height
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Fall back to raw bounds if the window geometry is unavailable.
|
||||
}
|
||||
|
||||
openPetOverlay(screenBounds)
|
||||
|
||||
return { ok: true, bounds: screenBounds }
|
||||
})
|
||||
ipcMain.handle('hermes:pet-overlay:close', async () => {
|
||||
closePetOverlay()
|
||||
|
||||
return { ok: true }
|
||||
})
|
||||
// Drag: the overlay reports a new absolute screen position (it already knows the
|
||||
// pointer's screen coords), we just move the window.
|
||||
ipcMain.on('hermes:pet-overlay:set-bounds', (_event, bounds) => {
|
||||
if (!petOverlayWindow || petOverlayWindow.isDestroyed() || !bounds) {
|
||||
return
|
||||
}
|
||||
|
||||
petOverlayWindow.setBounds({
|
||||
x: Math.round(bounds.x),
|
||||
y: Math.round(bounds.y),
|
||||
width: Math.max(80, Math.round(bounds.width)),
|
||||
height: Math.max(80, Math.round(bounds.height))
|
||||
})
|
||||
})
|
||||
// Click-through: the overlay window is a full rectangle but only the pet pixels
|
||||
// should be interactive. The renderer toggles this as the cursor enters/leaves
|
||||
// the sprite so transparent margins pass clicks to whatever is behind.
|
||||
ipcMain.on('hermes:pet-overlay:ignore-mouse', (_event, ignore) => {
|
||||
if (petOverlayWindow && !petOverlayWindow.isDestroyed()) {
|
||||
petOverlayWindow.setIgnoreMouseEvents(Boolean(ignore), { forward: true })
|
||||
}
|
||||
})
|
||||
// The overlay is a non-activating panel (focusable:false) so it never steals
|
||||
// the app's cmd/alt-tab anchor from the main window. But the pop-up composer
|
||||
// needs the keyboard, so the renderer asks us to flip it focusable + focus it
|
||||
// while the composer is open, then back to non-activating when it closes.
|
||||
ipcMain.on('hermes:pet-overlay:set-focusable', (_event, focusable) => {
|
||||
if (!petOverlayWindow || petOverlayWindow.isDestroyed()) {
|
||||
return
|
||||
}
|
||||
|
||||
petOverlayWindow.setFocusable(Boolean(focusable))
|
||||
if (focusable) {
|
||||
petOverlayWindow.focus()
|
||||
}
|
||||
})
|
||||
// Main renderer → overlay: forward the latest pet state for the overlay to render.
|
||||
ipcMain.on('hermes:pet-overlay:state', (_event, payload) => {
|
||||
if (petOverlayWindow && !petOverlayWindow.isDestroyed()) {
|
||||
petOverlayWindow.webContents.send('hermes:pet-overlay:state', payload)
|
||||
}
|
||||
})
|
||||
// Overlay → main renderer: control messages (pop back in, composer submit).
|
||||
ipcMain.on('hermes:pet-overlay:control', (_event, payload) => {
|
||||
if (!mainWindow || mainWindow.isDestroyed()) {
|
||||
return
|
||||
}
|
||||
|
||||
// The mail icon means "take me to the app": raise the main window (it may be
|
||||
// minimized or buried) before the renderer navigates to the latest thread.
|
||||
if (payload && payload.type === 'open-app') {
|
||||
if (mainWindow.isMinimized()) {
|
||||
mainWindow.restore()
|
||||
}
|
||||
|
||||
mainWindow.show()
|
||||
mainWindow.focus()
|
||||
}
|
||||
|
||||
mainWindow.webContents.send('hermes:pet-overlay:control', payload)
|
||||
})
|
||||
ipcMain.handle('hermes:bootstrap:reset', async () => {
|
||||
// Renderer's "Reload and retry" path. Clear the latched failure and
|
||||
// reset connection state so the next startHermes() call restarts the
|
||||
@ -6548,6 +6785,10 @@ function configureSpellChecker() {
|
||||
}
|
||||
|
||||
app.on('before-quit', () => {
|
||||
// The always-on-top overlay isn't a "real" app window; close it so a stray
|
||||
// pet can't keep the process alive or float over a quit app.
|
||||
closePetOverlay()
|
||||
|
||||
// Quitting mid-install should stop the installer, not orphan it.
|
||||
if (bootstrapAbortController) {
|
||||
try {
|
||||
|
||||
@ -7,6 +7,32 @@ contextBridge.exposeInMainWorld('hermesDesktop', {
|
||||
getGatewayWsUrl: profile => ipcRenderer.invoke('hermes:gateway:ws-url', profile),
|
||||
openSessionWindow: (sessionId, opts) => ipcRenderer.invoke('hermes:window:openSession', sessionId, opts),
|
||||
openNewSessionWindow: () => ipcRenderer.invoke('hermes:window:openNewSession'),
|
||||
petOverlay: {
|
||||
// Main renderer → main process: window lifecycle + drag. `request` is
|
||||
// `{ bounds, screen }`; resolves with the screen bounds it actually used.
|
||||
open: request => ipcRenderer.invoke('hermes:pet-overlay:open', request),
|
||||
close: () => ipcRenderer.invoke('hermes:pet-overlay:close'),
|
||||
setBounds: bounds => ipcRenderer.send('hermes:pet-overlay:set-bounds', bounds),
|
||||
setIgnoreMouse: ignore => ipcRenderer.send('hermes:pet-overlay:ignore-mouse', ignore),
|
||||
// Flip the overlay focusable (and focus it) while the composer needs keys.
|
||||
setFocusable: focusable => ipcRenderer.send('hermes:pet-overlay:set-focusable', focusable),
|
||||
// Main renderer → overlay (forwarded by main): push the latest pet state.
|
||||
pushState: payload => ipcRenderer.send('hermes:pet-overlay:state', payload),
|
||||
// Overlay → main renderer (forwarded by main): pop back in / composer submit.
|
||||
control: payload => ipcRenderer.send('hermes:pet-overlay:control', payload),
|
||||
// Overlay subscribes to state pushes.
|
||||
onState: callback => {
|
||||
const listener = (_event, payload) => callback(payload)
|
||||
ipcRenderer.on('hermes:pet-overlay:state', listener)
|
||||
return () => ipcRenderer.removeListener('hermes:pet-overlay:state', listener)
|
||||
},
|
||||
// Main renderer subscribes to overlay control messages.
|
||||
onControl: callback => {
|
||||
const listener = (_event, payload) => callback(payload)
|
||||
ipcRenderer.on('hermes:pet-overlay:control', listener)
|
||||
return () => ipcRenderer.removeListener('hermes:pet-overlay:control', listener)
|
||||
}
|
||||
},
|
||||
getBootProgress: () => ipcRenderer.invoke('hermes:boot-progress:get'),
|
||||
getConnectionConfig: profile => ipcRenderer.invoke('hermes:connection-config:get', profile),
|
||||
saveConnectionConfig: payload => ipcRenderer.invoke('hermes:connection-config:save', payload),
|
||||
|
||||
@ -38,6 +38,7 @@ import {
|
||||
unpinSession
|
||||
} from '../store/layout'
|
||||
import { respondToApprovalAction } from '../store/native-notifications'
|
||||
import { setPetOverlayOpenAppHandler, setPetOverlaySubmitHandler } from '../store/pet-overlay'
|
||||
import { $filePreviewTarget, $previewTarget, closeActiveRightRailTab } from '../store/preview'
|
||||
import {
|
||||
$activeGatewayProfile,
|
||||
@ -786,6 +787,41 @@ export function DesktopController() {
|
||||
updateSessionState
|
||||
})
|
||||
|
||||
// The popped-out pet drives two actions back into the app: send a prompt, and
|
||||
// open the most recent thread. Both are registered ONCE through refs that track
|
||||
// the latest callbacks — re-registering on every `submitText`/`resumeSession`
|
||||
// identity change left a brief window where the handler was nulled (cleanup
|
||||
// before re-register), which could drop a submit fired from the overlay (e.g.
|
||||
// creating a session from the new-session screen). The ref form keeps a stable,
|
||||
// always-current handler. Primary window only — it owns the overlay.
|
||||
const submitTextRef = useRef(submitText)
|
||||
submitTextRef.current = submitText
|
||||
const resumeSessionRef = useRef(resumeSession)
|
||||
resumeSessionRef.current = resumeSession
|
||||
|
||||
useEffect(() => {
|
||||
if (isSecondaryWindow()) {
|
||||
return
|
||||
}
|
||||
|
||||
setPetOverlaySubmitHandler(text => void submitTextRef.current(text))
|
||||
// Mail icon: $sessions is ordered most-recent-first; the pet is global (not
|
||||
// per session) so "most recent" is the right target. main.cjs already raised
|
||||
// the window before forwarding this.
|
||||
setPetOverlayOpenAppHandler(() => {
|
||||
const recent = $sessions.get()[0]
|
||||
|
||||
if (recent?.id) {
|
||||
void resumeSessionRef.current(recent.id)
|
||||
}
|
||||
})
|
||||
|
||||
return () => {
|
||||
setPetOverlaySubmitHandler(null)
|
||||
setPetOverlayOpenAppHandler(null)
|
||||
}
|
||||
}, [])
|
||||
|
||||
useGatewayBoot({
|
||||
handleGatewayEvent: handleDesktopGatewayEvent,
|
||||
onConnectionReady: c => {
|
||||
|
||||
38
apps/desktop/src/app/pet-overlay/overlay-root.tsx
Normal file
38
apps/desktop/src/app/pet-overlay/overlay-root.tsx
Normal file
@ -0,0 +1,38 @@
|
||||
import { StrictMode } from 'react'
|
||||
import { createRoot } from 'react-dom/client'
|
||||
|
||||
import { ErrorBoundary } from '@/components/error-boundary'
|
||||
import { ThemeProvider } from '@/themes/context'
|
||||
|
||||
import { PetOverlayApp } from './pet-overlay-app'
|
||||
|
||||
/**
|
||||
* Boot the pet-overlay window. Loaded by the same bundle as the main app but
|
||||
* via `?win=overlay`, so it shares CSS/atoms while mounting a minimal, transparent
|
||||
* surface (no app shell, no gateway, no I18n — the bubble strings are inline).
|
||||
*
|
||||
* The index.html boot script paints an OPAQUE themed background to avoid a flash
|
||||
* in normal windows; the overlay must be see-through, so we force every host
|
||||
* layer transparent with a late, high-specificity style tag.
|
||||
*/
|
||||
export function mountPetOverlay(): void {
|
||||
const style = document.createElement('style')
|
||||
style.textContent = 'html,body,#root{background:transparent !important;}'
|
||||
document.head.appendChild(style)
|
||||
|
||||
const root = document.getElementById('root')
|
||||
|
||||
if (!root) {
|
||||
return
|
||||
}
|
||||
|
||||
createRoot(root).render(
|
||||
<StrictMode>
|
||||
<ErrorBoundary label="pet-overlay">
|
||||
<ThemeProvider>
|
||||
<PetOverlayApp />
|
||||
</ThemeProvider>
|
||||
</ErrorBoundary>
|
||||
</StrictMode>
|
||||
)
|
||||
}
|
||||
317
apps/desktop/src/app/pet-overlay/pet-overlay-app.tsx
Normal file
317
apps/desktop/src/app/pet-overlay/pet-overlay-app.tsx
Normal file
@ -0,0 +1,317 @@
|
||||
import { useStore } from '@nanostores/react'
|
||||
import { useEffect, useRef, useState } from 'react'
|
||||
|
||||
import { PetBubble } from '@/components/pet/pet-bubble'
|
||||
import { PetSprite } from '@/components/pet/pet-sprite'
|
||||
import { Mail } from '@/lib/icons'
|
||||
import { $petActivity, $petInfo, setPetInfo } from '@/store/pet'
|
||||
import { setAwaitingResponse, setBusy } from '@/store/session'
|
||||
|
||||
/**
|
||||
* The pop-out overlay's only view: a transparent, draggable mascot with a mini
|
||||
* composer.
|
||||
*
|
||||
* This runs in a separate, gateway-less BrowserWindow (`?win=overlay`). It is a
|
||||
* pure puppet — the main renderer pushes the live pet state over IPC and we
|
||||
* mirror it into the same atoms the in-window pet reads, so `PetSprite` /
|
||||
* `PetBubble` render identically with zero extra logic.
|
||||
*
|
||||
* The window is a full rectangle but mostly transparent; we toggle OS-level
|
||||
* mouse click-through so only the sprite (or the open composer) is interactive
|
||||
* and the empty margins pass clicks through to whatever is behind.
|
||||
*
|
||||
* Gestures on the pet: drag to move it anywhere on screen (even outside the
|
||||
* app), shift-click to pop it back into the window, plain click to open a small
|
||||
* composer that sends a prompt to the most recent session. A mail icon (shown
|
||||
* only when a turn finished while you were away) raises the app on that thread.
|
||||
*/
|
||||
|
||||
// Below this much pointer travel, a press counts as a click, not a drag.
|
||||
const CLICK_SLOP_PX = 3
|
||||
|
||||
interface DragState {
|
||||
startX: number
|
||||
startY: number
|
||||
offX: number
|
||||
offY: number
|
||||
width: number
|
||||
height: number
|
||||
moved: boolean
|
||||
}
|
||||
|
||||
export function PetOverlayApp() {
|
||||
const info = useStore($petInfo)
|
||||
const [composerOpen, setComposerOpen] = useState(false)
|
||||
const [draft, setDraft] = useState('')
|
||||
// Mirrored from the main renderer: a finish landed while you were away.
|
||||
const [unread, setUnread] = useState(false)
|
||||
|
||||
const dragRef = useRef<DragState | null>(null)
|
||||
const petRef = useRef<HTMLDivElement | null>(null)
|
||||
const inputRef = useRef<HTMLInputElement | null>(null)
|
||||
const ignoreRef = useRef(true)
|
||||
const composerOpenRef = useRef(false)
|
||||
|
||||
const setIgnore = (ignore: boolean) => {
|
||||
if (ignoreRef.current !== ignore) {
|
||||
ignoreRef.current = ignore
|
||||
window.hermesDesktop?.petOverlay?.setIgnoreMouse(ignore)
|
||||
}
|
||||
}
|
||||
|
||||
// Mirror pushed state into the shared atoms so PetSprite/PetBubble just work.
|
||||
useEffect(() => {
|
||||
const off = window.hermesDesktop?.petOverlay?.onState(payload => {
|
||||
setPetInfo(payload.info)
|
||||
$petActivity.set(payload.activity ?? {})
|
||||
setBusy(Boolean(payload.busy))
|
||||
setAwaitingResponse(Boolean(payload.awaiting))
|
||||
setUnread(Boolean(payload.unread))
|
||||
})
|
||||
|
||||
// Tell the main renderer we're mounted so it pushes the current frame (the
|
||||
// subscribe-time pushes during open() can land before this view exists).
|
||||
window.hermesDesktop?.petOverlay?.control({ type: 'ready' })
|
||||
|
||||
return off
|
||||
}, [])
|
||||
|
||||
// Click-through: make only the sprite (or an open composer) interactive. With
|
||||
// ignore+forward, the renderer still receives mousemove so we can re-enable
|
||||
// hit-testing the moment the cursor returns to the pet.
|
||||
useEffect(() => {
|
||||
setIgnore(true)
|
||||
|
||||
const onMove = (ev: MouseEvent) => {
|
||||
if (dragRef.current || composerOpenRef.current) {
|
||||
setIgnore(false)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
const el = petRef.current
|
||||
|
||||
if (!el) {
|
||||
return
|
||||
}
|
||||
|
||||
const r = el.getBoundingClientRect()
|
||||
const over = ev.clientX >= r.left && ev.clientX <= r.right && ev.clientY >= r.top && ev.clientY <= r.bottom
|
||||
setIgnore(!over)
|
||||
}
|
||||
|
||||
window.addEventListener('mousemove', onMove)
|
||||
|
||||
return () => window.removeEventListener('mousemove', onMove)
|
||||
}, [])
|
||||
|
||||
// The whole window must stay interactive while the composer is open (so the
|
||||
// input keeps focus); focus it on open. The overlay is a non-activating panel
|
||||
// (so it never steals the app's cmd/alt-tab anchor) — flip it focusable while
|
||||
// the composer needs the keyboard, then back to non-activating when it closes.
|
||||
useEffect(() => {
|
||||
composerOpenRef.current = composerOpen
|
||||
|
||||
window.hermesDesktop?.petOverlay?.setFocusable(composerOpen)
|
||||
|
||||
if (composerOpen) {
|
||||
setIgnore(false)
|
||||
// The OS window has to become key first (setFocusable + focus happen in
|
||||
// the main process), so focus the input on the next frame.
|
||||
requestAnimationFrame(() => inputRef.current?.focus())
|
||||
}
|
||||
}, [composerOpen])
|
||||
|
||||
const onPetPointerDown = (e: React.PointerEvent) => {
|
||||
if (e.button !== 0) {
|
||||
return
|
||||
}
|
||||
|
||||
;(e.target as Element).setPointerCapture?.(e.pointerId)
|
||||
dragRef.current = {
|
||||
height: window.outerHeight,
|
||||
moved: false,
|
||||
offX: e.screenX - window.screenX,
|
||||
offY: e.screenY - window.screenY,
|
||||
startX: e.screenX,
|
||||
startY: e.screenY,
|
||||
width: window.outerWidth
|
||||
}
|
||||
}
|
||||
|
||||
const onPetPointerMove = (e: React.PointerEvent) => {
|
||||
const drag = dragRef.current
|
||||
|
||||
if (!drag) {
|
||||
return
|
||||
}
|
||||
|
||||
if (Math.hypot(e.screenX - drag.startX, e.screenY - drag.startY) > CLICK_SLOP_PX) {
|
||||
drag.moved = true
|
||||
}
|
||||
|
||||
window.hermesDesktop?.petOverlay?.setBounds({
|
||||
height: drag.height,
|
||||
width: drag.width,
|
||||
x: e.screenX - drag.offX,
|
||||
y: e.screenY - drag.offY
|
||||
})
|
||||
}
|
||||
|
||||
const onPetPointerUp = (e: React.PointerEvent) => {
|
||||
const drag = dragRef.current
|
||||
dragRef.current = null
|
||||
;(e.target as Element).releasePointerCapture?.(e.pointerId)
|
||||
|
||||
if (!drag) {
|
||||
return
|
||||
}
|
||||
|
||||
if (drag.moved) {
|
||||
// Remember the spot on the desktop (screen coords) so the pet reopens here
|
||||
// next time / after a restart.
|
||||
window.hermesDesktop?.petOverlay?.control({
|
||||
bounds: { height: drag.height, width: drag.width, x: e.screenX - drag.offX, y: e.screenY - drag.offY },
|
||||
type: 'bounds'
|
||||
})
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
// A clean click: shift pops the pet back in; otherwise toggle the composer.
|
||||
if (e.shiftKey) {
|
||||
window.hermesDesktop?.petOverlay?.control({ type: 'pop-in' })
|
||||
} else {
|
||||
setComposerOpen(open => !open)
|
||||
}
|
||||
}
|
||||
|
||||
const send = () => {
|
||||
const text = draft.trim()
|
||||
|
||||
if (text) {
|
||||
window.hermesDesktop?.petOverlay?.control({ text, type: 'submit' })
|
||||
}
|
||||
|
||||
setDraft('')
|
||||
setComposerOpen(false)
|
||||
}
|
||||
|
||||
const openApp = () => {
|
||||
// Hide the icon immediately; the main renderer also clears the source flag.
|
||||
setUnread(false)
|
||||
window.hermesDesktop?.petOverlay?.control({ type: 'open-app' })
|
||||
}
|
||||
|
||||
if (!info.enabled || !info.spritesheetBase64) {
|
||||
return null
|
||||
}
|
||||
|
||||
return (
|
||||
<div
|
||||
onPointerDown={e => {
|
||||
// Click on the transparent backdrop (not the pet/composer) dismisses
|
||||
// the composer.
|
||||
if (composerOpen && e.target === e.currentTarget) {
|
||||
setComposerOpen(false)
|
||||
}
|
||||
}}
|
||||
style={{
|
||||
alignItems: 'center',
|
||||
background: 'transparent',
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
height: '100vh',
|
||||
justifyContent: 'flex-end',
|
||||
paddingBottom: 24,
|
||||
userSelect: 'none',
|
||||
width: '100vw'
|
||||
}}
|
||||
>
|
||||
{composerOpen && (
|
||||
<input
|
||||
onChange={e => setDraft(e.target.value)}
|
||||
onKeyDown={e => {
|
||||
if (e.key === 'Enter' && !e.shiftKey) {
|
||||
e.preventDefault()
|
||||
send()
|
||||
} else if (e.key === 'Escape') {
|
||||
setComposerOpen(false)
|
||||
}
|
||||
}}
|
||||
placeholder="Message…"
|
||||
ref={inputRef}
|
||||
style={{
|
||||
background: 'var(--ui-bg-elevated)',
|
||||
border: '1px solid var(--ui-stroke-secondary)',
|
||||
borderRadius: 10,
|
||||
boxShadow: '0 6px 18px rgba(0,0,0,0.28)',
|
||||
color: 'var(--foreground)',
|
||||
fontSize: 12,
|
||||
marginBottom: 8,
|
||||
outline: 'none',
|
||||
padding: '7px 10px',
|
||||
width: 200
|
||||
}}
|
||||
value={draft}
|
||||
/>
|
||||
)}
|
||||
|
||||
<div
|
||||
onPointerDown={onPetPointerDown}
|
||||
onPointerMove={onPetPointerMove}
|
||||
onPointerUp={onPetPointerUp}
|
||||
ref={petRef}
|
||||
style={{
|
||||
alignItems: 'center',
|
||||
cursor: 'grab',
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
position: 'relative',
|
||||
touchAction: 'none'
|
||||
}}
|
||||
>
|
||||
<div style={{ marginBottom: 4 }}>
|
||||
<PetBubble />
|
||||
</div>
|
||||
<div style={{ lineHeight: 0, position: 'relative' }}>
|
||||
<PetSprite info={info} />
|
||||
|
||||
{/* Mail icon: only when a finish landed while you were away. Jumps to
|
||||
the app's most recent thread. Anchored to the sprite (kept inside
|
||||
its box so the overlay's click-through hit-test still catches it);
|
||||
stopPropagation keeps a click from starting a window drag. */}
|
||||
{unread && (
|
||||
<button
|
||||
aria-label="Open in Hermes"
|
||||
onClick={openApp}
|
||||
onPointerDown={e => e.stopPropagation()}
|
||||
onPointerUp={e => e.stopPropagation()}
|
||||
style={{
|
||||
alignItems: 'center',
|
||||
background: 'var(--ui-bg-elevated)',
|
||||
border: '1px solid var(--ui-stroke-secondary)',
|
||||
borderRadius: 999,
|
||||
boxShadow: '0 4px 14px rgba(0,0,0,0.22)',
|
||||
color: 'var(--foreground)',
|
||||
cursor: 'pointer',
|
||||
display: 'inline-flex',
|
||||
height: 24,
|
||||
justifyContent: 'center',
|
||||
padding: 0,
|
||||
position: 'absolute',
|
||||
right: 0,
|
||||
top: 0,
|
||||
width: 24
|
||||
}}
|
||||
title="Open in Hermes"
|
||||
type="button"
|
||||
>
|
||||
<Mail style={{ height: 13, width: 13 }} />
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@ -33,7 +33,7 @@ import { $gateway } from '@/store/gateway'
|
||||
import { dispatchNativeNotification } from '@/store/native-notifications'
|
||||
import { notify } from '@/store/notifications'
|
||||
import { requestDesktopOnboarding } from '@/store/onboarding'
|
||||
import { flashPetActivity, setPetActivity } from '@/store/pet'
|
||||
import { flashPetActivity, markPetUnread, setPetActivity } from '@/store/pet'
|
||||
import { clearAllPrompts, setApprovalRequest, setSecretRequest, setSudoRequest } from '@/store/prompts'
|
||||
import {
|
||||
setCurrentBranch,
|
||||
@ -50,7 +50,7 @@ import {
|
||||
} from '@/store/session'
|
||||
import { broadcastSessionsChanged } from '@/store/session-sync'
|
||||
import { clearSessionSubagents, pruneDelegateFallbackSubagents, upsertSubagent } from '@/store/subagents'
|
||||
import { $todosBySession, setSessionTodos, todoListActive } from '@/store/todos'
|
||||
import { setSessionTodos } from '@/store/todos'
|
||||
import { recordToolDiff } from '@/store/tool-diffs'
|
||||
import type { RpcEvent } from '@/types/hermes'
|
||||
|
||||
@ -904,11 +904,19 @@ export function useMessageStream({
|
||||
if (isActiveEvent) {
|
||||
setTurnStartedAt(null)
|
||||
|
||||
// Pet beat: celebrate a finished plan, else a clean-finish wave.
|
||||
const todos = $todosBySession.get()[sessionId] ?? []
|
||||
const done = todos.length > 0 && !todoListActive(todos)
|
||||
setPetActivity({ reasoning: false, toolRunning: false })
|
||||
flashPetActivity(done ? { celebrate: true } : { justCompleted: true })
|
||||
// Pet beat: a finished turn always celebrates — go straight to the
|
||||
// jump, never linger on the run/reason pose. One atom update (clears
|
||||
// toolRunning/reasoning AND sets celebrate together) so no stray "run"
|
||||
// frame leaks to the sprite — including the popped-out overlay, which
|
||||
// mirrors each activity change. The jump runs ~2 loops, then settles.
|
||||
flashPetActivity({ celebrate: true, reasoning: false, toolRunning: false }, 2200)
|
||||
|
||||
// Light up the pet's mail icon if the user wasn't looking when the turn
|
||||
// finished — a glanceable "new message" hint on the popped-out overlay.
|
||||
// Cleared when they open the app via the mail icon or refocus the window.
|
||||
if (typeof document !== 'undefined' && !document.hasFocus()) {
|
||||
markPetUnread()
|
||||
}
|
||||
}
|
||||
|
||||
if (payload?.usage) {
|
||||
|
||||
@ -3,8 +3,10 @@ import { useCallback, useEffect, useRef, useState } from 'react'
|
||||
|
||||
import { useGatewayRequest } from '@/app/gateway/hooks/use-gateway-request'
|
||||
import { persistString, storedString } from '@/lib/storage'
|
||||
import { $petInfo, type PetInfo, setPetInfo } from '@/store/pet'
|
||||
import { $petInfo, clearPetUnread, type PetInfo, setPetInfo } from '@/store/pet'
|
||||
import { $petOverlayActive, initPetOverlayBridge, popOutPet, restorePetOverlay } from '@/store/pet-overlay'
|
||||
import { $gatewayState } from '@/store/session'
|
||||
import { isSecondaryWindow } from '@/store/windows'
|
||||
import { useTheme } from '@/themes/context'
|
||||
|
||||
import { PetSprite } from './pet-sprite'
|
||||
@ -71,9 +73,13 @@ export function FloatingPet() {
|
||||
const { resolvedMode } = useTheme()
|
||||
const gatewayState = useStore($gatewayState)
|
||||
const info = useStore($petInfo)
|
||||
const overlayActive = useStore($petOverlayActive)
|
||||
|
||||
const [position, setPosition] = useState<Point>(loadPosition)
|
||||
const containerRef = useRef<HTMLDivElement | null>(null)
|
||||
// The facing mirror lives on the sprite wrapper, not the container, so the
|
||||
// speech bubble (a container child) never renders flipped/backwards.
|
||||
const spriteWrapRef = useRef<HTMLDivElement | null>(null)
|
||||
const petW = (info.frameW ?? 192) * (info.scale ?? 0.33)
|
||||
// Soft contact shadow, sized off the pet so every scale/species grounds the
|
||||
// same way (cf. lairp's per-actor feet ellipse). Lighter on light backgrounds.
|
||||
@ -116,6 +122,42 @@ export function FloatingPet() {
|
||||
}
|
||||
}, [gatewayState, active, requestGateway])
|
||||
|
||||
// Wire the overlay control channel once, only in the primary window — the
|
||||
// pop-out overlay belongs to it (main.cjs positions it against the main
|
||||
// window and routes control messages back to it).
|
||||
useEffect(() => {
|
||||
if (isSecondaryWindow()) {
|
||||
return
|
||||
}
|
||||
|
||||
return initPetOverlayBridge()
|
||||
}, [])
|
||||
|
||||
// Returning to the app (by any route, not just the mail icon) clears the pet's
|
||||
// "new message" hint — you've seen it now.
|
||||
useEffect(() => {
|
||||
if (isSecondaryWindow()) {
|
||||
return
|
||||
}
|
||||
|
||||
const onFocus = () => clearPetUnread()
|
||||
window.addEventListener('focus', onFocus)
|
||||
|
||||
return () => window.removeEventListener('focus', onFocus)
|
||||
}, [])
|
||||
|
||||
// Restore a popped-out pet on boot, once the pet has loaded (so we never spawn
|
||||
// an empty overlay window). Primary window only; runs at most once.
|
||||
const restoredRef = useRef(false)
|
||||
useEffect(() => {
|
||||
if (isSecondaryWindow() || restoredRef.current || !active) {
|
||||
return
|
||||
}
|
||||
|
||||
restoredRef.current = true
|
||||
restorePetOverlay()
|
||||
}, [active])
|
||||
|
||||
// A window resize must never strand the pet off-screen — re-clamp the
|
||||
// committed position (and persist it) whenever the viewport shrinks.
|
||||
useEffect(() => {
|
||||
@ -145,6 +187,17 @@ export function FloatingPet() {
|
||||
}
|
||||
|
||||
const rect = el.getBoundingClientRect()
|
||||
|
||||
// Shift-click pops the pet out into a free-floating desktop overlay (it can
|
||||
// leave the window and stays visible while Hermes is minimized) instead of
|
||||
// starting an in-window drag. Primary window only — the overlay is anchored
|
||||
// to it.
|
||||
if (e.shiftKey && !isSecondaryWindow()) {
|
||||
popOutPet({ height: rect.height, width: rect.width, x: rect.left, y: rect.top })
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
dragRef.current = { dx: e.clientX - rect.left, dy: e.clientY - rect.top, x: rect.left, y: rect.top }
|
||||
el.setPointerCapture(e.pointerId)
|
||||
el.style.cursor = 'grabbing'
|
||||
@ -163,10 +216,14 @@ export function FloatingPet() {
|
||||
drag.x = next.x
|
||||
drag.y = next.y
|
||||
// Mutate the DOM directly — no setState, so no re-render while dragging. The
|
||||
// mirror follows the pointer across the midline for the same reason.
|
||||
// mirror follows the pointer across the midline for the same reason; it
|
||||
// rides the sprite wrapper so the bubble stays upright.
|
||||
el.style.left = `${next.x}px`
|
||||
el.style.top = `${next.y}px`
|
||||
el.style.transform = facing(next.x, petW)
|
||||
|
||||
if (spriteWrapRef.current) {
|
||||
spriteWrapRef.current.style.transform = facing(next.x, petW)
|
||||
}
|
||||
},
|
||||
[petW]
|
||||
)
|
||||
@ -189,7 +246,9 @@ export function FloatingPet() {
|
||||
}
|
||||
}, [])
|
||||
|
||||
if (!info.enabled || !info.spritesheetBase64) {
|
||||
// While popped out, the desktop overlay window owns the mascot — hide the
|
||||
// in-window one so there aren't two.
|
||||
if (!info.enabled || !info.spritesheetBase64 || overlayActive) {
|
||||
return null
|
||||
}
|
||||
|
||||
@ -206,11 +265,9 @@ export function FloatingPet() {
|
||||
position: 'fixed',
|
||||
top: position.y,
|
||||
touchAction: 'none',
|
||||
transform: facing(position.x, petW),
|
||||
userSelect: 'none',
|
||||
zIndex: 60
|
||||
}}
|
||||
title={info.displayName || 'pet'}
|
||||
>
|
||||
<div
|
||||
aria-hidden
|
||||
@ -226,7 +283,7 @@ export function FloatingPet() {
|
||||
zIndex: 0
|
||||
}}
|
||||
/>
|
||||
<div style={{ lineHeight: 0, position: 'relative', zIndex: 1 }}>
|
||||
<div ref={spriteWrapRef} style={{ lineHeight: 0, position: 'relative', transform: facing(position.x, petW), zIndex: 1 }}>
|
||||
<PetSprite info={info} />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
126
apps/desktop/src/components/pet/pet-bubble.tsx
Normal file
126
apps/desktop/src/components/pet/pet-bubble.tsx
Normal file
@ -0,0 +1,126 @@
|
||||
import { useStore } from '@nanostores/react'
|
||||
import { useEffect, useMemo, useState } from 'react'
|
||||
|
||||
import { AlertCircle, Clock, type IconComponent } from '@/lib/icons'
|
||||
import { $petActivity, $petState, type PetState } from '@/store/pet'
|
||||
|
||||
/**
|
||||
* Speech bubble + status glyph for the popped-out pet overlay — the
|
||||
* "notification" half of the mascot. It externalizes what the agent is doing
|
||||
* (Codex-style) so a glance at the desktop pet replaces switching back to the
|
||||
* window. The in-window pet doesn't show it (the app itself is the surface);
|
||||
* only the overlay renders it.
|
||||
*
|
||||
* Text is derived purely from the same `$petState` / `$petActivity` the sprite
|
||||
* already reacts to, so it never drifts from the animation. The bubble is shown
|
||||
* only when there's something worth saying (working / reviewing / a transient
|
||||
* done/error beat / waiting on the user) and is hidden at plain idle.
|
||||
*/
|
||||
|
||||
interface Bubble {
|
||||
/** Optional — a glyph-only bubble collapses to a badge. */
|
||||
text?: string
|
||||
glyph?: IconComponent
|
||||
/** Tone → glyph color. Text stays neutral for legibility. */
|
||||
tone?: 'error' | 'wait'
|
||||
}
|
||||
|
||||
// A couple of phrasings per working state, rotated for a touch of life.
|
||||
const WORKING_LINES = ['working…', 'on it…', 'crunching…']
|
||||
const REVIEW_LINES = ['thinking…', 'reading…', 'reviewing…']
|
||||
|
||||
function bubbleFor(state: PetState, awaitingInput: boolean, tick: number): Bubble | null {
|
||||
switch (state) {
|
||||
// Finish beats are carried by the sprite/mail icon now; no extra done badge.
|
||||
case 'jump':
|
||||
case 'wave':
|
||||
return null
|
||||
|
||||
case 'failed':
|
||||
return { text: 'hit a snag', glyph: AlertCircle, tone: 'error' }
|
||||
|
||||
case 'run':
|
||||
return { text: WORKING_LINES[tick % WORKING_LINES.length] }
|
||||
|
||||
case 'review':
|
||||
return { text: REVIEW_LINES[tick % REVIEW_LINES.length] }
|
||||
|
||||
case 'waiting':
|
||||
return { text: 'your turn', glyph: Clock, tone: 'wait' }
|
||||
|
||||
default:
|
||||
// Idle: only speak up if the agent is blocked waiting on the user.
|
||||
return awaitingInput ? { text: 'your turn', glyph: Clock, tone: 'wait' } : null
|
||||
}
|
||||
}
|
||||
|
||||
const TONE_COLOR: Record<NonNullable<Bubble['tone']>, string> = {
|
||||
error: 'var(--ui-red)',
|
||||
wait: 'var(--ui-yellow)'
|
||||
}
|
||||
|
||||
export function PetBubble() {
|
||||
const state = useStore($petState)
|
||||
const activity = useStore($petActivity)
|
||||
const [tick, setTick] = useState(0)
|
||||
|
||||
const rotating = state === 'run' || state === 'review'
|
||||
|
||||
// Advance the phrasing while the agent keeps working; reset when it stops so
|
||||
// the next working spell starts on the first line.
|
||||
useEffect(() => {
|
||||
if (!rotating) {
|
||||
setTick(0)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
const id = window.setInterval(() => setTick(t => t + 1), 2600)
|
||||
|
||||
return () => window.clearInterval(id)
|
||||
}, [rotating])
|
||||
|
||||
const stateBubble = useMemo(
|
||||
() => bubbleFor(state, Boolean(activity.awaitingInput), tick),
|
||||
[state, activity.awaitingInput, tick]
|
||||
)
|
||||
const bubble: Bubble | null = stateBubble
|
||||
|
||||
if (!bubble) {
|
||||
return null
|
||||
}
|
||||
|
||||
const Glyph = bubble.glyph
|
||||
const hasText = Boolean(bubble.text)
|
||||
|
||||
return (
|
||||
<div
|
||||
style={{
|
||||
alignItems: 'center',
|
||||
// Solid, theme-driven surface (the prior --ui-bg-card mixes in
|
||||
// `transparent`, so the bubble was see-through).
|
||||
background: 'var(--ui-bg-elevated)',
|
||||
border: '1px solid var(--ui-stroke-secondary)',
|
||||
borderRadius: hasText ? 10 : 999,
|
||||
boxShadow: '0 4px 14px rgba(0,0,0,0.22)',
|
||||
color: 'var(--foreground)',
|
||||
display: 'inline-flex',
|
||||
fontSize: 11,
|
||||
fontWeight: 500,
|
||||
gap: hasText ? 5 : 0,
|
||||
lineHeight: 1,
|
||||
// Glyph-only bubbles collapse to a tight, symmetric badge.
|
||||
padding: hasText ? '5px 8px' : 5,
|
||||
pointerEvents: 'none',
|
||||
whiteSpace: 'nowrap'
|
||||
}}
|
||||
>
|
||||
{Glyph && (
|
||||
<span style={{ display: 'inline-flex' }}>
|
||||
<Glyph style={{ color: bubble.tone ? TONE_COLOR[bubble.tone] : 'currentColor', height: 13, width: 13 }} />
|
||||
</span>
|
||||
)}
|
||||
{bubble.text}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@ -9,7 +9,28 @@ const DEFAULT_LOOP_MS = 1100
|
||||
// Mirrors agent.pet.constants.DEFAULT_SCALE — fallback only; the gateway sends
|
||||
// the configured scale.
|
||||
const DEFAULT_SCALE = 0.33
|
||||
const DEFAULT_STATE_ROWS = ['idle', 'wave', 'run', 'failed', 'review', 'jump', 'extra1', 'extra2']
|
||||
// Mirrors agent.pet.constants.CODEX_STATE_ROWS (Petdex current taxonomy).
|
||||
const DEFAULT_STATE_ROWS = [
|
||||
'idle',
|
||||
'running-right',
|
||||
'running-left',
|
||||
'waving',
|
||||
'jumping',
|
||||
'failed',
|
||||
'waiting',
|
||||
'running',
|
||||
'review'
|
||||
]
|
||||
|
||||
const STATE_ALIASES: Record<PetState, string[]> = {
|
||||
idle: ['idle'],
|
||||
wave: ['wave', 'waving'],
|
||||
jump: ['jump', 'jumping'],
|
||||
run: ['run', 'running'],
|
||||
failed: ['failed'],
|
||||
review: ['review'],
|
||||
waiting: ['waiting']
|
||||
}
|
||||
|
||||
interface PetSpriteProps {
|
||||
info: PetInfo
|
||||
@ -80,10 +101,14 @@ function PetSpriteImpl({ info, zoom = 1 }: PetSpriteProps) {
|
||||
let drawnFrame = -1
|
||||
let drawnRow = -1
|
||||
|
||||
const rowIndex = (s: string) => {
|
||||
const idx = rows.indexOf(s)
|
||||
|
||||
return idx >= 0 ? idx : 0
|
||||
const rowIndexForState = (s: PetState): number => {
|
||||
for (const key of STATE_ALIASES[s] ?? [s]) {
|
||||
const idx = rows.indexOf(key)
|
||||
if (idx >= 0) {
|
||||
return idx
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// Resolve a state to the row it draws and its real frame count. A state
|
||||
@ -92,10 +117,10 @@ function PetSpriteImpl({ info, zoom = 1 }: PetSpriteProps) {
|
||||
const resolve = (s: PetState): { row: number; count: number } => {
|
||||
const real = framesByState?.[s] ?? frames
|
||||
if (real > 0) {
|
||||
return { row: rowIndex(s), count: real }
|
||||
return { row: rowIndexForState(s), count: real }
|
||||
}
|
||||
|
||||
return { row: rowIndex('idle'), count: Math.max(1, framesByState?.idle ?? frames) }
|
||||
return { row: rowIndexForState('idle'), count: Math.max(1, framesByState?.idle ?? frames) }
|
||||
}
|
||||
|
||||
const render = (now: number) => {
|
||||
|
||||
21
apps/desktop/src/global.d.ts
vendored
21
apps/desktop/src/global.d.ts
vendored
@ -1,3 +1,10 @@
|
||||
import type {
|
||||
PetOverlayBounds,
|
||||
PetOverlayControl,
|
||||
PetOverlayOpenRequest,
|
||||
PetOverlayStatePayload
|
||||
} from './store/pet-overlay'
|
||||
|
||||
export {}
|
||||
|
||||
declare global {
|
||||
@ -26,6 +33,20 @@ declare global {
|
||||
openSessionWindow: (sessionId: string, opts?: { watch?: boolean }) => Promise<{ ok: boolean; error?: string }>
|
||||
// Open (or focus) a compact secondary window on the new-session draft.
|
||||
openNewSessionWindow: () => Promise<{ ok: boolean; error?: string }>
|
||||
// The pop-out pet overlay: a transparent always-on-top window hosting only
|
||||
// the mascot. The main renderer drives it (open/close/drag + state push);
|
||||
// the overlay sends control messages back (pop-in, composer submit).
|
||||
petOverlay: {
|
||||
open: (request: PetOverlayOpenRequest) => Promise<{ ok: boolean; bounds?: PetOverlayBounds }>
|
||||
close: () => Promise<{ ok: boolean }>
|
||||
setBounds: (bounds: PetOverlayBounds) => void
|
||||
setIgnoreMouse: (ignore: boolean) => void
|
||||
setFocusable: (focusable: boolean) => void
|
||||
pushState: (payload: PetOverlayStatePayload) => void
|
||||
control: (payload: PetOverlayControl) => void
|
||||
onState: (callback: (payload: PetOverlayStatePayload) => void) => () => void
|
||||
onControl: (callback: (payload: PetOverlayControl) => void) => () => void
|
||||
}
|
||||
getBootProgress: () => Promise<DesktopBootProgress>
|
||||
getConnectionConfig: (profile?: null | string) => Promise<DesktopConnectionConfig>
|
||||
saveConnectionConfig: (payload: DesktopConnectionConfigInput) => Promise<DesktopConnectionConfig>
|
||||
|
||||
@ -51,6 +51,7 @@ import {
|
||||
IconLoader2 as Loader2Icon,
|
||||
IconLock as Lock,
|
||||
IconLogin as LogIn,
|
||||
IconMail as Mail,
|
||||
IconMessageCircle as MessageCircle,
|
||||
IconMessage2 as MessageSquareText,
|
||||
IconMicrophone as Mic,
|
||||
@ -154,6 +155,7 @@ export {
|
||||
Loader2Icon,
|
||||
Lock,
|
||||
LogIn,
|
||||
Mail,
|
||||
MessageCircle,
|
||||
MessageSquareText,
|
||||
Mic,
|
||||
|
||||
@ -26,20 +26,27 @@ if (import.meta.env.MODE !== 'production') {
|
||||
import('./app/chat/perf-probe')
|
||||
}
|
||||
|
||||
createRoot(document.getElementById('root')!).render(
|
||||
<StrictMode>
|
||||
<ErrorBoundary label="root">
|
||||
<QueryClientProvider client={queryClient}>
|
||||
<I18nProvider>
|
||||
<ThemeProvider>
|
||||
<HapticsProvider>
|
||||
<HashRouter>
|
||||
<App />
|
||||
</HashRouter>
|
||||
</HapticsProvider>
|
||||
</ThemeProvider>
|
||||
</I18nProvider>
|
||||
</QueryClientProvider>
|
||||
</ErrorBoundary>
|
||||
</StrictMode>
|
||||
)
|
||||
// The pet overlay rides this same bundle (`?win=overlay`) but mounts a tiny,
|
||||
// transparent, gateway-less surface instead of the full app. Branch before any
|
||||
// app-shell work so the overlay window stays cheap.
|
||||
if (new URLSearchParams(window.location.search).get('win') === 'overlay') {
|
||||
void import('./app/pet-overlay/overlay-root').then(({ mountPetOverlay }) => mountPetOverlay())
|
||||
} else {
|
||||
createRoot(document.getElementById('root')!).render(
|
||||
<StrictMode>
|
||||
<ErrorBoundary label="root">
|
||||
<QueryClientProvider client={queryClient}>
|
||||
<I18nProvider>
|
||||
<ThemeProvider>
|
||||
<HapticsProvider>
|
||||
<HashRouter>
|
||||
<App />
|
||||
</HashRouter>
|
||||
</HapticsProvider>
|
||||
</ThemeProvider>
|
||||
</I18nProvider>
|
||||
</QueryClientProvider>
|
||||
</ErrorBoundary>
|
||||
</StrictMode>
|
||||
)
|
||||
}
|
||||
|
||||
259
apps/desktop/src/store/pet-overlay.ts
Normal file
259
apps/desktop/src/store/pet-overlay.ts
Normal file
@ -0,0 +1,259 @@
|
||||
import { atom } from 'nanostores'
|
||||
|
||||
import { persistBoolean, persistString, storedBoolean, storedString } from '@/lib/storage'
|
||||
import { $petActivity, $petInfo, $petUnread, clearPetUnread, type PetActivity, type PetInfo } from '@/store/pet'
|
||||
import { $awaitingResponse, $busy } from '@/store/session'
|
||||
|
||||
/**
|
||||
* Controller for the pop-out pet overlay (main-renderer side).
|
||||
*
|
||||
* Shift-clicking the in-window pet "pops it out" into a transparent,
|
||||
* always-on-top OS window (created in electron/main.cjs) that can leave the
|
||||
* app's bounds and stays visible while Hermes is minimized. That window carries
|
||||
* NO gateway connection — this renderer remains the single source of truth and
|
||||
* pushes the live pet state to it over IPC. Control flows back (pop the pet back
|
||||
* in, submit a composer message) via `onControl`.
|
||||
*
|
||||
* The overlay renders the same `PetSprite` / `PetBubble` as the in-window pet by
|
||||
* mirroring the four reactive inputs of `$petState` (`$petInfo`, `$petActivity`,
|
||||
* `$busy`, `$awaitingResponse`) into its own copies of those atoms — so the
|
||||
* popped-out mascot is pixel-identical and needs zero bespoke render logic.
|
||||
*/
|
||||
|
||||
export interface PetOverlayBounds {
|
||||
x: number
|
||||
y: number
|
||||
width: number
|
||||
height: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Request to open the overlay window. `screen` says whether `bounds` are already
|
||||
* in absolute screen coordinates (a remembered/dragged spot) or in the main
|
||||
* window's viewport space (a fresh shift-click pop-out, which main.cjs converts
|
||||
* by adding the content origin).
|
||||
*/
|
||||
export interface PetOverlayOpenRequest {
|
||||
bounds: PetOverlayBounds
|
||||
screen?: boolean
|
||||
}
|
||||
|
||||
/** Everything the overlay needs to reproduce the live mascot. */
|
||||
export interface PetOverlayStatePayload {
|
||||
info: PetInfo
|
||||
activity: PetActivity
|
||||
busy: boolean
|
||||
awaiting: boolean
|
||||
/** Drives the overlay's mail icon: a finish landed while you were away. */
|
||||
unread: boolean
|
||||
}
|
||||
|
||||
export type PetOverlayControl =
|
||||
| { type: 'pop-in' }
|
||||
| { type: 'ready' }
|
||||
| { type: 'submit'; text: string }
|
||||
| { type: 'bounds'; bounds: PetOverlayBounds }
|
||||
| { type: 'open-app' }
|
||||
|
||||
// Persisted across restarts: was the pet popped out, and where on the desktop
|
||||
// did the user leave it. Keyed v1; bump if the bounds shape ever changes.
|
||||
const OVERLAY_ACTIVE_KEY = 'hermes.desktop.pet-overlay-active.v1'
|
||||
const OVERLAY_BOUNDS_KEY = 'hermes.desktop.pet-overlay-bounds.v1'
|
||||
|
||||
export const $petOverlayActive = atom(storedBoolean(OVERLAY_ACTIVE_KEY, false))
|
||||
|
||||
// Persist the in/out choice so a popped-out pet comes back popped out.
|
||||
$petOverlayActive.subscribe(active => persistBoolean(OVERLAY_ACTIVE_KEY, active))
|
||||
|
||||
function loadSavedBounds(): null | PetOverlayBounds {
|
||||
try {
|
||||
const raw = storedString(OVERLAY_BOUNDS_KEY)
|
||||
|
||||
if (!raw) {
|
||||
return null
|
||||
}
|
||||
|
||||
const parsed = JSON.parse(raw) as Partial<PetOverlayBounds>
|
||||
|
||||
if (
|
||||
typeof parsed.x === 'number' &&
|
||||
typeof parsed.y === 'number' &&
|
||||
typeof parsed.width === 'number' &&
|
||||
typeof parsed.height === 'number'
|
||||
) {
|
||||
return { height: parsed.height, width: parsed.width, x: parsed.x, y: parsed.y }
|
||||
}
|
||||
} catch {
|
||||
// fall through to null
|
||||
}
|
||||
|
||||
return null
|
||||
}
|
||||
|
||||
function saveBounds(bounds: PetOverlayBounds): void {
|
||||
persistString(OVERLAY_BOUNDS_KEY, JSON.stringify(bounds))
|
||||
}
|
||||
|
||||
// The overlay window is padded around the sprite so the bubble (above), the
|
||||
// drag area, and the pop-up composer all have room; the pet sits near the
|
||||
// bottom and the rest of the rectangle is transparent + click-through.
|
||||
const OVERLAY_PAD_X = 100
|
||||
const OVERLAY_PAD_Y = 200
|
||||
const OVERLAY_MIN_W = 240
|
||||
const OVERLAY_MIN_H = 300
|
||||
|
||||
let stateUnsubs: Array<() => void> = []
|
||||
let controlUnsub: (() => void) | null = null
|
||||
let submitHandler: ((text: string) => void) | null = null
|
||||
let openAppHandler: (() => void) | null = null
|
||||
|
||||
function currentPayload(): PetOverlayStatePayload {
|
||||
return {
|
||||
info: $petInfo.get(),
|
||||
activity: $petActivity.get(),
|
||||
busy: $busy.get(),
|
||||
awaiting: $awaitingResponse.get(),
|
||||
unread: $petUnread.get()
|
||||
}
|
||||
}
|
||||
|
||||
function pushNow(): void {
|
||||
window.hermesDesktop?.petOverlay?.pushState(currentPayload())
|
||||
}
|
||||
|
||||
/**
|
||||
* Open the overlay window and start mirroring live state into it. The main
|
||||
* process echoes back the actual screen bounds it used, which we persist so the
|
||||
* pet reopens exactly where the user left it.
|
||||
*/
|
||||
function openOverlay(request: PetOverlayOpenRequest): void {
|
||||
const api = window.hermesDesktop?.petOverlay
|
||||
|
||||
if (!api || stateUnsubs.length) {
|
||||
return
|
||||
}
|
||||
|
||||
$petOverlayActive.set(true)
|
||||
void api.open(request).then(res => {
|
||||
if (res?.bounds) {
|
||||
saveBounds(res.bounds)
|
||||
}
|
||||
|
||||
pushNow()
|
||||
})
|
||||
|
||||
// Mirror live state into the overlay. subscribe() fires immediately, so the
|
||||
// overlay also gets a first frame the moment it's ready (it asks via 'ready').
|
||||
stateUnsubs = [
|
||||
$petInfo.subscribe(pushNow),
|
||||
$petActivity.subscribe(pushNow),
|
||||
$busy.subscribe(pushNow),
|
||||
$awaitingResponse.subscribe(pushNow),
|
||||
$petUnread.subscribe(pushNow)
|
||||
]
|
||||
}
|
||||
|
||||
/**
|
||||
* Pop the pet out of the window. `petRect` is the in-window sprite's viewport
|
||||
* rect; we grow it to the padded overlay size and center the window on the
|
||||
* pet's old spot (main.cjs adds the window's screen origin). If the user has
|
||||
* popped out before, reopen at that remembered desktop spot instead.
|
||||
*/
|
||||
export function popOutPet(petRect: PetOverlayBounds): void {
|
||||
if ($petOverlayActive.get() || stateUnsubs.length) {
|
||||
return
|
||||
}
|
||||
|
||||
const saved = loadSavedBounds()
|
||||
|
||||
if (saved) {
|
||||
openOverlay({ bounds: saved, screen: true })
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
const width = Math.max(OVERLAY_MIN_W, Math.round(petRect.width + OVERLAY_PAD_X))
|
||||
const height = Math.max(OVERLAY_MIN_H, Math.round(petRect.height + OVERLAY_PAD_Y))
|
||||
const x = Math.round(petRect.x - (width - petRect.width) / 2)
|
||||
const y = Math.round(petRect.y - (height - petRect.height) / 2)
|
||||
|
||||
openOverlay({ bounds: { height, width, x, y }, screen: false })
|
||||
}
|
||||
|
||||
/**
|
||||
* Restore the overlay on boot if the pet was popped out when the app last
|
||||
* closed. Requires a remembered desktop spot — without one we fall back to the
|
||||
* in-window pet rather than spawning an orphan window at the origin.
|
||||
*/
|
||||
export function restorePetOverlay(): void {
|
||||
if (!window.hermesDesktop?.petOverlay || !$petOverlayActive.get() || stateUnsubs.length) {
|
||||
return
|
||||
}
|
||||
|
||||
const saved = loadSavedBounds()
|
||||
|
||||
if (!saved) {
|
||||
$petOverlayActive.set(false)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
openOverlay({ bounds: saved, screen: true })
|
||||
}
|
||||
|
||||
/** Pop the pet back into the window (closes the overlay window). */
|
||||
export function popInPet(): void {
|
||||
for (const off of stateUnsubs) {
|
||||
off()
|
||||
}
|
||||
|
||||
stateUnsubs = []
|
||||
$petOverlayActive.set(false)
|
||||
void window.hermesDesktop?.petOverlay?.close()
|
||||
}
|
||||
|
||||
/** Register the handler that turns an overlay composer submit into a real send. */
|
||||
export function setPetOverlaySubmitHandler(fn: ((text: string) => void) | null): void {
|
||||
submitHandler = fn
|
||||
}
|
||||
|
||||
/** Register the handler that opens the app to the most recent thread (mail icon). */
|
||||
export function setPetOverlayOpenAppHandler(fn: (() => void) | null): void {
|
||||
openAppHandler = fn
|
||||
}
|
||||
|
||||
/**
|
||||
* Wire the overlay→renderer control channel once. Returns a disposer. Idempotent
|
||||
* — a second call while already wired is a no-op.
|
||||
*/
|
||||
export function initPetOverlayBridge(): () => void {
|
||||
const api = window.hermesDesktop?.petOverlay
|
||||
|
||||
if (!api || controlUnsub) {
|
||||
return () => {}
|
||||
}
|
||||
|
||||
controlUnsub = api.onControl(payload => {
|
||||
if (payload?.type === 'pop-in') {
|
||||
popInPet()
|
||||
} else if (payload?.type === 'ready') {
|
||||
// The overlay just mounted — hand it the current frame.
|
||||
pushNow()
|
||||
} else if (payload?.type === 'submit' && typeof payload.text === 'string') {
|
||||
submitHandler?.(payload.text)
|
||||
} else if (payload?.type === 'bounds' && payload.bounds) {
|
||||
// The user dragged the overlay to a new desktop spot — remember it.
|
||||
saveBounds(payload.bounds)
|
||||
} else if (payload?.type === 'open-app') {
|
||||
// Mail icon: surface the app on the most recent thread (main.cjs already
|
||||
// focused the window before forwarding this) and mark it read.
|
||||
clearPetUnread()
|
||||
openAppHandler?.()
|
||||
}
|
||||
})
|
||||
|
||||
return () => {
|
||||
controlUnsub?.()
|
||||
controlUnsub = null
|
||||
}
|
||||
}
|
||||
@ -1,11 +1,11 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
import { derivePetState } from './pet'
|
||||
import { $petActivity, $petState, derivePetState, flashPetActivity, setPetActivity } from './pet'
|
||||
|
||||
describe('derivePetState', () => {
|
||||
it('rests at idle by default and while awaiting input', () => {
|
||||
it('rests at idle by default and uses waiting when awaiting input', () => {
|
||||
expect(derivePetState({})).toBe('idle')
|
||||
expect(derivePetState({ awaitingInput: true })).toBe('idle')
|
||||
expect(derivePetState({ awaitingInput: true })).toBe('waiting')
|
||||
})
|
||||
|
||||
it('runs when busy or a tool is executing', () => {
|
||||
@ -25,3 +25,18 @@ describe('derivePetState', () => {
|
||||
expect(derivePetState({ justCompleted: true, toolRunning: true })).toBe('wave')
|
||||
})
|
||||
})
|
||||
|
||||
describe('flashPetActivity', () => {
|
||||
it('clears stale sibling beats so a completion never inherits a prior error', () => {
|
||||
// A turn errors (sad), then the next turn finishes cleanly. The celebrate
|
||||
// beat must win — error is highest priority, so a merge-only flash would
|
||||
// keep the pet on the failed pose.
|
||||
setPetActivity({ error: true })
|
||||
flashPetActivity({ celebrate: true })
|
||||
|
||||
expect($petActivity.get().error).toBe(false)
|
||||
expect($petState.get()).toBe('jump')
|
||||
|
||||
setPetActivity({})
|
||||
})
|
||||
})
|
||||
|
||||
@ -11,7 +11,7 @@ import { $awaitingResponse, $busy } from '@/store/session'
|
||||
* `agent/pet/state.py` so the Python and TS surfaces never drift.
|
||||
*/
|
||||
|
||||
export type PetState = 'idle' | 'wave' | 'run' | 'failed' | 'review' | 'jump'
|
||||
export type PetState = 'idle' | 'wave' | 'run' | 'failed' | 'review' | 'jump' | 'waiting'
|
||||
|
||||
export interface PetInfo {
|
||||
enabled: boolean
|
||||
@ -45,7 +45,7 @@ export interface PetActivity {
|
||||
* Resolve the animation state from coarse activity signals.
|
||||
*
|
||||
* Priority (highest first) mirrors `agent.pet.state.derive_pet_state`:
|
||||
* error → celebrate → justCompleted → toolRunning → reasoning → busy → idle.
|
||||
* error → celebrate → justCompleted → toolRunning → reasoning → busy → awaitingInput → idle.
|
||||
*/
|
||||
export function derivePetState(activity: PetActivity): PetState {
|
||||
if (activity.error) {
|
||||
@ -72,12 +72,26 @@ export function derivePetState(activity: PetActivity): PetState {
|
||||
return 'run'
|
||||
}
|
||||
|
||||
if (activity.awaitingInput) {
|
||||
return 'waiting'
|
||||
}
|
||||
|
||||
return 'idle'
|
||||
}
|
||||
|
||||
export const $petInfo = atom<PetInfo>({ enabled: false })
|
||||
export const $petActivity = atom<PetActivity>({})
|
||||
|
||||
/**
|
||||
* Pet-local "you have a new message" flag, surfaced as the overlay's mail icon.
|
||||
* Deliberately not real unread tracking: it flips on when a turn finishes while
|
||||
* the app isn't focused, and off when the user opens the app via the mail icon
|
||||
* (or returns to the window). No persistence — it's a glance hint, not state.
|
||||
*/
|
||||
export const $petUnread = atom(false)
|
||||
export const markPetUnread = () => $petUnread.set(true)
|
||||
export const clearPetUnread = () => $petUnread.set(false)
|
||||
|
||||
/** Steady activity flags (toolRunning / reasoning) set + cleared by the stream. */
|
||||
export const setPetActivity = (next: Partial<PetActivity>) =>
|
||||
$petActivity.set({ ...$petActivity.get(), ...next })
|
||||
@ -85,9 +99,14 @@ export const setPetActivity = (next: Partial<PetActivity>) =>
|
||||
let flashTimer: ReturnType<typeof setTimeout> | undefined
|
||||
|
||||
/** Fire a transient reaction beat (error / celebrate / justCompleted) that
|
||||
* decays back to the steady state after `ms`. */
|
||||
* decays back to the steady state after `ms`.
|
||||
*
|
||||
* Each beat first clears its siblings so a stale one can't win the priority
|
||||
* race: without this, a completion beat (`celebrate`) would merge on top of a
|
||||
* lingering `error`, and `derivePetState` checks `error` first — so a clean
|
||||
* finish would render the sad/failed pose. */
|
||||
export const flashPetActivity = (next: Partial<PetActivity>, ms = 1600) => {
|
||||
setPetActivity(next)
|
||||
setPetActivity({ celebrate: false, error: false, justCompleted: false, ...next })
|
||||
clearTimeout(flashTimer)
|
||||
flashTimer = setTimeout(
|
||||
() => setPetActivity({ celebrate: false, error: false, justCompleted: false }),
|
||||
|
||||
@ -21,8 +21,8 @@ from agent.pet.constants import FRAME_H, FRAME_W, PetState
|
||||
|
||||
def test_derive_idle_default():
|
||||
assert state.derive_pet_state() is PetState.IDLE
|
||||
# awaiting input rests, doesn't run
|
||||
assert state.derive_pet_state(awaiting_input=True) is PetState.IDLE
|
||||
# awaiting input uses the dedicated waiting row when available.
|
||||
assert state.derive_pet_state(awaiting_input=True) is PetState.WAITING
|
||||
|
||||
|
||||
def test_derive_priority_order():
|
||||
@ -59,10 +59,31 @@ def test_todos_all_done():
|
||||
assert state.todos_all_done([_T("completed"), _T("pending")]) is False
|
||||
|
||||
|
||||
def test_state_row_index_maps_to_taxonomy():
|
||||
# row index must equal position in STATE_ROWS for every driveable state
|
||||
for st in PetState:
|
||||
assert constants.STATE_ROWS[constants.state_row_index(st)] == st.value
|
||||
def test_state_row_index_maps_to_supported_atlas_taxonomies():
|
||||
# Current Petdex sheets are 8 columns x 9 rows.
|
||||
assert constants.state_row_index(PetState.IDLE, 9) == 0
|
||||
assert constants.state_row_index(PetState.WAVE, 9) == 3
|
||||
assert constants.state_row_index(PetState.JUMP, 9) == 4
|
||||
assert constants.state_row_index(PetState.FAILED, 9) == 5
|
||||
assert constants.state_row_index(PetState.WAITING, 9) == 6
|
||||
assert constants.state_row_index(PetState.RUN, 9) == 7
|
||||
assert constants.state_row_index(PetState.REVIEW, 9) == 8
|
||||
|
||||
# Legacy Hermes/petdex sheets were 8 rows with Hermes state names packed in
|
||||
# order. Keep those readable instead of forcing old installs through the
|
||||
# newer Codex taxonomy.
|
||||
assert constants.state_row_index(PetState.WAVE, 8) == 1
|
||||
assert constants.state_row_index(PetState.RUN, 8) == 2
|
||||
assert constants.state_row_index(PetState.FAILED, 8) == 3
|
||||
assert constants.state_row_index(PetState.REVIEW, 8) == 4
|
||||
assert constants.state_row_index(PetState.JUMP, 8) == 5
|
||||
assert constants.state_row_index(PetState.WAITING, 8) == 0
|
||||
|
||||
# Alias rows resolve as expected.
|
||||
assert constants.state_row_index("wave", 9) == constants.state_row_index("waving", 9) == 3
|
||||
assert constants.state_row_index("jump", 9) == constants.state_row_index("jumping", 9) == 4
|
||||
assert constants.state_row_index("run", 9) == constants.state_row_index("running", 9) == 7
|
||||
|
||||
# unknown row names clamp to idle (row 0), never raise
|
||||
assert constants.state_row_index("nonsense") == 0
|
||||
|
||||
@ -161,8 +182,10 @@ def test_trims_trailing_blank_frames(tmp_path):
|
||||
|
||||
cols, rows = 8, 9
|
||||
sheet = Image.new("RGBA", (FRAME_W * cols, FRAME_H * rows), (0, 0, 0, 0))
|
||||
# row index → number of real (opaque) frames; the rest stay transparent.
|
||||
real = {0: 6, 1: 8, 2: 8, 3: 4, 4: 5, 5: 8} # idle wave run failed review jump
|
||||
# row index -> number of real (opaque) frames; the rest stay transparent.
|
||||
# Codex row taxonomy: idle, running-right, running-left, wave, jump, failed,
|
||||
# waiting, run, review.
|
||||
real = {0: 6, 3: 4, 4: 5, 5: 8, 7: 6, 8: 5}
|
||||
for r, k in real.items():
|
||||
for c in range(k):
|
||||
block = Image.new("RGBA", (FRAME_W, FRAME_H), (200, 80, 80, 255))
|
||||
@ -174,17 +197,25 @@ def test_trims_trailing_blank_frames(tmp_path):
|
||||
# Full rows cap at FRAMES_PER_STATE; ragged rows trim to their real count.
|
||||
assert r.frame_count("idle") == constants.FRAMES_PER_STATE
|
||||
assert r.frame_count("run") == constants.FRAMES_PER_STATE
|
||||
assert r.frame_count("failed") == 4
|
||||
assert r.frame_count("wave") == 4
|
||||
assert r.frame_count("jump") == 5
|
||||
assert r.frame_count("failed") == constants.FRAMES_PER_STATE
|
||||
assert r.frame_count("review") == 5
|
||||
|
||||
# Every stepped frame is non-empty — no blank flash for the trimmed states.
|
||||
for state in ("failed", "review"):
|
||||
for state in ("wave", "jump", "review"):
|
||||
for i in range(r.frame_count(state)):
|
||||
assert r.frame(state, i), f"{state}[{i}] rendered blank"
|
||||
|
||||
counts = render.state_frame_counts(str(sprite))
|
||||
assert counts == {
|
||||
"idle": 6, "wave": 6, "run": 6, "failed": 4, "review": 5, "jump": 6,
|
||||
"idle": 6,
|
||||
"wave": 4,
|
||||
"run": 6,
|
||||
"failed": 6,
|
||||
"review": 5,
|
||||
"jump": 5,
|
||||
"waiting": 0,
|
||||
}
|
||||
|
||||
|
||||
|
||||
@ -5007,6 +5007,27 @@ def _pet_frame_counts(spritesheet) -> dict:
|
||||
return {}
|
||||
|
||||
|
||||
def _pet_state_rows(spritesheet) -> list[str]:
|
||||
"""Row taxonomy for the concrete active pet sheet.
|
||||
|
||||
Hermes has to support both the legacy 8-row petdex atlas and the current
|
||||
Codex/petdex 9-row atlas. The desktop canvas gets this list and indexes it
|
||||
with the same `PetState` names the Python renderer uses.
|
||||
"""
|
||||
try:
|
||||
from PIL import Image
|
||||
|
||||
from agent.pet import constants
|
||||
|
||||
with Image.open(spritesheet) as image:
|
||||
row_count = max(1, image.height // constants.FRAME_H)
|
||||
return list(constants.state_rows_for_grid(row_count))
|
||||
except Exception: # noqa: BLE001 - cosmetic, never break the surface
|
||||
from agent.pet import constants
|
||||
|
||||
return list(constants.STATE_ROWS)
|
||||
|
||||
|
||||
@method("pet.info")
|
||||
def _(rid, params: dict) -> dict:
|
||||
"""Return the active petdex pet for surfaces that render sprites.
|
||||
@ -5058,7 +5079,7 @@ def _(rid, params: dict) -> dict:
|
||||
"framesByState": _pet_frame_counts(pet.spritesheet),
|
||||
"loopMs": constants.LOOP_MS,
|
||||
"scale": float(pet_cfg.get("scale", constants.DEFAULT_SCALE) or constants.DEFAULT_SCALE),
|
||||
"stateRows": list(constants.STATE_ROWS),
|
||||
"stateRows": _pet_state_rows(pet.spritesheet),
|
||||
},
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001 - cosmetic, never break the surface
|
||||
|
||||
@ -34,7 +34,8 @@ the agent's behavior** — the sprite is a display concern only. The feature is
|
||||
| A tool is executing | `run` |
|
||||
| The model is thinking/reading | `review` |
|
||||
| Turn in flight (unspecified) | `run` |
|
||||
| Waiting on you / nothing happening | `idle` |
|
||||
| Waiting on you | `waiting` (falls back to `idle` on legacy sheets) |
|
||||
| Nothing happening | `idle` |
|
||||
|
||||
## Rendering
|
||||
|
||||
@ -114,6 +115,25 @@ In the desktop app you can manage the pet two ways:
|
||||
Both adopt/toggle/resize the floating mascot in place — size changes apply
|
||||
instantly; adopting a new pet lights it up within a moment.
|
||||
|
||||
### Pop-out overlay
|
||||
|
||||
**Shift-click** the floating pet to pop it out into its own transparent,
|
||||
always-on-top desktop window. Out there it stays visible while Hermes is
|
||||
minimized (Codex-style), so a glance tells you what the agent is doing:
|
||||
|
||||
- **Speech bubble** — the pet shows what it's up to (`working…`, `thinking…`,
|
||||
`your turn`).
|
||||
- **Drag** it anywhere on screen, even outside the app's bounds. Its spot and
|
||||
in/out state persist across restarts.
|
||||
- **Click** it to open a mini composer and send a prompt to the most recent
|
||||
session without surfacing the app.
|
||||
- **Mail icon** — appears only when a turn finished while you were away; click it
|
||||
to raise the app on the most recent thread (and mark it read).
|
||||
- **Shift-click** again to pop it back into the window.
|
||||
|
||||
The overlay is a pure puppet of the in-app pet — it carries no separate gateway
|
||||
connection and never appears in the dock or app switcher.
|
||||
|
||||
## Configuration
|
||||
|
||||
All settings live under `display.pet` in `config.yaml`:
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user