seb/saltylab-firmware
1
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity

[P1] Salty Face — animated expression UI on touchscreen #370

New Issue
Closed
opened 2026-03-03 15:41:31 -05:00 by seb · 1 comment
seb commented 2026-03-03 15:41:31 -05:00
Owner
Copy Link

Goal

Salty's face on the 7" touchscreen — animated eyes and expressions that react to bot state.

Expressions

  • Happy/Idle: default following state, friendly eyes
  • Alert: obstacle detected, eyes widen
  • Confused/Searching: target lost, eyes look around
  • Sleeping: idle for a while, eyes close
  • Excited: target reacquired, speed up
  • Emergency: e-stop triggered, warning expression
  • Listening: mic active, visual feedback
  • Talking: text-to-speech active, mouth animation

Requirements

  • Fullscreen app on MageDok 7" (1024x600)
  • Smooth animations (minimum 30fps on Orin Nano GPU)
  • React to ROS2 topics: /saltybot/state, /saltybot/target_track, /saltybot/obstacles
  • Touch interaction: tap face to toggle status overlay
  • Tech: Qt/QML, PyGame, or web-based (Chromium kiosk) — whatever runs smoothest
  • GPU-accelerated rendering (Orin has decent GPU, use it)
  • Status bar: battery %, speed, following distance, sensor health

Inspiration

  • Cozmo/Vector robot eyes
  • EMO desktop pet
  • Keep it simple and expressive — two eyes + optional mouth is enough

Depends on

  • #369 Display setup
## Goal Salty's face on the 7" touchscreen — animated eyes and expressions that react to bot state. ## Expressions - **Happy/Idle:** default following state, friendly eyes - **Alert:** obstacle detected, eyes widen - **Confused/Searching:** target lost, eyes look around - **Sleeping:** idle for a while, eyes close - **Excited:** target reacquired, speed up - **Emergency:** e-stop triggered, warning expression - **Listening:** mic active, visual feedback - **Talking:** text-to-speech active, mouth animation ## Requirements - Fullscreen app on MageDok 7" (1024x600) - Smooth animations (minimum 30fps on Orin Nano GPU) - React to ROS2 topics: /saltybot/state, /saltybot/target_track, /saltybot/obstacles - Touch interaction: tap face to toggle status overlay - Tech: Qt/QML, PyGame, or web-based (Chromium kiosk) — whatever runs smoothest - GPU-accelerated rendering (Orin has decent GPU, use it) - Status bar: battery %, speed, following distance, sensor health ## Inspiration - Cozmo/Vector robot eyes - EMO desktop pet - Keep it simple and expressive — two eyes + optional mouth is enough ## Depends on - #369 Display setup
seb commented 2026-03-03 16:38:15 -05:00
Author
Owner
Copy Link

Architecture update: Face UI will be a web app (HTML/CSS/WebGL) running in Chromium kiosk mode via Cage Wayland compositor. See #374 for headless setup.

This means:

  • Face = WebGL canvas with animated eyes
  • Deaf communication (#371) = web page in same app
  • Video calls = WebRTC in same browser
  • Video playback = HTML5 video with NVDEC hardware decode
  • All one unified web stack, ~150MB total vs GNOME 800MB

Tech stack: HTML5 + CSS3 + WebGL (Three.js or PixiJS for face animations) + ROS2 bridge via rosbridge_server (WebSocket)

**Architecture update:** Face UI will be a web app (HTML/CSS/WebGL) running in Chromium kiosk mode via Cage Wayland compositor. See #374 for headless setup. This means: - Face = WebGL canvas with animated eyes - Deaf communication (#371) = web page in same app - Video calls = WebRTC in same browser - Video playback = HTML5 video with NVDEC hardware decode - All one unified web stack, ~150MB total vs GNOME 800MB Tech stack: HTML5 + CSS3 + WebGL (Three.js or PixiJS for face animations) + ROS2 bridge via rosbridge_server (WebSocket)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
shaytac sl-controls sl-firmware sl-jetson sl-mechanical sl-perception sl-uwb sl-webui seb
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: seb/saltylab-firmware#370
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?