saltylab-firmware/jetson/ros2_ws/src/saltybot_emotion_engine

SaltyBot Emotion Engine (Issue #429)

Context-aware facial expression and emotion selection system for SaltyBot.

Features

1. State-to-Emotion Mapping

Maps robot operational state to emotional responses:

• Navigation commands → Excited (high intensity)
• Social interactions → Happy/Curious/Playful
• Low battery → Concerned (intensity scales with severity)
• Balance issues → Concerned (urgent)
• System degradation → Concerned (moderate)
• Idle (no interaction >10s) → Neutral (with smooth fade)

2. Smooth Emotion Transitions

• Configurable transition durations (0.3–1.2 seconds)
• Easing curves for natural animation
• Confidence decay during uncertainty
• Progressive intensity ramping

3. Personality-Aware Responses

Configurable personality traits (0.0–1.0):

• Extroversion: Affects how eager to interact (playful vs. reserved)
• Playfulness: Modulates happiness intensity with people
• Responsiveness: Speed of emotional reactions
• Anxiety: Baseline concern level and worry responses

4. Social Memory & Familiarity

• Tracks interaction history per person
• Warmth modifier (0.3–1.0) based on relationship tier:
  • Stranger: 0.5 (neutral warmth)
  • Regular contact: 0.6–0.8 (warmer)
  • Known favorite: 0.9–1.0 (very warm)
• Positive interactions increase familiarity
• Warmth applied as intensity multiplier for happiness

5. Idle Behaviors

Subtle animations triggered when idle:

• Blink: ~30% of time, interval ~3–4 seconds
• Look around: Gentle head movements, ~8–10 second interval
• Breathing: Continuous oscillation (sine wave)
• Published as flags in emotion state

Topics

Subscriptions

Topic	Type	Purpose
/social/voice_command	saltybot_social_msgs/VoiceCommand	React to voice intents
/social/person_state	saltybot_social_msgs/PersonStateArray	Track people & engagement
/social/personality/state	saltybot_social_msgs/PersonalityState	Personality context
/saltybot/battery	std_msgs/Float32	Battery level (0.0–1.0)
/saltybot/balance_stable	std_msgs/Bool	Balance/traction status
/saltybot/system_health	std_msgs/String	System health state

Publications

Topic	Type	Content
/saltybot/emotion_state	std_msgs/String (JSON)	Current emotion + metadata

Emotion State JSON Schema

{
  "emotion": "happy|curious|excited|concerned|confused|tired|playful|neutral",
  "intensity": 0.0–1.0,
  "confidence": 0.0–1.0,
  "expression": "happy_intense|happy|happy_subtle|...",
  "context": "navigation_command|engaged_with_N_people|low_battery|...",
  "triggered_by": "voice_command|person_tracking|battery_monitor|balance_monitor|idle_timer",
  "social_target_id": "person_id or null",
  "social_warmth": 0.0–1.0,
  "idle_flags": {
    "blink": true|false,
    "look_around": true|false,
    "breathing": true|false
  },
  "timestamp": unix_time,
  "battery_level": 0.0–1.0,
  "balance_stable": true|false,
  "system_health": "nominal|degraded|critical"
}

Configuration

Edit config/emotion_engine.yaml:

personality:
  extroversion: 0.6       # 0=introvert, 1=extrovert
  playfulness: 0.5        # How playful with people
  responsiveness: 0.8     # Reaction speed
  anxiety: 0.3            # Baseline worry level

battery_warning_threshold: 0.25    # 25% triggers mild concern
battery_critical_threshold: 0.10   # 10% triggers high concern

update_rate_hz: 10.0      # Publishing frequency

Running

From launch file

ros2 launch saltybot_emotion_engine emotion_engine.launch.py

Direct node launch

ros2 run saltybot_emotion_engine emotion_engine

Integration with Face Expression System

The emotion engine publishes /saltybot/emotion_state which should be consumed by:

• Face expression controller (applies expressions based on emotion + intensity)
• Idle animation controller (applies blink, look-around, breathing)
• Voice response controller (modulates speech tone/style by emotion)

Emotion Logic Flow

Input: Voice command, person tracking, battery, etc.
  ↓
Classify event → determine target emotion
  ↓
Apply personality modifiers (intensity * personality traits)
  ↓
Initiate smooth transition (current emotion → target emotion)
  ↓
Apply social warmth modifier if person-directed
  ↓
Update idle flags
  ↓
Publish emotion state (JSON)

Example Usage

Subscribe and monitor emotion state:

ros2 topic echo /saltybot/emotion_state

Example output (when person talks):

{
  "emotion": "excited",
  "intensity": 0.85,
  "confidence": 0.9,
  "expression": "surprised_intense",
  "context": "navigation_command",
  "triggered_by": "voice_command",
  "social_target_id": "person_42",
  "social_warmth": 0.75,
  "idle_flags": {"blink": false, "look_around": true, "breathing": true},
  "timestamp": 1699564800.123
}

Development Notes

• Emotion types are defined in EmotionType enum
• Transitions managed by EmotionTransitioner class
• Idle behaviors managed by IdleBehaviorManager class
• Social memory managed by SocialMemoryManager class
• Add new emotions by extending EmotionType and updating _map_emotion_to_expression()
• Adjust transition curves in EmotionTransitioner.transition_curves dict

Future Enhancements

1. Machine learning model for context → emotion prediction
2. Voice sentiment analysis to modulate emotion
3. Facial expression feedback from /social/faces/expressions
4. Multi-person emotional dynamics (ensemble emotion)
5. Persistent social memory (database backend)
6. Integration with LLM for contextual emotion explanation