sl-perception
a9717cd602
New packages:
saltybot_scene_msgs — 4 msgs (SceneObject, SceneObjectArray, RoomClassification, BehaviorHint)
saltybot_scene — 3 nodes + launch + config + TRT build script
Nodes:
scene_detector_node — YOLOv8-nano TRT FP16 (target ≥15 FPS @ 640×640);
synchronized RGB+depth input; filters scene classes
(chairs, tables, doors, stairs, pets, appliances);
3D back-projection via aligned depth; depth-based hazard
scan (HazardClassifier); room classification at 2Hz;
publishes /social/scene/objects + /social/scene/hazards
+ /social/scene/room_type
behavior_adapter_node — adapts speed_limit_mps + personality_mode from room
type and hazard severity; publishes BehaviorHint on
/social/scene/behavior_hint (on-change + 1Hz heartbeat)
costmap_publisher_node — converts SceneObjectArray → PointCloud2 disc rings
for Nav2 obstacle_layer + MarkerArray for RViz;
publishes /social/scene/obstacle_cloud
Modules:
yolo_utils.py — YOLOv8 preprocess/postprocess (letterbox, cx/cy/w/h decode,
NMS), COCO+custom class table (door=80, stairs=81, wet=82),
hazard-by-class mapping
room_classifier.py — rule-based (object co-occurrence weights + softmax) with
optional MobileNetV2 TRT/ONNX backend (Places365-style 8-class)
hazard_classifier.py — depth-only hazard patterns: drop (row-mean cliff), stairs
(alternating depth bands), wet floor (depth std-dev), glass
(zero depth + strong Sobel edges in RGB)
scripts/build_scene_trt.py — export YOLOv8n → ONNX → TRT FP16; optionally build
MobileNetV2 room classifier engine; includes benchmark
Topic map:
/social/scene/objects SceneObjectArray ~15+ FPS
/social/scene/room_type RoomClassification ~2 Hz
/social/scene/hazards SceneObjectArray on hazard
/social/scene/behavior_hint BehaviorHint on-change + 1 Hz
/social/scene/obstacle_cloud PointCloud2 Nav2 obstacle_layer
/social/scene/object_markers MarkerArray RViz debug
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
43 lines
1.4 KiB
XML
43 lines
1.4 KiB
XML
<?xml version="1.0"?>
|
|
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
|
|
<package format="3">
|
|
<name>saltybot_scene</name>
|
|
<version>0.1.0</version>
|
|
<description>
|
|
Semantic scene understanding for saltybot — YOLOv8-nano TRT FP16 object detection,
|
|
room classification (rule-based + MobileNetV2), hazard detection (stairs/drops/wet/glass),
|
|
Nav2 costmap obstacle publishing, and behavior adaptation.
|
|
</description>
|
|
<maintainer email="seb@vayrette.com">seb</maintainer>
|
|
<license>MIT</license>
|
|
|
|
<buildtool_depend>ament_cmake</buildtool_depend>
|
|
<buildtool_depend>ament_cmake_python</buildtool_depend>
|
|
|
|
<depend>rclpy</depend>
|
|
<depend>std_msgs</depend>
|
|
<depend>sensor_msgs</depend>
|
|
<depend>geometry_msgs</depend>
|
|
<depend>vision_msgs</depend>
|
|
<depend>nav_msgs</depend>
|
|
<depend>visualization_msgs</depend>
|
|
<depend>tf2_ros</depend>
|
|
<depend>tf2_geometry_msgs</depend>
|
|
<depend>cv_bridge</depend>
|
|
<depend>image_transport</depend>
|
|
<depend>message_filters</depend>
|
|
<depend>saltybot_scene_msgs</depend>
|
|
|
|
<exec_depend>python3-numpy</exec_depend>
|
|
<exec_depend>python3-opencv</exec_depend>
|
|
|
|
<test_depend>ament_copyright</test_depend>
|
|
<test_depend>ament_flake8</test_depend>
|
|
<test_depend>ament_pep257</test_depend>
|
|
<test_depend>python3-pytest</test_depend>
|
|
|
|
<export>
|
|
<build_type>ament_cmake</build_type>
|
|
</export>
|
|
</package>
|