Overview

EMORI (Embodied Multimodal Observation and Response Interface) is an ARISE-funded project that addresses a gap in healthcare training: hospital administrative staff are usually the first point of contact for patients and their families, often in a stressed or distressed state, yet they lack safe environments to practice de-escalation and empathetic communication. EMORI closes this gap with embodied social robotics: an AI-powered training companion built on the Reachy Mini, a compact open-source desktop robot, and combined with Pokamind’s multimodal role-play training platform. The result is an engaging new tool that lets staff practice challenging conversations on demand, receive immediate multimodal feedback, and build emotional resilience, on their own schedule and without personal devices.

Objectives

1. Develop an embodied training companion on Reachy Mini for de-escalation and empathetic communication practice
2. Adapt training in real time to each learner’s communication style (facial expression, body language, prosody)
3. Contribute open-source ROS 2 modules that extend ROS4HRI with context analysis and engagement-driven behavior generation
4. Validate the system in a real healthcare setting through a pilot at BSA’s Hospital Municipal de Badalona

Technology

The system operates through three integrated layers:

• Perception: Reachy Mini’s camera, microphones and speakers feed Pokamind’s multimodal models, which track facial expressions, body language and vocal prosody.
• Understanding: continuous analysis of the learner’s social signals and of the scenario context drives real-time adaptation of pacing, explanation depth and feedback tone.
• Interaction: the training platform runs on the robot’s integrated touchscreen while a ROS4HRI module translates detections into complementary robot behaviors (gaze shifts, head movements, verbal backchannels and expressions) that actively maintain learner engagement.

Two open-source contributions anchor the project:

• hri_context_analyzer (the Context Analysis Module): a Vulcanexus-compatible ROS 2 package that extends ROS4HRI beyond perception, publishing new /humans/persons/*/context topics along REP-155 conventions — escalation patterns, emotional triggers, topic and relational dynamics, and healthcare-specific situational awareness.
• hri_engagement_behaviors: a generic, platform-independent module mapping engagement levels to robot behaviors, filling the behavior-generation gap in the ROS4HRI ecosystem.

Session analytics are delivered through a Powered-by-FIWARE application (NGSI-LD, TimescaleDB, QuantumLeap) with real-time trainer dashboards, progress tracking and automated alerts.

Partnership

The consortium pairs Pokamind AB (coordinator, Sweden), whose AI-powered role-play training platform is already deployed across 20,000 employees, with Badalona Serveis Assistencials (end user, Spain), a public healthcare organization serving 430,000+ residents, and IIIA-CSIC as subcontractor, driving the ROS4HRI development.

Expected Impact

• For staff: device-free, always-available training during natural breaks, reducing burnout and improving how difficult conversations are handled
• For patients: better-communicated care from the very first point of contact (SDG 3, SDG 4, SDG 8)
• For the robotics community: reusable open-source ROS4HRI modules for context analysis and engagement-aware behavior generation, documented on affordable hardware
• For Europe: a home-grown proof that workplace AI can be built to make workers better at their jobs, not to replace them