In a little over a month, the IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS'26, will take place in Pittsburgh (USA), from the 27th of September to the 1st of October, and we will be there: PhD student Luca Pallonetto will present the lab’s new paper “The SOSI Toolkit: Ontology, Synthetic Dataset, and Benchmark for Social Situation Understanding in Robotics”.

Robots operating in human environments must interpret not only the physical layout of a scene, but also the social situation unfolding in it. Whether a robot should approach, wait, interrupt or reroute depends on social context that is often ambiguous in raw perception, and current systems typically rely on ad hoc, task-specific labels and heuristics, which makes progress hard to compare and even harder to accumulate.

SOSI (SOcial SItuation Interpretation) takes a different route: a unified neuro-symbolic toolkit built around three coordinated pillars:

• an ontology that defines social situations in a machine-readable and compositional way, drawing on interaction science, proxemics and social signal processing;
• a synthetic dataset generated directly from the ontology, with explicit labels and controlled variation along key dimensions (such as cardinality, affective valence and situation formality);
• a benchmark protocol covering semantic separability, retrieval, and synthetic-to-real transfer, with a CLIP-based contrastive baseline showing that SOSI-style supervision improves retrieval on real robot observations.

The initial dataset release contains 19,200 images, and the ontology itself is already available as a standalone resource (SOSI – Social Situation Ontology v1.0.1).

The paper is part of the SWEET project, and is co-authored with Lorenzo Ferrini and Séverin Lemaignan, as well as Silvia Rossi at University of Naples Federico II.

See you in Pittsburgh!

Related project

SWEET (Horizon Europe)

active

Social aWareness for sErvicE roboTs