Workshop on Evaluating Social Assistive Robots in Healthcare

Location: ICSR 2022 – 14th International Conference on Social Robotics, Florence, Italy

Date: December 16, 2022

Overview

Social assistive robots (SARs) perform tasks that involve human-robot interaction, assessment of a person’s activity state, manipulation of objects, navigation in cluttered environments, etc. Evaluating the performance of these robots in real-world conditions is necessary to ensure their reliability, safety, and acceptance in society, especially when operating in a healthcare context, where they may interact with older adults, or persons with physical, sensory and cognitive impairments. The robots and the tasks they perform can be evaluated in several ways, such as lab studies, real-world user trials, robotics competitions, etc.

The main objective of this workshop is to discuss methods and share best practices on evaluating assistive robots. We are particularly interested in methods which involve assessing the robot in real conditions, such as a home or an assisted living facility and discussing advantages and missing aspects of such evaluations. Evaluation of functional aspects, such as perception, human-robot interaction or manipulation performance, and non-functional aspects, such as sociability, usability, trust, acceptability and safety, are of interest.

Speakers

Sara Cooper, Robotics Software Engineer and Technical Expert, PAL Robotics

Luca Iocchi, Professor, Università di Roma “La Sapienza”

Schedule

9:00 - 9:15	Introduction and Opening
9:15 - 10:00	Invited Talk by Sara Cooper "Challenges of deploying assistive robots at real-life scenarios"
10:00 - 11:00	Paper Presentations
11:00 - 11:15	Break
11:15 - 12:00	Invited Talk by Luca Iocchi "Knowledge Representation and Evaluation of Social Cognitive Robots"
12:00 - 13:00	Interactive Session
13:00 - 13:15	Wrap Up

Call for Papers

We have invited participants to submit short papers (up to 4 pages) in the standard IEEE conference format. Submitted papers went through a single-blind review process. Contributions that have been accepted will be presented during the 'Paper Presentations' session. You can find below the list of accepted papers.

Topics of interest included, but were not limited to:

• Methods of evaluating functional aspects of SARs, including, but not limited
  to activity recognition, gesture recognition, object handovers, dialogue management, navigation in cluttered environments and manipulation of objects
• Methods of evaluating non-functional aspects of SARs, including, but not limited to, trustworthiness, acceptability, safety, usability and reliability
• Replicable evaluation strategies
• Test-beds and infrastructure for evaluation
• Wearable and ambient sensors for developing and evaluating SARs
• Development of standards and test procedures
• Benchmarking competitions for SARs in healthcare
• User trials in homes or health facilities

 

Accepted papers

The following papers were presented during the 'Paper presentations' session. There are no official proceedings from the workshop, but a copy of the papers (with the author's permission) are available for download below.

• Garcia-Bedoya, O., Lorente, J. T., Murcia, S., HEART-MET Assistive Robot Challenge with VideoMAE Transformers
• Song, S.,  IROS2022 HEART-MET Competition Report: Effectiveness of Jensen-Shannon Divergence for Probability Distribution Measurement under Few Training Dataset
• Shervedani, A., Oh, K., Abbasi, B., Monaikul, N., Rysbek, Z., Eugenio, B.D., Zefran, M., Evaluating Multimodal Interaction of Robots Assisting Older Adults
• Kumar, S., Edan, Y., Inclusion of different robots and tasks in HRI evaluation – when and why?