Workshop on Evaluating Social Assistive Robots in Healthcare
Location: ICSR 2022 – 14th International Conference on Social Robotics, Florence, Italy
Date: December 16, 2022
Social assistive robots (SARs) perform tasks that involve human-robot interaction, assessment of a person’s activity state, manipulation of objects, navigation in cluttered environments, etc. Evaluating the performance of these robots in real-world conditions is necessary to ensure their reliability, safety, and acceptance in society, especially when operating in a healthcare context, where they may interact with older adults, or persons with physical, sensory and cognitive impairments. The robots and the tasks they perform can be evaluated in several ways, such as lab studies, real-world user trials, robotics competitions, etc.
The main objective of this workshop is to discuss methods and share best practices on evaluating assistive robots. We are particularly interested in methods which involve assessing the robot in real conditions, such as a home or an assisted living facility and discussing advantages and missing aspects of such evaluations. Evaluation of functional aspects, such as perception, human-robot interaction or manipulation performance, and non-functional aspects, such as sociability, usability, trust, acceptability and safety, are of interest.
Sara Cooper, Robotics Software Engineer and Technical Expert, PAL Robotics
Luca Iocchi, Professor, Università di Roma “La Sapienza”
Call for Papers
We have invited participants to submit short papers (up to 4 pages) in the standard IEEE conference format. Submitted papers went through a single-blind review process. Contributions that have been accepted will be presented during the 'Paper Presentations' session. You can find below the list of accepted papers.
Topics of interest included, but were not limited to:
- Methods of evaluating functional aspects of SARs, including, but not limited
to activity recognition, gesture recognition, object handovers, dialogue management, navigation in cluttered environments and manipulation of objects
- Methods of evaluating non-functional aspects of SARs, including, but not limited to, trustworthiness, acceptability, safety, usability and reliability
- Replicable evaluation strategies
- Test-beds and infrastructure for evaluation
- Wearable and ambient sensors for developing and evaluating SARs
- Development of standards and test procedures
- Benchmarking competitions for SARs in healthcare
- User trials in homes or health facilities
Paper submissions (up to 4 pages long) were made through EasyChair in the IEEE format.
Deadline for paper submission: November 28th, 2022
Paper acceptance notification: December 5th, 2022
Camera ready paper: December 10th, 2022
The following papers were presented during the 'Paper presentations' session. There are no official proceedings from the workshop, but a copy of the papers (with the author's permission) are available for download below.
- Garcia-Bedoya, O., Lorente, J. T., Murcia, S., HEART-MET Assistive Robot Challenge with VideoMAE Transformers
- Song, S., IROS2022 HEART-MET Competition Report: Effectiveness of Jensen-Shannon Divergence for Probability Distribution Measurement under Few Training Dataset
- Shervedani, A., Oh, K., Abbasi, B., Monaikul, N., Rysbek, Z., Eugenio, B.D., Zefran, M., Evaluating Multimodal Interaction of Robots Assisting Older Adults
- Kumar, S., Edan, Y., Inclusion of different robots and tasks in HRI evaluation – when and why?
Deadline for paper submission: November 28, 2022
Paper acceptance notification: December 5, 2022
Camera ready paper: December 10, 2022
Workshop: December 16, 2022
Santosh Thoduka, Hochschule Bonn-Rhein-Sieg
Nico Hochgeschwender, Hochschule Bonn-Rhein-Sieg
Praminda Caleb-Solly, University of Nottingham
Mauro Dragone, Heriot-Watt University
Filippo Cavallo, University of Florence
Guilluame Avrin, National Metrology and Testing Laboratory
Virgine Barbosa, National Metrology and Testing Laboratory