The HEART-MET competition
The Healthcare Robotics Technologies - Metrified (HEART-MET) competition aims to address the growing age-related healthcare costs and quality of life problems, which threaten the stability and safety of the social security systems in charge of paying and maintaining high quality of healthcare. The competition addresses tasks that may directly impact the user's well-being, in a complex and unstructured domestic environment. This implies the use of a wide variety of functionalities, and requires a strict validation of the safety of the robotic behaviours.
The robots are evaluated on functionalities such as object detection, activity recognition, object handover, task-oriented grasping and speech understanding, which are required to complete tasks such as assessing the activity state of a person, delivery items such as medicines to a person and preparing and transporting a drink. The field campaigns will be conducted at certified test-beds with physical robots, whereas the cascade campaigns will be conducted online, where performance is evaluated on datasets collected during the field campaigns.
The HEART-MET evaluation plan available below describes the competition and its evaluation and will be continuously updated. Additional information will be posted here as well.
HEART-MET Campaigns
The HEART-MET Field Campaign 2023 will take place at the premises of University of Florence on September 25-29, 2023.
The available benchmarks are: Object detection, Person detection, Object sorting task.
ICRA 2023 METRICS HEART-MET Physically Assistive Robot Challenge
The ICRA 2023 METRICS HEART-MET Physically Assistive Robot Challenge, which will take place at ExCeL London from May 29th to June 1st, 2023. In this challenge, an assistive robot is tasked with fetching an item for a person in a home. The person communicates to the robot the item they would like, and where it is located.
2022 Cascade Campaign. 2nd Edition
The METRICS HEART-MET Assistive Robot Challenge was organised at ICSR 2022 and conducted entirely online based on datasets. The challenge included in this competition is activity recognition. The activity recognition datasets comprise of short video clips of persons performing different activities in domestic environments. The competition will include the Codalab stage - evaluation is performed on a test dataset.
The METRICS HEART-MET robotics benchmarking competition consists of several benchmarks including activity and gesture recognition, object handover, object detection and grasping, and delivering items. The competition targets robots that function in an assistive capacity in a home, or healthcare environment such as a nursing home or hospital.
The METRICS HEART-MET Assistive Robot Challenge at IROS 2022 will be conducted entirely online based on datasets. Three challenges are included in this competition: gesture recognition, activity recognition, and handover failure detection. Learn more about the campaign here.
The first HEART-MET field campaign did take place at the Cobot Maker Space at the University of Nottingham. The competition included a shared benchmark with the European Robotics League (ERL) Professional competition. HEART-MET targets robots that function in an assistive capacity in a home, or healthcare environment such as a nursing home or hospital. ERL Professional (https://www.eu-robotics.net/robotics_league/erl-professional/about/index.html) is a competition framework for mobile robot assistants in professional settings such as factories, grocery stores, hospitals etc. Typical tasks include locating and transporting items and performing collaborative tasks with humans. Therefore, this competition did benchmark robots on several perception, manipulation and human-robot interaction tasks. Results are announced here.
The HEART-MET cascade campaign did consist of four different challenges: activity recognition, gesture recognition, object detection and handover failure detection. The challenges were based on datasets collected from robots performing each of the functionalities in labs set up as homes. The robots require these functionalities when operating in an assistive capacity for persons who have care needs at home or in a care facility.