The HEART-MET cascade campaign

We are happy to announce results of the test phase METRICS HEART-MET Assistive Robot Challenge at IROS 2022, conducted entirely online based on datasets.

Challenges

Three challenges were included in this competition: gesture recognition, activity recognition, and handover failure detection.

The gesture and activity recognition datasets comprise of short video clips of persons performing different gestures and activities in domestic environments, which need to be recognized. The datasets also contain out-of-distribution (OOD) samples, including corrupted and irrelevant videos - therefore the task requires detection of OOD samples as well.

• Activity Recognition: https://codalab.lisn.upsaclay.fr/competitions/6720
• Gesture Recognition: https://codalab.lisn.upsaclay.fr/competitions/6716

The handover failure detection dataset consists of video and force-torque data from two robot platforms performing robot-to-human and human-to-robot handovers. The task is to detect the outcome of the handover action, including several failure modes which are induced during execution.

• Handover Failure Detection: https://codalab.lisn.upsaclay.fr/competitions/6757

Stages

The competition is held in two stages:

1. Codalab stage - evaluation is performed on a test dataset
2. Remote execution stage - winners of the Codalab stage will be evaluated remotely on a robot at the Ambient Assisted Living Lab at Heriot-Watt University

Schedule

• Start of Codalab development phase:  5th August 2022
• Start of Codalab test phase:                   24th September 2022
• End of Codalab phase:                            30th September 2022
• Remote Execution stage:                       19th October 2022

Results of the test phase

It is our pleasure to announce the winners of the Codalab phase of the METRICS HEART-MET Assistive Robot Challenge. Results are also published on the IROS 2022 webpage.

Activity Recognition:

First place: Soonyong Song
Electronics and Telecommunications Research Institute (ETRI), South Korea

Second place: Mukil Saravanan, Darren Alton Dsouza and Dr. Abhra Roy Chowdhury
RIL, Indian Institute of Science, Bangalore

 

Gesture Recognition:

First place: Soonyong Song
Electronics and Telecommunications Research Institute (ETRI), South Korea

Remote Execution Phase

The remote execution phase took place on October 19. The winner of both benchmarks, Soonyong Song from Electronics and Telecommunications Research Institute (ETRI), participated to the remote execution.

Workshop

We plan to invite winning participants to a workshop to present their results. Details will be announced later.