1st RAMI Cascade Campaign

for Aerial Robots

It is our pleasure to announce the first edition of METRICS RAMI competition for aerial robots, to be held in conjunction with IROS 2021 (September 27th – October 1st). This competition will take place fully virtually using data generated during previous RAMI campaigns. The competition is open to everyone all around the world and is led by CATEC (Advanced Center for Aerospace Technologies, Seville, Spain), one of the reference research centres in Europe devoted to aerial robotic technologies.

RAMI competition aims at addressing Inspection and Maintenance (I&M) tasks achieved by aerial and underwater robots, since they are involved in the most promising applications in this sector due to the risks and costs associated with works at height or underwater inspections performed by human operators. This competition will focus on aerial robots only, and the evaluation process will mainly involve tasks related to autonomous navigation and data acquisition for I&M purposes. There will be two different challenges (Functionality Benchmarks or FBM), and both will be evaluated virtually using data generated by the competition organizers:

FBM1: precise navigation without GNSS, since I&M activities may take place in environments with poor GNSS coverage, or even indoors.

  • The execution of this FBM consists in assessing the accuracy of a localization system for the autonomous navigation of aerial robots using only onboard sensors. The evaluation will be based on the comparison of the team’s localization solution with respect to a precise motion capture system. Teams will be provided with several sample flight datasets for testing their solutions, with the aerial robot performing specific trajectories. The scoring will be based on the Root Mean Squared Error (RMSE) of the provided trajectory with respect to the ground truth trajectory that the aerial robot followed during an evaluation dataset that was not previously released to the teams.

FBM2: automatic detection of defects using advanced AI algorithms, which is important for inspectors when they face considerable amounts of data to review.

  • The execution of this FBM consists of assessing the performance of a surface defect detection system. Teams will be provided with a publicly available dataset for training their algorithms. The assessment will be based on an offline analysis of images obtained by the aerial robot, which will show several surface defects artificially placed along with the testing scenario. Precision, recall and F-measure metrics will be used to assess the performance of each team, comparing the results with ground truth labels.


If you are interested in participating, please fill this form

Photo: © Dose Media 

More information:

Contact us at im(a)metricsproject.eu

Check RAMI’s Evaluation Plan