1st RAMI Cascade Campaign
for Marine Robots
It is our pleasure to announce the first edition of METRICS RAMI Cascade Evaluation Campaign for Marine Robots. This competition will take place fully virtually using data generated during previous RAMI campaigns. The competition is open to everyone all around the world and is led by NATO STO CMRE (Centre for Maritime Research and Experimentation, La Spezia, Italy), one of the reference research centres in Europe devoted to marine robotics technologies.
The RAMI (Robotics for Asset Maintenance and Inspection) competition is organized in the framework of the METRICS (Metrological Evaluation and Testing of Robots in International CompetitionS) EU project and aims at addressing Inspection & Maintenance tasks achieved by aerial and underwater robots in risky and/or hostile environments where human intervention is challenging or impossible, where the direct link with an operator could not be guaranteed and where autonomous decisions are necessary to reduce the operational time of the inspection tasks and ensure repeatability while maintaining an appropriate safety level for the mission. The evaluation process of RAMI competitions mainly involves tasks related to autonomous navigation, data acquisition, detection, classification and autonomous decision-making for inspection purposes. Since aerial and underwater domains are very different, both domains will be evaluated separately in two different tracks.
The RAMI marine competition tasks are inspired by the following user story.
A gas&oil offshore site has to be investigated after that a malfunction has been reported by the plant safety systems.
From what is known, a pipeline has started leaking and an explosion may occur soon.
A robotics team composed of underwater robots (AUVs) is ready to intervene.
It is time for the emergency team to act. The priorities are to reach the area of the accident, and to assess and quantify the entity of the leak. Then the robots have to reach the pipe assembly area for quantifying the damage to the plant and for identifying which pipe has been damaged and is responsible for the leak.
Finally, the robots must intervene on the plant itself by closing a valve to stop the leak to prevent the explosion and further damages to the environment.
Description of the Cascade Evaluation Campaign for Marine Robots
METRICS RAMI Cascade Evaluation Campaign for Marine Robots will focus on the classification, identification and localization of Object of Potential Interest (OPIs) in underwater images. A training dataset containing OPI images will be proposed to teams to support the development of classification, identification and OPI localization software. The developed software will then be applied by the Organizing Committee (OC) to a test set of OPI images to produce the scoring for each team and the consequent ranking.
How exchanging software and images?
A Docker image is available at Docker Hub for the teams. The Docker image will contain an Ubuntu18.04 image and the OPI image training dataset.
Each team must install their own dependencies, their solutions with every single file needed to run them, and it must output the results into the indicated path with the format indicated. Then, this Docker images must be uploaded to run the competitor's solution with the final evaluation data.
The documents containing the detailed description of the 1st RAMI Cascade Campaign Marine Robots Rules and the setup, basic usage of Docker, as well as the required format and names for uploading the team Docker image, are attached to the "More Information" box.
- A team registers to the competition by sending team information to firstname.lastname@example.org. Once accepted, the team becomes an official RAMI entry and can upload their Docker image for the evaluation (see Call for participation section).
- Teams have to download from Docker Hub a Docker image containing Ubuntu 18.04 and the OPIs training set.
- Teams will have to provide to the RAMI Marine Robots Organizing Committee their software that is able to classify, identify and localize the OPIs present in the images.
- The team software will be evaluated by the Organizing Committee on a test set of images. From the evaluation process, a scoring will be produced for each team and teams will be ranked.
Call for participation
To enter the competition, applicant teams must send an e-mail to the Organizing Committee at email@example.com expressing their intention to participate containing the following information:
- Team Name.
- Team Logo (optional).
- Team website/social media (if any).
- Name of team leader.
- Picture of team leader.
- When the team was established (year).
- Team Description (including names and nationalities of the team members).
- Brief CVs of the team members (please attach a pdf).
- Team Sponsors (if any):
Contact Details (will not be published):
- Team E-Mail.
The OC will reply the team leader stating that the team has become an official RAMI entry.
OPI images and competition objective
The proposed OPIs have been collected underwater during other RAMI events and divided into five classes: colored buoys, black numbers on yellow pipes, black numbers over red background, red marker on yellow pipe, no OPIs.
Some examples are the following:
Figure 1. Image of a red buoy.
Figure 2. A “3” black number over the yellow pipe.
A set of images will be proposed to the software developed by teams. The software will have, for each image of the final test set, to produce a file containing:
- A timestamp;
- The class the image belongs to;
- The instance of the class (e.g. “red” color of the buoy or number “3” of the black numbers);
- The position of the centroid of the detected OPI in the image plane.
All the details are reported in the 1st RAMI Cascade Campaign Marine Robots Rules document.
6th May 2022 - The Docker images with the training set will be available for downloading on Docker Hub. Official RAMI entries can then upload their images.
10th October 2022 – Deadline for teams to register to the competition.
30th October 2022 – Deadline for uploading the Docker images by teams. By that time, each team has to clearly indicate to the Organizing Committee which is the official Docker image to be evaluated.
Official information concerning rules, interpretations, and information about the competition can be found on METRICS website, or you can contact us at firstname.lastname@example.org.
Once one team becomes an official RAMI entry, the team leader will serve as the primary point of contact with the OC.
Photo: © Andrea Cesarini
- RAMI’s Evaluation Plan
- 1st RAMI Cascade Campaign Description
- Docker installation guide and user manual
Contact us at im(a)metricsproject.eu
May 6, 2022 - The Docker images with the training set will be available for downloading.
October 10 2022 – Deadline for teams to register to the competition.
October 30 2022 – Deadline for uploading the Docker images by teams.
Dr Gabriele Ferri (CMRE, Italy)
Alessandro Faggiani (CMRE, Italy)
Dr Tommaso Fabbri (CMRE, Italy)