Description of the Cascade Evaluation Campaign for Marine Robots
METRICS RAMI Cascade Evaluation Campaign for Marine Robots will focus on the classification, identification and localization of Object of Potential Interest (OPIs) in underwater images. A training dataset containing OPI images will be proposed to teams to support the development of classification, identification and OPI localization software. The developed software will then be applied by the Organizing Committee (OC) to a test set of OPI images to produce the scoring for each team and the consequent ranking.
What is new in 2023?
In 2023 Campaign we have expanded the training dataset with new images, which you can find in the dataset under the corresponding 2023 folders. Additional numbers over the red background are present. Now numbers can be “4”, “5”, “6”, “7” and “8”.
We have also added a new class, the manipulation objects, which may have, for the first time, multiple OPIs inside the same image.Manipulation objects class may contain the ring-pole and the x-shaped valve, which are used during the field campaigns for the manipulation tasks.
How exchanging software and images?
A Docker image is available at Docker Hub for the teams. The Docker image will contain an Ubuntu18.04 image and the OPI image training dataset.
Each team must install their own dependencies, their solutions with every single file needed to run them, and it must output the results into the indicated path with the format indicated. Then, this Docker images must be uploaded to run the competitor's solution with the final evaluation data.
The documents containing the detailed description of the 1st RAMI Cascade Campaign Marine Robots Rules and the setup, basic usage of Docker, as well as the required format and names for uploading the team Docker image, are attached to the "More Information" box.
Competition steps
- A team registers to the competition by sending team information to im@metricsproject.eu. Once accepted, the team becomes an official RAMI entry and can upload their Docker image for the evaluation (see Call for participation section).
- Teams have to download from Docker Hub a Docker image containing Ubuntu 18.04 and the OPIs training set.
- Teams will have to provide to the RAMI Marine Robots Organizing Committee their software that is able to classify, identify and localize the OPIs present in the images.
- The team software will be evaluated by the Organizing Committee on a test set of images. From the evaluation process, a scoring will be produced for each team and teams will be ranked.
Call for participation
To enter the competition, applicant teams must send an e-mail to the Organizing Committee at im@metricsproject.eu expressing their intention to participate containing the following information:
Team Details
- Team Name.
- Institution/Company.
- Team Logo (optional).
- Team website/social media (if any).
- Name of team leader.
- Picture of team leader.
- When the team was established (year).
- Team Description (including names and nationalities of the team members).
- Brief CVs of the team members (please attach a pdf).
- Team Sponsors (if any):
Contact Details (will not be published):
- Team E-Mail.
- Address
- Phone
The OC will reply the team leader stating that the team has become an official RAMI entry.
OPI images and competition objective
The proposed OPIs have been collected underwater during other RAMI events and divided into five classes: colored buoys, black numbers on yellow pipes, black numbers over red background, red marker on yellow pipe, no OPIs.
Some examples are the following:
