Warning: Uninitialized string offset 0 in /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/endpoints/class-wp-rest-application-passwords-controller.php on line 1

Warning: Uninitialized string offset 0 in /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/endpoints/class-wp-rest-application-passwords-controller.php on line 1

Warning: Cannot modify header information - headers already sent by (output started at /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/endpoints/class-wp-rest-application-passwords-controller.php:1) in /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/class-wp-rest-server.php on line 1831

Warning: Cannot modify header information - headers already sent by (output started at /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/endpoints/class-wp-rest-application-passwords-controller.php:1) in /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/class-wp-rest-server.php on line 1831

Warning: Cannot modify header information - headers already sent by (output started at /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/endpoints/class-wp-rest-application-passwords-controller.php:1) in /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/class-wp-rest-server.php on line 1831

Warning: Cannot modify header information - headers already sent by (output started at /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/endpoints/class-wp-rest-application-passwords-controller.php:1) in /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/class-wp-rest-server.php on line 1831

Warning: Cannot modify header information - headers already sent by (output started at /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/endpoints/class-wp-rest-application-passwords-controller.php:1) in /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/class-wp-rest-server.php on line 1831

Warning: Cannot modify header information - headers already sent by (output started at /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/endpoints/class-wp-rest-application-passwords-controller.php:1) in /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/class-wp-rest-server.php on line 1831

Warning: Cannot modify header information - headers already sent by (output started at /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/endpoints/class-wp-rest-application-passwords-controller.php:1) in /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/class-wp-rest-server.php on line 1831

Warning: Cannot modify header information - headers already sent by (output started at /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/endpoints/class-wp-rest-application-passwords-controller.php:1) in /data01/virt85915/domeenid/www.metricsproject.eu/htdocs/wp-includes/rest-api/class-wp-rest-server.php on line 1831
{"id":804,"date":"2020-07-09T11:13:32","date_gmt":"2020-07-09T11:13:32","guid":{"rendered":"https:\/\/metricsproject.eu\/?page_id=804"},"modified":"2021-01-08T16:03:13","modified_gmt":"2021-01-08T16:03:13","slug":"evaluation-framework","status":"publish","type":"page","link":"https:\/\/metricsproject.eu\/methodology\/evaluation-framework\/","title":{"rendered":"Evaluation Framework"},"content":{"rendered":"
\n
\n\t

METRICS evaluation framework<\/h1>\n

The evaluation framework that will be implemented during the METRICS competitions is based on metrology concepts<\/strong> and will build on previous projects aiming at organizing robotics competitions (e.g. SciRoc, RockEU2, RoCKIn, euRathlon, ECHORD++, EURON, EuRoC and RoboCup).<\/p>\n

METRICS evaluation paradigm consists in comparing reference data (the \u201cground truth\u201d annotated by human experts or provided by measuring instruments in the test facility) with hypothesis data (the behaviour or output produced automatically by the intelligent system). This comparison allows the estimation of the performance, the reliability and other characteristics such as the efficiency of robots. The evaluation can concern the entire system (during TBM) or the main technological bricks taken independently (during FBM).<\/p>\n

This approach to evaluation is based on fundamental metrological principles:<\/p>\n