The Eurobench platform is the central site where all benchmarking material is brought to the community. We can highlight two key pillars: the access to protocols, and the benchmark of experiments following these protocols.

The different protocols gathered in Eurobench are available from the platform, which is a front-end to a database containing and linking to appropriate documentation. A large majority of the protocols developed in Eurobench are uploaded and accessible from the website.

The experiment benchmarking relies on a set of benchmarking algorithms that have been developed by around 20 different entities. Targeting harmonization or standardization of the practices and software methods, we designed a guide presenting the benchmarking principles of Eurobench, and collected experimental data format, promoting as much as possible the use of these principles in the design of the algorithms. Most of the metrics associated to the protocols are directly accessible from the Eurobench GitHub space and can be used by any interested person, with a spirit of transparency and open collaboration, respecting the developer licenses.

Figure 1: Documentation of the Eurobench software methodology

Figure 2: The Eurobench space on GitHub, a collection of software tools developed in Eurobench, including several protocol metrics

Based on this harmonization, we developed appropriate tools for enabling an automatic benchmark of experimental data, process common for all Eurobench protocols. This automation mechanism is deployed onto the Eurobench platform.

Figure 3: Illustration of a protocol as presented on the Eurobench Benchmarking Platform

Projects involved

There are 14 projects and more than 20 entities involved in developing and implementing the Eurobench platform. The following projects are listed below:

DYSTURBANCE: Dynamic and static pusher to benchmark balance. A stability analysis under external perturbation.
Bench: a Biomechanical tEsting platform for of sit to staNd assessment with an instrumented Chair.
STEPbySTEP: Systematic Test of Exoskeleton Products by a Stairs-based Testbed Evaluation Protocol
UDBenchmark: Ups and Downs: benchmarking gait stability of wearable robotic assistive devices and humanoid robots on slopes and lateral inclines
PEPATO: PErformance indicators of spatiotemporal PATterns of the spinal muscle coordination Output during walking with an exoskeleton
BeStaBle: Benchmarking System for Assessment of Balance Performance
COMTEST: Complete Modular Test Stand for Human and Humanoid Posture Control and Balance
TestEd: Benchmark for Benchmarking Industrial Exoskeleton Technology: Testbed and Software routine development for scenario “Moving in Narrow Space”
B.E.A.T.: Balance Evaluation Automated Testbed
MADROB: Modular Active Door for RObot Benchmarking
BEAST: Benchmark enabling active shopping trolley beast
BULLET: Benchmarking Upper Limbs Loads on Exoskeleton Testbeds
BenchBalance: A device to apply well defined perturbations for benchmarking balance capabilities of wearable robots

The Eurobench platform, a unified benchmarking software for remote testing