Loading Events

« All Events

  • This event has passed.

Humanoids 2019 Workshop: Performance indicators and benchmarking facilities for bipedal locomotion

October 15, 2019

Benchmarking has generally been identified as an important instrument for evaluating robot performance and predicting how robots will satisfy the special needs of users. This becomes increasingly important for human-centered robots, such as humanoid and wearable robots, as they move out of the lab and have to perform in real-life situations. Recent benchmarking efforts included several international robotics competitions which received a lot of attention, such as the Cybathlon, DARPA Robotic Challenge, RoboCub or Rockin. While they often address a more qualitative performance of specific tasks, there still is a clear lack of unified benchmarking scenarios and quantified key performance indicators.

The workshop is organized in the context of the Eurobench project, which aims to create the first unified benchmarking framework for robotic systems in Europe. This framework will allow companies and researchers to test the performance of their robots at any stage of development and to get reliable information about robots they may consider purchasing. The project is mainly focused on bipedal locomotion and therefore on humanoid robots as well as lower limb exoskeletons and prostheses, but the general methodology can also be extended to other robotic technologies. The Eurobench project is not only establishing two benchmarking facilities (one for humanoids and one for wearable robots) that will be accessible to the entire academic and industrial robotics community. It is also creating unified benchmarking testbeds, standardized tests and related performance indicators and a benchmarking software, that will allow using the tests in robotics labs worldwide. Benchmarks, on the one hand, measure elementary locomotion skills with respect to stability, robustness, and motions on different terrains and are on the other hand inspired by real-world use cases.

Since the Eurobench project applies a cascade funding scheme, many research teams are contributing through sub-projects which are selected in two separate calls. Projects of the first call (which have recently started), are contributing to the definition and setup of the benchmarking scheme, while projects of the second call (due next year) will be evaluating the benchmarking facility and at the same time improve their robotic systems.

Workshop objectives:

  • Bring together members of the Eurobench project, the subprojects and other members of the robotics community to share ideas and concepts of benchmarking
  • Promote the benchmarking facilities and the possibility of creating additional hubs
  • Inform about the upcoming Eurobench sub-call
  • Get further input on the needs on benchmarking testbeds and performance indicators from industry and academia
  • Collect requirements for benchmarking databases
  • Establish links to ongoing robotics competitions

Topics of interest:

  • Benchmarking bipedal locomotion
  • Performance indicators for robot motions
  • Humanoid robots
  • Wearable robots / exoskeletons / orthoses / prostheses
  • Benchmarking testbeds
  • Real-world scenarios
  • Stability, balance, perturbations

Call for posters:

The deadline for presenting poster / interactive contributions in the topics of interest of this workshop ended on September the 20th, 20119. the notification of acceptance: Will be on Sept 22th, 2019 (earlier submissions will receive an earlier response)


Katja Mombaur, Heidelberg University, Germany
Felix Aller, Heidelberg University, Germany
Diego Torricelli, CSIC, Spain

More information about the workshop

Humanoids 2019 Workshop: Performance indicators and benchmarking facilities for bipedal locomotion


October 15, 2019
Event Category:


Hilton Toronto
145 Richmond St W
Toronto, Ontario M5H 2L2 Canada