A Unified Framework for Coordinated Multi-Arm Motion Planning

Abstract

Coordination is essential in the design of dynamic control strategies for multi-arm robotic systems. Given the complexity of the task and dexterity of the system, coordination constraints can emerge from different levels of planning and control. Primarily, one must consider task-space coordination, where the robots must coordinate with each other, with an object or with a target of interest. Coordination is also necessary in joint space, as the robots should avoid self-collisions at any time. We provide such joint-space coordination by introducing a centralized inverse kinematics (IK) solver under self-collision avoidance constraints, formulated as a quadratic program and solved in real-time. The space of free motion is modeled through a sparse non-linear kernel classification method in a data-driven learning approach. Moreover, we provide multi-arm task-space coordination for both synchronous or asynchronous behaviors. We define a synchronous behavior as that in which the robot arms must coordinate with each other and with a moving object such that they reach for it in synchrony. In contrast, an asynchronous behavior allows for each robot to perform independent point-to-point reaching motions. To transition smoothly from asynchronous to synchronous behaviors and vice versa, we introduce the notion of synchronization allocation. We show how this allocation can be controlled through an external variable, such as the location of the object to be manipulated. Both behaviors and their synchronization allocation are encoded in a single dynamical system. We validate our framework on a dual-arm robotic system and demonstrate that the robots can re-synchronize and adapt the motion of each arm while avoiding self-collision within milliseconds. The speed of control is exploited to intercept fast moving objects whose motion cannot be predicted accurately.

Video of Robot Experiments

• Multi-arm coordination for synchronous reaching motion with Virtual-Object DS (VO-DS) [1,2]

• Muli-arm coordination for synchronous/asynchronous reaching motions [3]

• Multi-arm self-collision avoidance [3,4]

Code

Following we list all of the code repositories made available for this project, including:

• Code for estimation of second-order DS in MATLAB
  https://github.com/sinamr66/SESODS_lib

• Code for Execution of second-order DS in C++
  https://github.com/sinamr66/LPV

• Code for Multi-arm DS (synchronous/asynchronous reaching) in C++
  https://github.com/sinamr66/Multiarm_ds

• Code for Generating the Self-Collision Avoidance Dataset in C++
  https://github.com/sinamr66/SCA_data_construction

• Code for Learning the Self-Collision Avoidance function in MATLAB
  https://github.com/nbfigueroa/SCA-Boundary-Learning

• Code for Evaluation of SVM function and gradient in MATLAB and C++
  https://github.com/nbfigueroa/SVMGrad

• Code for Centralized IK solver in C++
  https://github.com/sinamr66/QP_IK_solver

References

[1] Mirrazavi Salehian, S. S., Figueroa, N. and Billard, A. (2016) “Coordinated multi-arm motion planning: Reaching for moving objects in the face of uncertainty”. In Proceedings of Robotics: Science and Systems XVI , Arbor, Michigan, USA. Received Best Student paper Award. Nominated for Best Conference Paper Award. Best Systems Paper Award. link

[2] Mirrazavi Salehian, S. S., Figueroa, N. and Billard, A. (2017) “Dynamical System-based Motion Planning for Multi-Arm Systems: Reaching for moving objects”. In Proceedings of International Joint Conference on Artificial Intelligence 2017, Melbourne, Australia. link

[3] Mirrazavi Salehian, S. S., Figueroa, N. and Billard, A. (2018) “A Unified Framework for Coordinated Multi-Arm Motion Planning”. The International Journal of Robotics Research. link

[4] Figueroa, N., Mirrazavi Salehian, S. S. and Billard, A. (2018) “Multi-Arm Self-Collision Avoidance: A Sparse Solution for a Big Data Problem”. In Proceedings of the Third Machine Learning in Planning and Control of Robot Motion (MLPC) Workshop. ICRA. link

Contact

Nadia Figueroa (nadia.figueroafernandez AT epfl dot ch) or Sina Mirrazavi (sina.mirrazavi AT epfl dot ch)

Acknowledgments

This work was supported by the EU project Cogimon H2020-ICT-23-2014.