Project page for work on Learning Complex Sequential Tasks from Demonstrations

This project is maintained by nbfigueroa

Learning Complex Sequential Manipulation Tasks from Demonstration


Motivated by the current state-of-the-art in Robot Learning from Demonstration (LfD), in this work, we tackle two central issues in the learning pipeline: namely, dealing with (1) heterogeneity and (2) unstructuredness in demonstrations of complex manipulation tasks. We build upon our work on transform-invariant segmentation and action discovery [1], to learn the underlying action sequence of tasks demonstrated in different reference frames or contexts. We then construct and parametrize a multi-phase task-space control architecture, boot-strapped by the segmented data and model parameters learned from the action discovery approach. Successful case studies of the proposed methodology are presented for uni/bi-manual cooking tasks demonstrated through kinesthetic teaching.

Video of Robot Experiments


Following we list all of the code repositories made available for this project, including:


[1] Figueroa, N. and Billard, A. (2018) Transform-Invariant Non-Parametric Clustering of Covariance Matrices and its Application to Unsupervised Joint Segmentation and Action Discovery. In submission. arXiv link

[2] Figueroa, N., Pais, A. L. and Billard, A. (2016) Learning Complex Sequential Tasks from Demonstration: A Pizza Dough Rolling Case Study. In Proceedings of the 2016 ACM/IEEE International Conference on Human-Robot Interaction. HRI Pioneers Workshop. link

[3] Beetz, M., Bessler, D., Winkler, J., Bartels, G., Billard, A., Figueroa, N., Pais, A. L. and et al. (2016) Open Robotics Research Using Web-based Knowledge Services. In Proceedings of the International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 2016. link

[4] Figueroa, N. and Billard, A. (2017) Learning Complex Manipulation Tasks from Heterogeneous and Unstructured Demonstrations. In Proceedings of Workshop on Synergies between Learning and Interaction. IROS’2017 link


Nadia Figueroa (nadia.figueroafernandez AT epfl dot ch)


This work was supported by the EU project Robohow.