Task and Skill Planning

TASP composes heterogeneous robot skills — including off-the-shelf, learned, and force-controlled policies — into long-horizon plans.

Abstract

Task and motion planning (TAMP) is a well-established approach for solving long-horizon robot planning problems. Although TAMP methods have historically assumed that each task-level robot action, or skill, can be reduced to kinematic motion planning, recent work has explored integrating closed-loop controllers and learned skills into TAMP-style systems.

Our approach integrates pre-existing, heterogeneous robot skills - including learned, force-controlled, and black-box policies - into a hierarchical planner while preserving the object-centric failure reasoning of typical TAMP solvers. We leverage Composable Interaction Primitives (CIPs) to synthesize head and tail motion plans bridging consecutive skills, facilitating both planning-time refinement and execution-time adjustment.

We validate our Task and Skill Planning (TASP) approach through real-world experiments on a bimanual manipulator and a mobile manipulator, demonstrating that CIPs enable diverse robots to combine heterogeneous skills to solve complex, long-horizon tasks, including multi-room mobile manipulation problems with non-monotonic task structure.

Video Demonstrations

BibTeX

@inproceedings{hedegaard2026tasp,
  title     = {Task and Skill Planning: Hierarchical Robot Planning with Black-Box Skills},
  author    = {Hedegaard, Benned and Wei, Yichen and Yang, Ziyi and Jaafar, Ahmed
               and Tellex, Stefanie and Konidaris, George and Shah, Naman},
  booktitle = {Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)},
  year      = {2026}
}

Acknowledgments

This work was supported by ONR REPRISM MURI N00014-24-1-2603, ONR grant 00014-22-1-2592, and the Robotics and AI Institute (RAI).