Learning of Parameters in Behavior Trees for Movement Skills

Matthias Mayr, Faseeh Ahmad, Konstantinos Chatzilygeroudis, Luigi Nardi, Volker Krueger

Research output: Chapter in Book/Report/Conference proceedingPaper in conference proceedingpeer-review

Abstract

Reinforcement Learning (RL) is a powerful mathematical framework that allows robots to learn complex skills by trial-and-error. Despite numerous successes in many applications, RL algorithms still require thousands of trials to converge to high-performing policies, can produce dangerous behaviors while learning, and the optimized policies (usually modeled as neural networks) give almost zero explanation when they fail to perform the task. For these reasons, the adoption of RL in industrial settings is not common. Behavior Trees (BTs), on the other hand, can provide a policy representation that a) supports modular and composable skills, b) allows for easy interpretation of the robot actions, and c) provides an advantageous low-dimensional parameter space. In this paper, we present a novel algorithm that can learn the parameters of a BT policy in simulation and then generalize to the physical robot without any additional training. We leverage a physical simulator with a digital twin of our workstation, and optimize the relevant parameters with a black-box optimizer. We showcase the efficacy of our method with a 7-DOF KUKAiiwa manipulator in a task that includes obstacle avoidance and a contact-rich insertion (peg-in-hole), in which our method outperforms the baselines.
Original languageEnglish
Title of host publication2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
PublisherIEEE - Institute of Electrical and Electronics Engineers Inc.
Pages7572
Number of pages7
ISBN (Electronic)978-1-6654-1715-0
ISBN (Print)978-1-6654-1714-3
DOIs
Publication statusPublished - 2021 Dec 16
EventIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2021 - Prague, Czech Republic
Duration: 2021 Sept 272021 Oct 1

Conference

ConferenceIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2021
Country/TerritoryCzech Republic
CityPrague
Period2021/09/272021/10/01

Subject classification (UKÄ)

  • Computer Science
  • Robotics

Free keywords

  • Reinforcement Learning
  • Robotics
  • Skills
  • Manipulators
  • Bayesian Optimization
  • Learning

Fingerprint

Dive into the research topics of 'Learning of Parameters in Behavior Trees for Movement Skills'. Together they form a unique fingerprint.

Cite this