Supporting Semantic Capture during Kinesthetic Teaching of Collaborative Industrial Robots

Industrial robot systems being deployed today do not contain domain knowledge to aid robot operators in setup and operational use. To gather such knowledge in a robotic context requires mechanisms for entering and capturing semantic data. Such mechanisms would allow a system to gradually build a working vocabulary while interacting with the environment and operators, valuable for the bootstrapping system knowledge and ensuring the data collection over time. This paper presents a prototype user interface that assists the kinesthetic teaching mode of a collaborative industrial robot, allowing for the capture of semantic information while working with the robot in day-to-day use. Two modalities, graphical point-and-click and natural language, support capture of semantic context and the building of a working vocabulary of the environment while modifying or creating robot programs. A semantic capture experiment illustrates the approach.