Human-Robot Cooperation (HRC)
In human-robot cooperation, the robot cooperates with humans to accomplish the task together, which is the future form of robot applications in our daily life. My research aims to study how the robot learns to cooperate with humans in a realistic scenario. My work identified two critical problems in real-world HRC: 1) should the robot learn the task first or learn the human partner’s behavior first? 2) the human usually only has a general goal (e.g., general direction or area in motion planning) at the beginning of the cooperation, which needs to be clarified to a specific goal (i.e., an exact position) during cooperation. 3) To solve the identified problems, I proposed a Hierarchical Task Decomposition method and an Evolutionary Value Learning method.
Forming Real-World Human-Robot Cooperation for Tasks With General Goal
Lingfeng Tao; Michael Bowman; Jiucai Zhang; Xiaoli Zhang
IEEE Robotics and Automation Letters
In human-robot cooperation, the robot cooperates with humans to accomplish the task together. Existing approaches assume the human has a specific goal during the cooperation, and the robot infers and acts toward it. However, in real-world environments, a human usually only has a general goal (e.g., general direction or area in motion planning) at the beginning of the cooperation, which needs to be clarified to a specific goal (i.e., an exact position) during cooperation. The specification process is interactive and dynamic, which depends on the environment and the partner’s behavior. The robot that does not consider the goal specification process may cause frustration to the human partner, elongate the time to come to an agreement, and compromise team performance. This work presents the Evolutionary Value Learning approach to model the dynamics of the goal specification process with State-based Multivariate Bayesian Inference and goal specificity-related features. This model enables the robot to enhance the process of the human’s goal specification actively and find a cooperative policy in a Deep Reinforcement Learning manner. Our method outperforms existing methods with faster goal specification processes and better team performance in a dynamic ball balancing task with real human subjects.
Learn Task First or Learn Human Partner First: A Hierarchical Task Decomposition Method for Human-Robot Cooperation
Lingfeng Tao; Michael Bowman; Jiucai Zhang; Xiaoli Zhang
2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
Applying Deep Reinforcement Learning (DRL) to Human-Robot Cooperation (HRC) in dynamic control problems is promising yet challenging as the robot needs to learn the dynamics of the controlled system and the dynamics of the human partner. In existing research, the robot powered by DRL adopts coupled observation of the environment and the human partner to learn both dynamics simultaneously. However, such a learning strategy is limited in terms of learning efficiency and team performance. This work proposes a novel task decomposition method with a hierarchical reward mechanism that enables the robot to learn the hierarchical dynamic control task separately from learning the human partner’s behavior. The method is validated with a hierarchical control task in a simulated environment with human subject experiments. Our method also provides insight into the design of the learning strategy for HRC. The results show that the robot should learn the task first to achieve higher team performance and learn the human first to achieve higher learning efficiency.
