TY - GEN
T1 - Theory of Mind abilities of Large Language Models in Human-Robot Interaction
T2 - 19th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2024
AU - Verma, Mudit
AU - Bhambri, Siddhant
AU - Kambhampati, Subbarao
N1 - Publisher Copyright:
© 2024 Copyright held by the owner/author(s)
PY - 2024/3/11
Y1 - 2024/3/11
N2 - Large Language Models (LLMs) have shown exceptional generative abilities in various natural language and generation tasks. However, possible anthropomorphization and leniency towards failure cases have propelled discussions on emergent abilities of LLMs especially on Theory of Mind (ToM) abilities in Large Language Models. While several false-belief tests exists to verify the ability to infer and maintain mental models of another entity, we study a special application of ToM abilities that has higher stakes and possibly irreversible consequences: Human Robot Interaction. In this work, we explore the task of Perceived Behavior Recognition, where a robot employs an LLM to assess the robot's generated behavior in a manner similar to human observer. We focus on four behavior types, namely - explicable, legible, predictable, and obfuscatory behavior which have been extensively used to synthesize interpretable robot behaviors. The LLMs goal is, therefore to be a human proxy to the agent, and to answer how a certain agent behavior would be perceived by the human in the loop, for example "Given a robot's behavior X, would the human observer find it explicable?". We conduct a human subject study to verify that the users are able to correctly answer such a question in the curated situations (robot setting and plan) across five domains. A first analysis of the belief test yields extremely positive results inflating ones expectations of LLMs possessing ToM abilities. We then propose and perform a suite of perturbation tests which breaks this illusion, i.e. Inconsistent Belief, Uninformative Context and Conviction Test. The high score of LLMs on vanilla prompts showcases its potential use in HRI settings, however to possess ToM demands invariance to trivial or irrelevant perturbations in the context which LLMs lack. We report our results on GPT-4 and GPT-3.5-turbo.
AB - Large Language Models (LLMs) have shown exceptional generative abilities in various natural language and generation tasks. However, possible anthropomorphization and leniency towards failure cases have propelled discussions on emergent abilities of LLMs especially on Theory of Mind (ToM) abilities in Large Language Models. While several false-belief tests exists to verify the ability to infer and maintain mental models of another entity, we study a special application of ToM abilities that has higher stakes and possibly irreversible consequences: Human Robot Interaction. In this work, we explore the task of Perceived Behavior Recognition, where a robot employs an LLM to assess the robot's generated behavior in a manner similar to human observer. We focus on four behavior types, namely - explicable, legible, predictable, and obfuscatory behavior which have been extensively used to synthesize interpretable robot behaviors. The LLMs goal is, therefore to be a human proxy to the agent, and to answer how a certain agent behavior would be perceived by the human in the loop, for example "Given a robot's behavior X, would the human observer find it explicable?". We conduct a human subject study to verify that the users are able to correctly answer such a question in the curated situations (robot setting and plan) across five domains. A first analysis of the belief test yields extremely positive results inflating ones expectations of LLMs possessing ToM abilities. We then propose and perform a suite of perturbation tests which breaks this illusion, i.e. Inconsistent Belief, Uninformative Context and Conviction Test. The high score of LLMs on vanilla prompts showcases its potential use in HRI settings, however to possess ToM demands invariance to trivial or irrelevant perturbations in the context which LLMs lack. We report our results on GPT-4 and GPT-3.5-turbo.
KW - Large Language Models
KW - Reasoning
KW - Theory of Mind
UR - http://www.scopus.com/inward/record.url?scp=85188097509&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85188097509&partnerID=8YFLogxK
U2 - 10.1145/3610978.3640767
DO - 10.1145/3610978.3640767
M3 - Conference contribution
AN - SCOPUS:85188097509
T3 - ACM/IEEE International Conference on Human-Robot Interaction
SP - 36
EP - 45
BT - HRI 2024 Companion - Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction
PB - IEEE Computer Society
Y2 - 11 March 2024 through 15 March 2024
ER -