Optimizing human supervision of multi-agent systems

Optimizing human supervision of multi-agent systems PI: Panagiotis Artemiadis This proposal will focus on understanding how multi-agent robotic systems can be controlled and coordinated by one human supervisor. More specifically, we will focus on understanding how human supervisors perceive and plan high-level functions for a swarm of semi-autonomous agents. We will define basic principles of communication and utilize them in building the language of supervised autonomy between the human operator and the multiagent system. The overarching goal of the proposed research is to unveil brain mechanisms that perceive multi-agent systems information, and model them in order to define methods for extracting centralized and/or de-centralized control commands for a multi-agent system. More specifically we will pursue two specific objectives: Objective #1: Identify underlying brain perception mechanisms of high-level multi-agent goals: Using a simulated multi-agent environment, we are going to investigate the centralized and decentralized multi-agent control parameters and behaviors that are crucial for understanding the system high-level goals by a human supervisor. By varying the control parameters and characterizing the human responses to these, we will gain insight of the swarm behavioral features that provide the user with the necessary information to infer high-level goals. We hypothesize that there exist high level brain functions that can perceive behavioral data of a group of multiple agents. We are going to determine those mechanisms using human Electroencephalography (EEG) recordings while the subject will infer information from the multi-sensory input provided by the simulated swarm. We will investigate how different brain areas are activated and inter-connected in order to infer highlevel goals and behaviors of the swarm. Finally we will determine the key parameters of the centralized and decentralized multi-agent controllers that evoke optimum responses at the human supervisor level, in order to optimize an actionable knowledge framework and communication protocol (language) between humans and multi-agent systems. Objective #2: Define computational principles and methods for novel brainswarm control interfaces (BSCI): We are going to investigate the mapping between brain states and high-level control actions for multi-agent systems. Both centralized and de-centralized control architectures will be tested. We have recently shown that humans are able to learn mapping functions between their neural activity and artificial systems controllers. Modulation of neural activity in order to directly control external (out of the body) artificial systems is the next step towards human embedded controllers, going beyond traditional techniques based on neural decoders. In this objective, we are going to leverage our recent findings and build mapping functions between brain states and reference signals for the multi-agent system controllers. We will develop a brain-swarm control interface (BSCI) that will use EEG signals to communicate high-level control variables to multi-agent systems. The system will be tested in a simulation environment in a plethora of scenarios relevant to Department of Defense (DoD) missions and objectives.