Machine Learning Qualification Process and Impact to System Assurance

To address the technical challenges associated with the verification, validation, assurance, and trust of Artificial Intelligence and Machine Learning (AI/ML) in safety critical applications, ARiA in partnership with Arizona State University (ASU) proposed the framework of a Machine Learning Qualification Process (MLQP) in response to a Small Business Technology Transfer (STTR) solicitation. The MLQP incorporates measures and metrics to qualify data sets and models and considerations for the use of data cards, feature cards, and model cards. The US Army Combat Capabilities Development Command Armaments Center (DEVCOM AC) has been developing a roadmap [1] to mitigate the risks associated with the development and deployment of AI/ML enabled systems. The proposed MLQP addresses many of the key challenges and considerations from that roadmap to enable the development of assured and trusted AI/ML enabled systems. This paper will examine how the proposed approach can be leveraged as a tool to build assurance into the cycle of AI/ML development and deployment to ensure safe and reliable systems and the alignment to Army assurance practices as well as DoD guidance.