TY - GEN
T1 - Search Based Testing for Code Coverage and Falsification in Cyber-Physical Systems
AU - Thibeault, Quinn
AU - Khandait, Tanmay
AU - Pedrielli, Giulia
AU - Fainekos, Georgios
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - State of the art test case generation tools vary in the way they exploit knowledge of the system under test. While experimental results demonstrate that such knowledge can improve the efficiency of the test case generation process, how to (automatically) retrieve and integrate such information in test generation is non-trivial. Especially, in the case of Cyber-Physical Systems (CPS), the problem is more challenging since CPS have not only software requirements but also physical functional requirements. In this work, we instrument a system's source code to extract run-time information related to the behavior of the system to determine the path taken through the source code itself. Given our focus on CPS, this information enables us to compute a metric representing the total coverage of the system behaviors for a series of inputs. We present the framework to instrument the code, and our method for using the instrumented code to calculate coverage-related robustness. We also demonstrate how to integrate this approach into a search to generate tests that can manage the trade-off between code coverage and identification of functionally unsafe behaviors (falsification). Our results show that our approach is more efficient than Uniform Random sampling in covering CPS operating modes (code). In addition, we show that our approach promotes better falsifications when violating inputs are present in a, possibly small, subset of the CPS states.
AB - State of the art test case generation tools vary in the way they exploit knowledge of the system under test. While experimental results demonstrate that such knowledge can improve the efficiency of the test case generation process, how to (automatically) retrieve and integrate such information in test generation is non-trivial. Especially, in the case of Cyber-Physical Systems (CPS), the problem is more challenging since CPS have not only software requirements but also physical functional requirements. In this work, we instrument a system's source code to extract run-time information related to the behavior of the system to determine the path taken through the source code itself. Given our focus on CPS, this information enables us to compute a metric representing the total coverage of the system behaviors for a series of inputs. We present the framework to instrument the code, and our method for using the instrumented code to calculate coverage-related robustness. We also demonstrate how to integrate this approach into a search to generate tests that can manage the trade-off between code coverage and identification of functionally unsafe behaviors (falsification). Our results show that our approach is more efficient than Uniform Random sampling in covering CPS operating modes (code). In addition, we show that our approach promotes better falsifications when violating inputs are present in a, possibly small, subset of the CPS states.
UR - http://www.scopus.com/inward/record.url?scp=85174391417&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85174391417&partnerID=8YFLogxK
U2 - 10.1109/CASE56687.2023.10260576
DO - 10.1109/CASE56687.2023.10260576
M3 - Conference contribution
AN - SCOPUS:85174391417
T3 - IEEE International Conference on Automation Science and Engineering
BT - 2023 IEEE 19th International Conference on Automation Science and Engineering, CASE 2023
PB - IEEE Computer Society
T2 - 19th IEEE International Conference on Automation Science and Engineering, CASE 2023
Y2 - 26 August 2023 through 30 August 2023
ER -