Complex value autoencoder for machinery audio generation

Virtual factories are evolving to add more advanced technologies by providing high-fidelity simulations for process optimization and factory management. Data from various sensors, such as cameras and thermal detectors, have become very common, but the incorporation of audio data remains largely unexplored. Audio is crucial in process monitoring and root cause analysis, providing early warnings of potential problems such as tool wear or spindle failures. Generating audio signals for different process states could revolutionize how factories predict failures and manage machinery, shifting from reactive to proactive monitoring. This paper proposes the first step in a new method to create audio and seamlessly integrate it into Virtual factories, enabling real-time sound generation that mirrors the sounds produced by different machinery and processes. For this paper, a dataset is created using the audio of a table CNC machine. The data set is formed using a simple Design of Experiments (DOE) with two process conditions and three levels. The model discussed in the paper is a convolutional auto-encoder that leverages the Short-Time Fourier Transform (STFT) of audio as input to generate audio. The model was trained for 6 h and can recreate audio that looks very close to the real audio with a mean square error of about 0.2027. Integrating audio with virtual reality (VR) will enhance the immersive experience of virtual factories, making training and process simulations more immersive. By generating real-time machine sounds, virtual reality environments will allow operators to practice identifying mechanical problems based on auditory cues, thus improving situational awareness and decision-making. This multi-sensory approach will bring better learning outcomes, reduce errors, and enhance safety in industrial training, which will enable users to anticipate and respond to potential machinery problems more effectively.