Applying Sound-Based Analysis at Porsche Production: Towards Predictive Maintenance of Production Machines Using Deep Learning and Internet-of-Things Technology
Situation faced: All mechanical and mechatronic devices are subject to wear, tear and breakdown. Failure of such devices can cause significant costs, e.g., in automotive factories. Established predictive maintenance approaches usually require deep integration with the specific machine. Such approaches are not practically feasible because of technical, legal and financial restrictions. A non-intrusive, lightweight and generic solution approach is desired.
Action taken: A solution concept was developed which, at its heart, is based on deep learning algorithms that monitor sound sequences captured from a microphone, analyze them and return classification results for use in further steps of a control loop, such as planning actions and execution steps. We named this approach the ‘Sound Detective’ and it was evaluated by retrofitting a coffee machine using simple microphones to capture production sounds. The sound sequences are subsequently analyzed using neural networks developed in Keras and TensorFlow. During prototyping, multiple kinds of neural networks and architectures were tested and the experiment was realized with two different kinds of coffee machines to validate the generalizability of the solution to different platforms.
Results achieved: The prototype can analyze sounds produced by a mechanical machine and classify different states. The technical realization relies on cheap commodity hardware and open-source software, demonstrating the applicability of existing technologies and the feasibility of the implementation. Especially, it was described that the proposed approach can be applied to solve predictive maintenance tasks.
Lessons learned: The present work demonstrates the feasibility of the Sound Detective’s reference architecture and discusses challenges and learnings during implementation. Specifically, key learnings include the importance of data quality, preprocessing and consistency, influences of the experimental setup on real-world prediction performance and the relevance of microcomputers, the target hardware and type of the programming language for complex analyses.
The presented work was, in parts, funded by MHP Management—und IT—Beratung GmbH and Dr. Ing. h.c. F. Porsche AG. In no particular order, the authors thank Alice Chan, Judith Gabbert, Belal Chaudhary, Claudio Weck and Roman Siejek for their valuable input.
- Dr. Ing. h.c. F. Porsche AG – Porsche Deutschland (2018) Dr. Ing. h.c. F. Porsche AG – Porsche Deutschland. Available at: http://www.porsche.de. Accessed 31 Jan 2018
- Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning, vol 1. MIT Press, CambridgeGoogle Scholar
- Keras.io (2018) Keras Documentation. Available at: https://keras.io/. Accessed 31 Jan 2018
- Librosa.github.io (2018) LibROSA—librosa 0.5.1 documentation. Available at: http://librosa.github.io/librosa/index.html. Accessed 31 Jan 2018
- Mhp.com (2018) MHP – a Porsche company. Available at: http://www.mhp.com Accessed 31 Jan 2018
- Mobley RK (2002) An introduction to predictive maintenance. Butterworth-Heinemann, AmsterdamGoogle Scholar
- Mqtt.org (2018) MQTT. Available at: http://mqtt.org/. Accessed 31 Jan 2018
- Piczak KJ (2015) Environmental sound classification with convolutional neural networks. In: 2015 I.E. 25th international workshop on Machine learning for signal processing (MLSP), September 2015 (pp. 1–6). IEEEGoogle Scholar
- Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958Google Scholar
- Sundermeyer M, Schlüter R, Ney H (2012) LSTM neural networks for language modeling. In: Thirteenth Annual Conference of the International Speech Communication AssociationGoogle Scholar
- Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112Google Scholar
- TensorFlow (2018) TensorFlow. Available at: https://www.tensorflow.org/. Accessed 31 Jan 2018
- Upton E, Halfacree G (2014) Raspberry Pi user guide. Wiley, ChichesterGoogle Scholar
- Wright GL, Wright MA (2015) U.S. Patent No. 8,983,677. Washington, DC: U.S. Patent and Trademark OfficeGoogle Scholar