Active-Code Replacement in the OODIDA Data Analytics Platform
- 448 Downloads
OODIDA (On-board/Off-board Distributed Data Analytics) is a platform for distributing and executing concurrent data analytics tasks. It targets fleets of reference vehicles in the automotive industry and has a particular focus on rapid prototyping. Its underlying message-passing infrastructure has been implemented in Erlang/OTP. External Python applications perform data analytics tasks. Most work is performed by clients (on-board). A central cloud server performs supplementary tasks (off-board). OODIDA can be automatically packaged and deployed, which necessitates restarting parts of the system, or all of it. This is potentially disruptive. To address this issue, we added the ability to execute user-defined Python modules on clients as well as the server. These modules can be replaced without restarting any part of the system and they can even be replaced between iterations of an ongoing assignment. This facilitates use cases such as iterative A/B testing of machine learning algorithms or modifying experimental algorithms on-the-fly.
KeywordsDistributed computing Code replacement Erlang
This research was financially supported by the project On-board/Off-board Distributed Data Analytics (OODIDA) in the funding program FFI: Strategic Vehicle Research and Innovation (DNR 2016-04260), which is administered by VINNOVA, the Swedish Government Agency for Innovation Systems. It took place in the Fraunhofer Cluster of Excellence “Cognitive Internet Technologies”. Simon Smith and Adrian Nilsson helped with a supplementary part of the implementation of this feature and carried out the performance evaluation. Ramin Yahyapour (University of Göttingen) provided insightful comments during a poster presentation.
- 1.Chen, H., Yu, J., Chen, R., Zang, B., Yew, P.C.: Polus: a powerful live updating system. In: 29th International Conference on Software Engineering (ICSE 2007), pp. 271–281. IEEE (2007)Google Scholar
- 2.Gu, T., Cao, C., Xu, C., Ma, X., Zhang, L., Lu, J.: Javelus: a low disruptive approach to dynamic software updates. In: 2012 19th Asia-Pacific Software Engineering Conference, vol. 1, pp. 527–536. IEEE (2012)Google Scholar
- 3.McMahan, H.B., Moore, E., Ramage, D., Hampson, S., et al.: Communication-efficient learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05629 (2016)
- 5.Ulm, G., Gustavsson, E., Jirstrand, M.: OODIDA: On-board/off-board distributed data analytics for connected vehicles. arXiv preprint arXiv:1902.00319 (2019)
- 6.Ulm, G., Smith, S., Nilsson, A., Gustavsson, E., Jirstrand, M.: Facilitating rapid prototyping in the OODIDA data analytics platform via active-code replacement. arXiv preprint arXiv:1903.09477 (2019)