Abstract
Internet of things (IoT) devices increasingly permeate everyday life and provide vital and convenient information. Augmented reality (AR) enables the embedding of this information in the environment using visualizations that can contextualize data for various applications such as Smart Home. Current applications providing a visual representation of the information are often limited to graphs or bar charts, neglecting the variety of possible coherence between the subject and the visualization. We present a setup for real-time AR-based visualizations of data collected by IoT devices. Three distinct battery-powered IoT microcontroller systems were designed and programmed. Each is outfitted with numerous sensors, i.e. for humidity or temperature, to interact with the developed AR application through a network connection. The AR application was developed using Unity3D and the Vuforia AR SDK for Android-based mobile devices with the goal of providing processed and visualized information that is comprehensible for the respective context. Inspired by weather applications for mobile devices, the visualization contains animated dioramas, with changing attributes based on the input data from the IoT microcontroller. This work contains the configuration of the IoT microcontroller hardware, the network interface used, the development process of the AR application, and its usage, complemented by possible future extensions described in an outlook.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdelfattah, A.H., Sabirov, R.F., Ivanov, B.L., Lushnov, M.A., Sabirov, R.A.: Calibration of soil humidity sensors of automatic irrigation controller. BIO Web Conf. 17, 00249 (2020). https://doi.org/10.1051/bioconf/20201700249
Aggarwal, R., Singhal, A.: Augmented reality and its effect on our life. In: 2019 9th International Conference on Cloud Computing, Data Science & Engineering (Confluence), pp. 510–515. IEEE (2019). https://doi.org/10.1109/CONFLUENCE.2019.8776989
AndreyGraphics: Low Poly Pack (2018). https://assetstore.unity.com/packages/3d/environments/low-poly-pack-94605. Accessed 10 June 2021
Ardafruit: Adafruit SHT31-D Temperature and Humidity Sensor (2020). https://github.com/adafruit/Adafruit_SHT31. Accessed 10 June 2021
Ardafruit: Adafruit Unified Sensor Driver (2020). https://github.com/adafruit/Adafruit_Sensor. Accessed 10 June 2021
Ardafruit: Adafruit TSL2591 Library (2021). https://github.com/adafruit/Adafruit_TSL2591_Library. Accessed 10 June 2021
Arduino: Arduino IDE 1.8.10 (2021). https://www.arduino.cc/en/software. Accessed 10 June 2021
Arduino: ESP8266 Web Server (2021). https://github.com/esp8266/Arduino/tree/master/libraries/ESP8266WebServer. Accessed 10 June 2021
Arduino: Package ESP8266 (2021). http://arduino.esp8266.com/stable/package_esp8266com_index.json. Accessed 10 June 2021
bblanchon: ArduinoJson (2021). https://github.com/bblanchon/ArduinoJson. Accessed 10 June 2021
Componets101: NodeMCU ESP8266 (2021). https://components101.com/development-boards/nodemcu-esp8266-pinout-features-and-datasheet. Accessed 10 June 2021
Datasheet, A.: ESP-12EWiFi Module (2018). https://cdn-learn.adafruit.com/assets/assets/000/078/658/original/TSL2591_DS000338_6-00.pdf?1564168468. Accessed 10 June 2021
Dogan, E.: LowPoly Water (2018). https://assetstore.unity.com/packages/tools/particles-effects/lowpoly-water-107563. Accessed 10 June 2021
EspressivSystems: ESP8266EX Datasheet (2020). https://www.espressif.com/sites/default/files/documentation/0a-esp8266ex_datasheet_en.pdf. Accessed 10 June 2021
Fielding, R.T., Taylor, R.N.: Architectural styles and the design of network-based software architectures. Ph.D. thesis (2000)
Fritzing: Install Fritzing (2021). https://fritzing.org/download/. Accessed 10 June 2021
Kesler, D., Arntz, A., Friedhoff, J., Eimler, S.C.: Mill instructor: teaching industrial CNC procedures using virtual reality. In: 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), pp. 231–234. IEEE (14122020–18122020). https://doi.org/10.1109/AIVR50618.2020.00048
Kucera, E., Haffner, O., Kozak, S.: Connection between 3D engine unity and microcontroller arduino: a virtual smart house. In: 2018 Cybernetics and Informatics (K&I), pp. 1–8. IEEE (2018). https://doi.org/10.1109/CYBERI.2018.8337531
NXP: UM10204I2C-bus specification and user manual (2014). https://www.nxp.com/docs/en/user-guide/UM10204.pdf. Accessed 10 June 2021
Droms, R.: Dynamic Host Configuration Protocol (1997). https://datatracker.ietf.org/doc/html/rfc2131. Accessed 10 June 2021
RunemarkStudio: PolyDesert (2020). https://assetstore.unity.com/packages/3d/environments/landscapes/polydesert-107196. Accessed 10 June 2021
Sensirion: Datasheet SHT3x-DIS (2017). https://cdn-shop.adafruit.com/product-files/2857/Sensirion_Humidity_SHT3x_Datasheet_digital-767294.pdf. Accessed 10 June 2021
Ai-thinker Team: ESP-12EWiFi Module (2015). https://components101.com/asset/sites/default/files/component_datasheet/ESP12E%20Datasheet.pdf. Accessed 10 June 2021
Unity Technologies: SpotLight (2021). https://docs.unity3d.com/ScriptReference/Experimental.GlobalIllumination.SpotLight.html. Accessed 18 June 2021
Unity Technologies: Unity 3D (2021). https://unity.com/de. Accessed 10 June 2021
ValdayTeam: Low Poly Nature Pack (Lite) (2015). https://assetstore.unity.com/packages/3d/environments/landscapes/low-poly-nature-pack-lite-40444. Accessed 10 June 2021
Vuforia: Getting Started with Vuforia Engine in Unity (2021). https://library.vuforia.com/articles/Training/getting-started-with-vuforia-in-unity.html. Accessed 18 June 2021
Acknowledgments
The presented work is partly supported by the Institute of Positive Computing funded by the Federal Ministry of Education and Research Germany. Equipment used has been partly funded by the initiative for quality improvement in teaching of the Institute of Computer Science. The authors thank Pasquale Hinrichs for reviewing the manuscript and Dustin Keßler for his support in the development process.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Arntz, A., Adler, F., Kitzmann, D., Eimler, S.C. (2022). Augmented Reality Supported Real-Time Data Processing Using Internet of Things Sensor Technology. In: Streitz, N.A., Konomi, S. (eds) Distributed, Ambient and Pervasive Interactions. Smart Living, Learning, Well-being and Health, Art and Creativity. HCII 2022. Lecture Notes in Computer Science, vol 13326. Springer, Cham. https://doi.org/10.1007/978-3-031-05431-0_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-05431-0_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-05430-3
Online ISBN: 978-3-031-05431-0
eBook Packages: Computer ScienceComputer Science (R0)