Abstract
In this lab, we implement data connectors between various parts of the system, such that data can be forwarded end-to-end, from IoT devices to cloud tools. The setup for this lab is shown in Fig. 8.1, and it includes Raspberry Pi, WiFi router, workstation, and server. To enable data exchange between the different components, we make use of three different connectors: (1) sensor controller (MQTT-to-AMQP brokers), to forward the sensor data coming from MQTT broker running on Raspberry Pi to RabbitMQ running on the server, (2) Data pipeline (AMQP broker-to-HTTP server), to index the data received from AMQP broker into the Elasticsearch database, both running on the server, and (3) actuator controller (HTTP server/AMQP broker-to-MQTT broker), to forward data from RabbitMQ and Elasticsearch to the MQTT broker.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Libraries
Paho Python Client: https://www.eclipse.org/paho/index.php?page=clients/python/index.php
Pika library: https://pika.readthedocs.io/en/stable/
Requests library: https://requests.readthedocs.io/en/latest/user/install/
Tools
Docker Engine installation on Ubuntu: https://docs.docker.com/engine/install/ubuntu/
OpenSSH Server: https://ubuntu.com/server/docs/service-openssh
K3s container orchestrator: https://rancher.com/docs/k3s/latest/en/installation/install-options/
mosquitto-clients (v1.6.9-1): https://packages.ubuntu.com/focal/mosquitto-clients
Package: amqp-tools (0.10.0-1): https://packages.ubuntu.com/focal/amqp-tools
Eclipse Mosquitto (Docker): https://hub.docker.com/_/eclipse-mosquitto
RabbitMQ helm chart: https://github.com/bitnami/charts/tree/master/bitnami/rabbitmq
Logstash helm chart: https://github.com/elastic/helm-charts/tree/7.17/logstash
Elasticsearch helm chart: https://github.com/elastic/helm-charts/tree/7.17/elasticsearch
Other Resources
HTTP client in Python: https://github.com/tubskns/notelab-code/blob/master/rpi/http_client.py
AMQP client in Python: https://github.com/tubskns/notelab-code/blob/master/rpi/amqp_client.py
MQTT client in Python: https://github.com/tubskns/notelab-code/blob/master/rpi/mqtt_client.py
Logstash output plugins: https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html
Logstash input plugins: https://www.elastic.co/guide/en/logstash/current/plugins-inputs-rabbitmq.html
Sensor controller prototype in Python: https://github.com/tubskns/notelab-code/blob/master/rpi/sensor_controller_proto.py
Actuator controller prototype in Python: https://github.com/tubskns/notelab-code/blob/master/rpi/actuator_controller_proto.py
Logstash configuration file: https://github.com/tubskns/notelab-code/blob/master/server/logstash_conf.yml
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Jukan, A., Masip-Bruin, X., Dizdarević, J., Carpio, F. (2023). Lab 5—Prototyping Data Connectors. In: Network of Things Engineering (NoTE) Lab. Springer, Cham. https://doi.org/10.1007/978-3-031-20635-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-20635-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20634-4
Online ISBN: 978-3-031-20635-1
eBook Packages: EngineeringEngineering (R0)