Keywords

1 Introduction

Today there are many applications of augmented reality that can be classified into six large groups: medicine; assembly, maintenance and repair of the complex equipment; addition of the different information to the existing objects; control of robots, unmanned aerial vehicles, etc.; gaming and entertainment industry; military [1, 2]. In addition, the use of augmented reality for the management and monitoring of telecommunication networks is a special interest [3, 4]. Also, augmented reality is used to analyze traffic flows in the Vehicular Ad-hoc networks (VANET) [5,6,7] and other transport networks to detect congestions [8, 9]. There are a few applications that allow user to manage various engines and devices [10], informing drivers about the road situation [11], and control the UAVs [12]. Flying ubiquitous sensor networks (FUSN) based on UAVs is gaining more popularity with arising of the Industrial Internet of Things (IIoT). Among the industries that demonstrate the greatest interest in the introduction of the Industrial Internet, can be called mining, engineering, agriculture, and transport. It is widely believed that data collection from the devices and data analysis are the main ideas of the FUSN. At the same time the important aspect is missed, that the transfer of a rapidly growing raw data in processing centers requires new technological solutions and methods of the network planning [13, 14]. For some applications, for example, observing over the territory, cameras may be installed on the UAVs. Such systems apply new restrictions on the application and impose new requirements to the telecommunication networks, because of high sensitivity of the UAVs to the network performance. The accuracy and maneuverability of UAV management depends on the quality of service and quality of experience.

Due to the fact that there are many fundamentally different services, it is necessary to develop models for the interaction of network elements while providing the services of the augmented reality, the traffic model, the user’s behavior model, and the density model of both users and objects of augmented reality.

In this paper, we investigated traffic pattern for the augmented reality applications that use video camera for control, observation, monitoring, surveillance etc.

2 Related Works

By now, several works on new traffic patterns have been known in the context of the introduction of the concept of the Internet of Things (IoT). Thus, in [15] authors investigated the solutions for a smart city service based on the technology of augmented reality and Internet of Things infrastructure. They described the implementation of delivering the important information about the bus arrival times, tourist landmarks to the citizens, using bus-mounted IoT devices which transmit the data to the associated cloud servers. D2D communication that reduces the traffic load on the network and transfers data directly between the two devices is well suited for implementing such applications. It is also worth noting that today much attention is paid to models of downloading traffic D2D [16, 17]. In the paper [18] traffic modeling and simulation of multiplayer real-time games and M2M applications are presented. Authors evaluate the impact of additional simulated traffic on the performance of mobile wireless network. The researches from Japan in [19] review 5G network to perform augmented reality applications. They investigated the influence of the bandwidth and the latency on the development and implementation augmented reality services. In [20] the features of augmented reality applications researched. They consider network requirements, such as bandwidth, delay and propose to reevaluate the network infrastructure to support such applications. In this paper [21] traffic in Ubiquitous sensor networks is analyzed. The new on-off model for the source traffic simulating described. During the investigation authors determine how parameters of such model influence on the traffic at the sink.

These works are important and allow us to move forward in analyzing traffic, but for augmented reality there are features associated with the emergence of new parameters such as traverse speed, the angle of sector of the review, density of objects, which are planned to be taken into account in the developed model.

The goal of the investigation is the development a traffic pattern, depending on the UAV’s position changes in the augmented reality applications.

3 Interaction Model

To develop an interaction model of the network elements, we chose as an example an application for the city area monitoring. In the observed area, the objects are people, their personal devices, buildings, cars, traffic conditions, etc. Installing the camera on the UAV allows the pilot to detect the obstructions, to control the UAV, collect data from land sensors and observe everything in the surveillance area. For example, the pilot can determine the object by the sensor located on it and get all the necessary information with the augmented reality glasses or visually identify the object independently if it does not have sensors.

The pilot controls the actions of UAV through the AR device (glasses or helmet) by means of turns and nods, Fig. 1. The camera is installed on the UAV, video stream displayed on the AR device that is on the head of the pilot. Based on the video stream, pilot can define the UAV location and control both the UAV itself and the camera to change the viewing angle. So, it is necessary to transmit two streams simultaneously with different requirements to quality of transmission; the first one is a video stream, the second is a control stream. Also, objects found in the field of view may transmit information to the AR device. Depending on the movement of the UAV or the camera number and type of objects in the field of view may change, which affects both the video stream itself and the control flow.

Fig. 1.
figure 1

Interaction of the main elements in the provision of AR services.

Full size image

4 Traffic Pattern

For the description of the traffic made by service it is necessary to connect volume given by the user and the user of data in case of changes of his surrounding:

  • service space model;

  • user surrounding model;

  • behavior model.

As a service space we will understand an information model of physical three-dimensional space in which there can be a user of service. The information model includes the description of some objects which are in this space of \( X = \left\{ {\bar{x}_{1} ,\bar{x}_{1} , \ldots ,\bar{x}_{n} } \right\} \), where n is total number of objects.

The model of a surrounding of the user is a subspace of service space, i.e. a part of space limited by perception opportunities (model of these opportunities). A surrounding is usually bound to position of the user in space of service and includes a set of objects of \( X^{U} = \left\{ {\bar{x}_{1}^{(U)} ,\bar{x}_{2}^{(U)} , \ldots ,\bar{x}_{n}^{(U)} } \right\} \), where k is a number of the objects, which are in the area of perception of the user.

The behavior model describes changes of position of the user and his surrounding in service space. Changes in a surrounding of the user can be caused by relocation of the user, and relocation of objects in service space. The change caused by appearance in a surrounding of the user of a new object \( \bar{x}_{i} \), leads to a request of data on this object.

The algorithm of implementation of service shall provide performance of the following functions:

  • identification of an event of change of a surrounding and computation of parameters of change;

  • information request about surrounding change;

  • receiving data and its display.

The possible diagram of a data interchange is provided on Fig. 2.

Fig. 2.
figure 2

Diagram of a Data Interchange by provision of AR service.

Full size image

As a surrounding also as well as space of service is limited (area of perception) physical three-dimensional space, changes in it can be described as a stream of events connected to appearance of objects in it. Objects can enter through its boundaries owing to relocation of these boundaries or objects. In the first case, relocation of boundaries is connected to behavior of the user, and in the second to behavior of objects. In that and other case quality of functioning of system will depend on its ability to timely service events of this flow.

Thus, the task of support of quality of service can be considered as a choice of parameters of system (productivity, throughput, distribution of its functionality) from the characteristic of an event stream and load of the system made by this flow.

Properties of an event stream substantially define properties of a data stream between system elements. For example, in system of positioning on the district map such flow is defined by events of change of coordinates of the user and is defined by characteristics of his movement, in system with using AR glasses it will be determined by their orientation in space and, probably, by the events connected to transferring data about objects in field of view or their characteristics.

It is obvious that characteristics of a flow will depend on distribution and characteristics of objects in service space, and also characteristics of movement of the UAV.

Let’s make an assumption that objects in space of service are distributed in a random way (form a Poisson field) and are fixed, only the UAV is mobile. Then, change of position of the UAV is equivalent to change of its surrounding. Taking into account properties of space of service and surrounding, this change can be described by the volume or the area.

Properties of the traffic generated by the UAV device, depends on the distribution of objects of the UAV surrounding and the UAV behavior (motion and rotation). Poisson field is a one of the possible distribution of the objects. This distribution may not very close to real world distribution, bat it is very convenient to get final solutions for the queuing system model. The arguments for the Poisson field are that we do not know for now the exact distribution of the objects in the real world, but we know exactly the solutions for the Poisson field. This model can be used for comparison and quality angling of the service system.

In case information about surrounding of the user represents the image, received by the videocamera, set on the unmanned aerial vehicle, for determination of field of restriction of a surrounding of the user it is necessary to consider features of movement of this device and the transferring video camera. Let’s consider 2D option of a surrounding (terrestrial objects) and will describe a surrounding of the user by radius sector r. Such model is closer to a real situation, than model of a circle [22] since the video camera has limited viewing angle, and probably rather high line speed of movement restricts opportunities regarding the circle review. We will mean the line speed of relocation of the UAV v a constant. Then, during t change of a surrounding will be defined by number of new objects in the area defining a surrounding of the user.

As an example, when using the UAV, the district map with the conditional text and graphics images or designations of objects can be model of a surrounding. The high-speed link of video transmission is more subject to changes of conditions of reception, than low-speed link of signal transmission of control. In case of decline in quality of transmission channel of video, the image, represented to the user can be changed for the district map which is available on the local server.

Let’s estimate quantity of new objects in a surrounding during t as

$$ n(t) = \tilde{S}\left( {L(t)} \right)\rho $$
(1)

where \( \tilde{S}\left( {L(t)} \right) \) - is square of change of a surrounding;

ρ- density of objects (objects/sq.m). The density is equal to the number of objects to the service area ratio.

The model illustrating relocation of the user and change of its surrounding is given in Fig. 3. Offset of the sector representing a surrounding of the user from starting point on distance L is led to formation of area (the shaded area) which defines surrounding change. Objects in the field are identified according to a service provision algorithm, therefore requests for provision of additional information are created.

Fig. 3.
figure 3

Change of Surrounding in case of translational motion.

Full size image

From the given figure, the area of the shaded area can be determined by a formula (2)

$$ \tilde{S}\left( L \right) = \tilde{S}\left( r \right) - \tilde{S}\left( {r - L} \right) = \left\{ {\begin{array}{*{20}l} {\frac{\alpha }{2}\left( {2rL - L^{2} } \right)} \hfill & {L \le r} \hfill \\ {\frac{{\alpha \,r^{2} }}{2}} \hfill & {L > r} \hfill \\ \end{array} } \right. $$
(2)

The number of new objects in the area can be defined as

$$ n(t) = \tilde{S}\left( L \right)\rho $$
(3)

where ρ - density of objects (objects/sq.m),

α - the angle of sector of the review (radians).

Dependence of the number of new objects the area on the time, for different angle and velocity, is depicted on the Fig. 4.

Fig. 4.
figure 4

Dependence of the number of new objects the area on the time (moving)

Full size image

The picture 4 shows the dependence of the number of new objects in the environment during the time t for two different viewing angles π/2 and π (radians), which are close to the typical angle of sector of the review of the camera lenses (normal focal length, short focal length), and also for two values of the speed 5 and 10 m/s, which is close to the typical speed of UAV motion for video recording of the terrain.

As you can see from this picture, when the viewing angle or speed increases, the number of new objects in the area also rises over a time interval. Stabilization of the number of objects in the case as a speed exceeds over some value occurs when a complete change of all objects in the area takes place during the considered period of time. This is because their number is determined by the area of the sector.

Considering this process in progress, i.e. in case of movement of the user, there is the event stream (data requests).

Intensity of an event stream (data requests) can be defined as number of objects in a small increment of the area of the considered figure

$$ \lambda_{r} = \frac{{d\tilde{S}\left( L \right)}}{dL}\left| {L = 0} \right.\;\rho v $$
(4)

where ρ - density of objects (objects/sq.m),

v - traverse speed (m/s).

Derivative of formula (2) in a point L = 0.

$$ \frac{{d\tilde{S}\left( L \right)}}{dL}\left| {_{L = 0} = \alpha \left( {r - L} \right)} \right|_{L = 0} = \alpha {\kern 1pt} r $$
(5)

Then, consider (4) и (5)

$$ \lambda_{r} = \alpha {\kern 1pt} {\kern 1pt} r\rho v $$
(6)

Considering properties of the Poisson field accepted for model the quantity of objects in some limited area is accidental, distributed under the Poisson law and depends only on the area (or volume) of the considered area. Therefore, for the accepted model the flow of requests will represent the elementary flow for which the probability of arrival of k of requests for an interval of time of t will be defined as

$$ p_{k} = \frac{{\left( {\lambda_{r} t} \right)^{k} }}{k!}e^{{ - \lambda_{r} t}} = \frac{{\left( {\alpha {\kern 1pt} {\kern 1pt} r\rho vt} \right)^{k} }}{k!}e^{{ - \alpha {\kern 1pt} {\kern 1pt} r\rho vt}} $$
(7)

Probability of k arrival requests during the time interval given in the Fig. 5.

Fig. 5.
figure 5

Probability density of k requests during the time interval (moving)

Full size image

With translational motion the device can make rotational motions. The model of rotational motion is given in Fig. 6.

Fig. 6.
figure 6

Change of Surrounding in case of turn.

Full size image

The square of the shaded area can be determined by a formula (8)

$$ \tilde{S}\left( \beta \right) = \tilde{S}\left( 0 \right) - \tilde{S}\left( \beta \right) = \left\{ \begin{aligned} \frac{{\beta \,r^{2} }}{2}\quad \quad \beta \le \alpha \hfill \\ \frac{{\alpha \,r^{2} }}{2}\quad \quad \beta > \alpha \hfill \\ \end{aligned} \right. $$
(8)

The number of new objects in the area can be defined as

$$ n(\beta ) = \tilde{S}\left( \beta \right)\rho $$
(9)

where ρ - density of objects (objects/sq.m),

  • β - the turning angle (radians),

  • α - the angle of sector of the review (radians).

Dependence of the number of new objects the area on the time, for different angle, is depicted on the Fig. 7.

Fig. 7.
figure 7

Dependence of the number of new objects the area on the time (turn)

Full size image

Considering this process in progress, i.e. in case of movement of the user the event stream (requests of data) takes place.

Intensity of an event stream (requests about data) can be defined as number of objects in a small increment of the area of the considered figure

$$ \lambda_{r} = \frac{{d\tilde{S}\left( \beta \right)}}{d\beta }\left| {_{\beta = 0} } \right.\,\,\rho \,w $$
(10)

where ρ - density of objects (objects/sq.m), w - angular speed of turn (rad/s).

Derivative of expression (2) in a point β = 0

$$ \frac{{d\tilde{S}\left( L \right)}}{dL}\left| {_{\beta \; = \;0} } \right. = \,\frac{{r^{2} }}{2} $$
(11)

With (10) and (11)

$$ \lambda_{r} = \frac{{r^{2} \rho \,w}}{2} $$
(12)

Taking into account model of a Poisson field, the flow of requests will represent the elementary flow, for which the probability of arrival of k- requests for an interval of time t will be defined as

$$ p_{k} = \frac{{\left( {\lambda_{r} t} \right)^{k} }}{k!}e^{{ - \lambda_{r} t}} = \frac{{\left( {\frac{{r^{2} \rho \,w}}{2}t} \right)^{k} }}{k!}e^{{ - \frac{{r^{2} \rho \,w}}{2}t}} $$
(13)

Probability of k arrival requests during the time interval given in the Fig. 8.

Fig. 8.
figure 8

Probability density of k requests for an interval of time (turn)

Full size image

Figure 9 shows the ratio of angular speed of turn (rad/s) and the line speed of relocation for the equal traffic intensity and the angles of the review \( \alpha = \frac{\pi }{2} \) and \( \alpha = \frac{\pi }{4} \).

Fig. 9.
figure 9

The ratio of angular speed of turn (rad/s) and the line speed of relocation (m/s) for the equal traffic intensity.

Full size image

For example, upon rotation β = π/2 for the one second traffic intensity is equivalent to the traffic intensity of the linear motion with speed of 50 m/s (180 km/h). In this example, the radius is selected to be 50 m. This ratio of produced traffic and UAV motion features should be considered when choosing a method of selection and implementation of data delivery.

The traffic flow made as a result of service provision is defined by a flow of replies to the data requests. Generally, both the single data packet, and a flow of packets (transmission of video or audio data) can be the response. Intensity of this flow can be described as

$$ \lambda_{s} = \lambda_{r} \eta $$
(14)

where η - average quantity of the packets necessary for processing of a request.

By transmission of video data can exceed intensity of requests in tens and hundreds times. Taking into account requirements to quality it leads to essential growth of requirements to throughput of communication network.

The physical amount of surrounding of the user is, as a rule, commensurable with a radius of wireless technologies used for the PAN organization, for example group of Wi-Fi standards. Many objects of AR services (elements of city infrastructure, vehicles, household appliances) can be equipped with nodes of access and necessary data which can be presented to users. Therefore, use of D2D technologies can be the possible decision, providing essential lowering of a traffic on a communication network.

Intensity of this flow will be defined as

$$ \lambda_{s} = \lambda_{r} \eta_{D2D} $$
(15)

where η D2D = (1 - γ)η, γ - part of the objects of surrounding supporting D2D technology.

Certainly, application of D2D of technologies is possible only when objects of service are the physical entities mentioned above, which can be equipped with the respective communication centers. Rather wide range of services requires interaction with remote data bases and the solution of tasks of provision of quality.

5 Conclusion and Discussion

The model considered above allows analysis of the AR service traffic regardless of the service implementation methods, both on the server and client components. The specific method of implementation can significantly influence the nature of the traffic. Implementation is characterized by distribution of data handling functions between the client and server applications, and also methods of data selection. In particular, it is the organization of a buffer memory (cache) of the client application, determination of its size, speed of updating, formations of data requests with the forecast of movement. Control of these parameters allows to select the most acceptable usage mode of resources depending on requirements and behavior of the client application or device.

The article represents the investigation of the Augmented Reality scenario that uses UAV for data and video collection. This application may be widely used for the monitoring of the remote and inaccessible areas, restrained urban conditions with the busy streets and roads. The UAV with a video camera installed is used as an intermediate element. The user of the application through the AR glasses may see not only the picture from the video camera, but also an additional information about objects on the picture, due to the data collection from the sensors and online data analysis on the servers.

During the investigation, the service model of the AR application with the UAV was developed. From this service model, it appears that the characteristics of the created traffic depend on the UAV behavior, UAV surroundings, and the way of data transmission.

Further authors develop the traffic model that considers the UAV-based AR service characteristics, such as a limited lookout angle, speed of the UAV, and a number of objects in the service area.

The proposed model estimates traffic parameters of the new, previously not investigated application. For this reason, authors began the investigation with the Poisson field model for the object distribution, as it has known solutions, and the real objects distribution on the territory is unclear. In the future works authors are going to provide models for the other distributions.