Abstract
This paper presents a prototype sensor network for monitoring urban traffic. The sensor network node, equipped with a low-resolution camera, observes the street and detects moving objects. Object detection is based on the custom video segmentation algorithm, using dual background subtraction, edge detection and shadow detection, running on dedicated multi-processor SoC hardware. The number and the speed of the detected objects are transmitted using a low-power license-free radio transceiver to another neighboring node. All the nodes create a self-organized network, data are aggregated at the nodes and passed further to the nodes closer to data sinks. Finally, information about the traffic flow is collected from the sinks and visualized on a PC. The prototype sensor network node has been realized in two versions: FPGA and ASIC. The ASIC version consumes approximately 500 mW and it can be powered from a photovoltaic solar panel combined with a single cell Li-Po battery. The comparison of power consumption of both versions has also been made. Apart from collecting traffic data, the proposed sensor network can gather environmental data, such as the temperature, the acoustic noise or the intensity of the sunlight. The set of 26 prototype sensors has been mounted on street lamp-poles on streets and tested in real conditions.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Sensor networks are very useful in collecting data from large areas. Many potential examples of sensor networks can be found in the literature [1–4]. The sensor network can also be effectively used for urban traffic. Traffic monitoring systems are very useful in providing management of the traffic flow and they can help in increasing the throughput and safety of the transportation and provide valuable information for planning of future road developments. Typically, detection of moving vehicles is done by inductive loops, passive infrared sensors, magnetometers, microphones, radars or microwave sensors [5–8]. Also, high-resolution cameras connected to the monitoring center using high-bandwidth cables or fiber optic links are often used but collecting data from remote sensors is usually very expensive as far as installation and usage are concerned. In recent years, an automatic video detection has been becoming more popular in the literature. There are many algorithms and systems for this purpose. Unfortunately, most of them work with high-resolution images and require significant computing power, which is not suitable for low-power sensor networks. The paper [9] contains a survey of various image detection algorithms, from simple background subtraction to optical flow and stereo vision methods. In [10], the authors present the algorithm dedicated for existing street cameras, which uses background subtraction with the Bayesian Network Classifier. The background subtraction is also used in the algorithm described in [11, 12], where pixels are modeled as the mixture of Gaussians and the pixel model is learned using the EM algorithm. The mixture of Gaussian for low-level tracking is also proposed in [13], while high-level position estimation is done using the Kalman filter. The PC-based system described in [14] utilizes the probability-based foreground extraction, with color image processing and shadow removal. Two separate algorithms for day and night operation are proposed in [15]: the spatio-temporal analysis is used during daytime, the morphological analysis of headlights is used at night, the collected information is processed with the forward chaining rule production system. Another approach is described in [16], where the outlines of the detected cars are represented as quadratic B-spline curves. The 3-D model consisting of the edge segments is matched with 2-D image segments in [17]. In [18] an adaptive background estimation on image divided into non-overlapped blocks is described, with PC-based implementation capable of 15 fps image processing. In [19, 20], FPGA-based implementations are proposed, where some regions of interest are defined and analyzed in the observed image. The self-contained computer system running Linux with the attached camera is described in [21], where the authors demonstrate the application of simple background detection algorithm. Traffic detection based on frame differencing, background subtraction and virtual induction coils are presented in [22]. [23] contains camera detection systems evaluation and optimal camera placement considerations. Other video detection systems use statistical approach using principal component analysis and independent component analysis [24], neural network approach with multilayer feed-forward neural network [25], support vector machine [26] or symmetry-based vehicle detection [27, 28].
The motivation of the presented work was to propose the low cost method (in terms of the hardware and the maintenance costs) of measuring the city traffic. The set of small, low-power, autonomous devices (i.e. sensor network nodes), taken out-of-the-box and attached to the street lamp-poles, is able to evaluate the traffic using their built-in cameras and to transmit traffic data to the central computer, using the self-organized radio network (even ad-hoc installation is possible in emergency situations). Data collected by the network can be used for many purposes, such as: informing citizens by local radio station or internet, detecting extraordinary traffic events, guiding the emergency vehicles to achieve their goal more quickly or interacting with the traffic lights to improve the traffic flow, resulting in a decrease in traffic jams and environmental pollution. Compared to the standard vehicle detection method with the inductive loops, the use of small sensor network nodes has many benefits: simple installation, lack of wire connections, independent radio communication, possibility to estimate the speed, direction and size of the vehicles, capture and transmission of single images to the monitoring center.
In this paper, a sensor network for monitoring the traffic of the vehicles on the street of a city is described. The design target is a low-power sensor network node capable of estimating the traffic flow, due to the carefully developed video detection algorithms and proper hardware/software co-design. The node of the sensor network is designed to be installed on street lamp-poles to observe the scene from the location high above the road. Analysis of the video stream locally by each node of the sensor network significantly reduces the amount of data which needs to be transmitted over the radio, further decreasing the energy usage. The nodes are capable of detecting car traffic; additionally, single images from the nodes’ cameras can be sent, informing the operator, for example, about snow on the street.
The proposed sensor network node (Fig. 1) uses a typical low-cost camera, thus the installation near the street lamp is mandatory to enable observation in the night.
The sensor network can be quickly and simply installed without the need for expensive cabling or permissions, since the nodes have a built-in, low-power radio working on license-free frequency band (see Fig. 2) and they can be powered from the solar panel.
The layout of the paper is as follows: in section II, the authors present the algorithm and realization of the image processing for moving object detection. Section III describes the protocol and organization of the radio communication network. The details about the realization of the prototype nodes are given in section IV. The results of the sensor network operation and conclusions are presented in sections V and VI, respectively
2 Moving Object Detection
Monitoring of the traffic with a camera is a very challenging task. It requires an image-processing algorithm capable of detecting cars under variable light conditions. Implementation of an image-processing algorithm in the sensor network node is further constrained by the limited resources of the node. The sensor network node is usually a low-power device using as simple hardware as possible to decrease the size and the power consumption. Due to this fact, a careful design of an image-processing algorithm which would fit into those limited hardware resources is very important. In the presented solution, a monochrome camera is used to reduce the complexity of the hardware by about 3 times at a cost of decreasing the segmentation sensitivity; additionally, the image resolution has been decreased to 128 × 128 pixels.
-
A.
Low–Level Image Processing
In the presented implementation, the non-model-based approach for moving object detection with background subtraction is used, where each current frame from the camera is subtracted from the background image stored in the memory. Among the algorithms mentioned in section I, the background subtraction is the easiest to implement in hardware, but it does not provide in its basic form the detection quality acceptable for the outdoor vehicle detection. The complete block diagram of the image-processing algorithm is shown in Fig. 3. The detailed description of the low-level image segmentation algorithm can be found in [29], where the earlier work of the authors is presented.
The authors decided to use two background models concurrently [30–33]: long-term with non-selective update and short-term with selective background update. In this way, the stopped objects are initially not included into the selective background, but after a while become a part of the non-selective background. After being part of the non-selective background, the stopped object is not detected any more, thus it can be quickly included into the selective background too.
Both background models assume single Gaussian distribution of pixel intensities for time t with the average brightness μ t and the standard deviation σ t updated using running mode, which can be very efficiently realized in hardware.
The detection results, using inequality (1), in the form of the masks: m N and m S from non-selective and selective background model, respectively, are combined into a single binary mask m B , using special combination of and and or operations, similarly as described in [32]. For the simplicity of the hardware, apart from the current pixel, instead of eight neighboring pixels, only the four previously analyzed pixels are used. To improve the segmentation quality, two edge detection blocks have been introduced: temporal edge detection and spatial edge detection, resulting in mask m ET and m ES , respectively. To further increase the selectivity of the object detection, several additional blocks have been added, such as shadow detection block (mask m SH ) or highlight detection block detecting highlights from the car lights in the night (mask m HI ) and very dark pixels on a bright background (mask m X ). The basic detection of shadows is done by comparing the decrease in brightness [34]. The Final Processing block from Fig. 3 is responsible for basic morphological operations (erosion followed by dilation, using 2 × 2 rectangular structuring element) to condition the final mask m BEHSX . The final detection result is processed by the Hough transform based block, which helps to obtain convex, filled blobs representing the detected objects. The Hough transform with the rectangular structuring element improves the shape of the blobs (Fig. 4). The mask m V is the final detected vehicle mask image of elements equal to 0 or 1, where 1 denotes the pixels belonging to the detected moving objects.
All the image-processing operations shown in Fig. 3 have been selected and adapted for efficient hardware implementation. The details of the pipelined implementation of the algorithm from Fig. 3, as well as the segmentation results, can be found in [29]. In this paper, instead of typical pipelined hardware, a set of concurrently working, dedicated Pixel Processors (PP) has been developed to enable the modifications of the algorithm in the ASIC implementation. The PPs have been designed as scalable modules in VHDL: 16-bit, 8-bit and 1-bit versions are provided. The system contains nine PPs, as shown in Fig. 5. The processors are connected to the common data bus via a programmable switch matrix, where the bus contains data of the currently processed pixel. Each processor can read any of 128 data bits, but it can write only to the exclusively assigned selection of bits, except from 1-bit processors which can write to each bit of the bus. The 1-bit processors have a built-in memory for caching the previously analyzed pixels - data from this memory are used to perform simplified morphological operations that require information about the neighboring pixels, such as erosion or dilation. The processing tasks have been divided among the processors, as listed in Table 1.
The camera can view the street from various angles. To estimate the speed and the size of the moving objects, the image must be geometrically transformed. An example of the transformation is shown in Fig. 6, presenting the input image before and after the transformation. In the real system, the mask m V is transformed into the mask m TF . The image transformation is realized by the hardware block equipped with the memory block containing the image-mapping coordinates.
The detected blobs are labeled and their basic parameters are calculated, such as: object’s boundaries, the center of the object and the area in pixels. The labeling and calculating of the parameters of the objects is done during pixel by pixel revision of the image by the selected PPs.
-
B.
High–Level Tracking
The detection results from PPs in the form of the table containing the basic parameters of the blobs are passed to the main 32-bit processor of the system, where the objects are traced, classified and measured. The detected blobs are usually noisy and inaccurate, i.e. they can change shape, disappear, split or join with the other blobs. The main processor services the radio network and it also has to extract as much information as possible to infer the moving cars, but the limited computing resources did not allow the use of the Kalman filter approach in real time, therefore a simplified method has been used. For that purpose, a set of if-then heuristic rules has been applied, working on simple rectangular models of the blobs. For every picture frame, each detected blob is always modeled as the rectangle B (denoted as blob B in the further text) with the assigned attributes shown in Table 2.
For simplicity, all the average values are calculated as running mean and the movement path is approximated with a straight line. The overlapping of the blobs in the current and the previous frame is detected by the PP and both such blobs are treated as representing the same moving object, providing the continuity of the existence of the blob’s rectangle B. If the blob B disappears, the virtual model M is created, continuing the movement of the blob B, using the last known values for size, direction and speed of the disappeared blob B. The attributes of the model M are listed in Table 3.
A simple example of blob processing is shown in Fig. 7, where at the time t, a new blob has been detected and marked as B t (the speed \( \overline {\text{V}} {\text{B,t}} \) and the direction \( \overline \alpha {\text{B}},{\text{t}} \) are unknown at this time). At the next time step t + 1, the blob changes its position and shape, but, due to the overlapping with the blob from time t, it is treated as the same moving blob, so the attributes of B t are copied to B t+1, also the first approximation of the speed of the blob \( {{\overline {\text{V}} }_{{{\text{B,t}} + 1}}} \) can be calculated, as well as \( {{\overline \alpha }_{{{\text{B}},{\text{t}} + 1}}} \). At the time step t + 2, the blob disappears, but its existence is represented by the newly created model M t+2 with the following attributes:
When the size of the blob B changes (which could mean joining with another blob or splitting), new objects are created: a new model M, as a continuation of the B, and the new blob (or blobs in the case of splitting) representing the current blob. As an example, two moving blobs are shown in Fig. 8, represented by B a and B b at the time t. In the next frame at the time t + 1, both blobs have joined together and from this moment B a is representing the connected blobs. However, due to the size change of the blob B a and disappearing of the blob B b , the blobs B a and B b from time t have been converted into the models M d and M c , respectively. Later, at the time t + 2, the blob B a splits again into two blobs B a and B b ; the overlapping models M c and M d are then absorbed by the blobs B a and B b , respectively, providing the continuity of their movement.
Having a set of blobs B and models M for each frame, the main processor executes the heuristic rules to extract data about moving vehicles. For example, if any blob and model are close together and have similar speed and direction of movement, depending on the situation, the model can be deleted or its position can be centered with the blob. Small models that are close to a bigger one and have similar speed and direction are deleted. The models that overlap with each other and move in the similar direction and with similar speed are consolidated. The most important rules concerning the blobs and models are summarized in Table 4 and below:
High level tracking rules:
-
rule #1
If BlobSizeChange(B a,t , B a,t+1) ≥ 50 % then
M a,t+1 = NewModelFromBlob(B a,t )
B b,t+1 = NewBlob(B a,t+1)
DeleteBlob(B a,t+1)
-
rule #2
If BlobDisappeared(B t ) then M t+1 = NewModelFromBlob(B t )
-
rule #3
If Distance(B, M) < D min and \( \left| {{{\text{V}}_M} - {{{\overline {\text{V}} }}_B}} \right| < {\text{V}}\,min \) and AngleDifference(\( {{\overline \alpha }_{B}},{{\alpha }_{M}} \)) < α min and t 0B < t 0M then BlobAbsorbsModel(B, M).
-
rule #4
If Overlap(B, M) > 50 % and \( {{\overline {{V}} }_B} = 0 \) and V M = 0 then DeleteModel(M)
-
rule #5
If SpeedDifference(\( {{{V}}_{{{{{M}}_{{a}}}}}}{, }{{{V}}_{{{{{M}}_{{b}}}}}} \)) < 50 % and AngleDifference(\( {{\alpha }_{{{{{\text{M}}}_{{\text{a}}}}}}},\,{{\alpha }_{{{{{\text{M}}}_{{\text{b}}}}}}} \)) < α min and Distance(M a , M b ) < D min and (\( {{{\text{S}}}_{{{{M}_{a}}}}} = {\text{small}} \) or \( {{\text{S}}_{{{{{M}}_{{b}}}}}} = {\text{small}} \)) then ConsolidateModels(M a , M b )
-
rule #6
If SpeedDifference(\( {{{V}}_{{{{{M}}_{{a}}}}}}{, }{{{V}}_{{{{{M}}_{{b}}}}}} \)) < 50 % and AngleDifference(\( {{\alpha }_{{{{{\text{M}}}_{{\text{a}}}}}}},\,{{\alpha }_{{{{{\text{M}}}_{{\text{b}}}}}}} \)) < α min and Overlap(M a , M b ) < min(\( {\text{t}} - {{\text{t}}_{{0{{{M}}_{{a}}}}}} \), 100) then ConsolidateModels(M a , M b )
-
rule #7
If SpeedDifference(\( {{{V}}_{{{{{M}}_{{a}}}}}}{, }{{{V}}_{{{{{M}}_{{b}}}}}} \)) < 50 % and AngleDifference(\( {{\alpha }_{{{{{\text{M}}}_{{\text{a}}}}}}},\,{{\alpha }_{{{{{\text{M}}}_{{\text{b}}}}}}} \)) < α min and (Overlap(M a , M b ) < min(\( {\text{t}} - {{\text{t}}_{{0{{{M}}_{{a}}}}}} \), 100) or Overlap(M a , M b ) < min(\( {{t}} - {{{t}}_{{0{{{M}}_{{b}}}}}} \), 100)) then ConsolidateModels(M b , M a )
Rules used for conversion of blob B into model M:
-
rule #8
If NewModelFromBlob(B) and \( {{\overline {\text{V}} }_{\text{B}}} \) and \( {{\overline \alpha }_{{\text{B}}}} \) are not known then create a new short-term model M using the parameters of B and T M = T MIN .
Short-term model is supposed to live shortly and it will disappear after T MIN frames. It is used to track “blinking” moving blobs, for which it is difficult to estimate \( {{\overline {\text{V}} }_{\text{B}}} \) and \( {{\overline \alpha }_{{\text{B}}}} \) due to their short existence on the screen. T MIN = 6 has been set experimentally for frame rate 32 fps.
-
rule #9
If NewModelFromBlob(B) and \( {{\overline {\text{V}} }_{\text{B}}} \) and \( {{\overline \alpha }_{{\text{B}}}} \) are known and \( {{\overline {\text{V}} }_{\text{B}}} < {{V}_{\text{MAX}}} \) and not (is_human(B)) and \( {{\text{q}}_{\text{B}}} > {{\text{Q}}_{\text{MIN}}} \) then create new model M using the parameters of B.
where:
- V MAX :
-
the maximum allowed speed for the vehicle
- D min :
-
the minimum allowed distance (assumed 4 pixels)
- α min :
-
the minimum allowed angle, assumed α min = 45°
The function is_human() is a simple and rough estimate, if the detected blob could be a person walking along the street (assuming that the camera is situated high above the road) and is calculated as:
where:
- V MIN :
-
the maximum speed for walking human
- P HMIN , P HMAX :
-
the constant defining the minimum and maximum number of pixels, respectively, for the blob recognized as a person. The values of P HMIN and P HMAX are calculated to represent the area of 0.75 m2 and 3 m2, respectively, at the image after geometrical transformation.
- Q MIN :
-
the constant defining the minimum allowed object’s movement path quality q B .
The quality q B of the movement path of the blob B is calculated according to the equations:
where:
- \( {{\overline {\Delta {\text{X}}} }_{{{\text{B,t}}}}},\,{{\overline {\Delta {\text{Y}}} }_{{{\text{B,T}}}}} \) :
-
the average differences between the traveled distance by the blob B at two consecutive time moments t-1 and t for the direction x and y, respectively.
- Q :
-
acceptable change of \( {{\overline {\Delta {\text{X}}} }_{{{\text{B,t}}}}},\,{{\overline {\Delta {\text{Y}}} }_{{{\text{B,T}}}}} \) for consecutive frames, assumed value of Q = 2.
$$ {\text{BlobSizeChange}}\left( {{{\text{B}}_{\text{t}}},{{B}_{{{\text{t}} + 1}}}} \right) = max\left( {\frac{{\left| {{{\text{X}}_{{{\text{B,t}}}}} - {{\text{X}}_{{{\text{B,t}} + 1}}}} \right|}}{{max\left( {{{\text{X}}_{{{\text{B,t}}}}},{{\text{X}}_{{{\text{B,t}}}}} + 1} \right)}},\frac{{\left| {{{\text{Y}}_{{{\text{B,t}}}}} - {{\text{Y}}_{{{\text{B,t}} + 1}}}} \right|}}{{max\left( {{{\text{Y}}_{{{\text{B,t}}}}},{{\text{Y}}_{{{\text{B,t}}}}} + 1} \right)}}} \right) $$(5) - Distance(B, M):
-
the shortest distance between the outlines of two rectangles: B and M. If the rectangles overlap, the distance is 0.
$$ SpeedDifference\left( {{{\text{V}}_1}{,}{{\text{V}}_2}} \right) = \frac{{\left| {{{\text{V}}_1} - {{\text{V}}_2}} \right|}}{{max\left( {{{\text{V}}_1},{{\text{V}}_2}} \right)}}100\% $$(6) - AngleDifference(α 1, α 2):
-
the difference between two angles α 1 and α 2.
- Overlap(B, M):
-
returns % of overlapping area between the rectangles B and M with respect to the area of B.
- BlobAbsorbsModel(B, M):
-
the attributes’ values of the model M are copied to the blob B and the model M is deleted.
- ConsolidateModels(M a , M b ):
-
the models M a and M b are compared and the model which exists shorter is deleted, the remaining model is updated with the speed and moving direction as the average of the respective values from M a and M b .
The rules have been presented in a simplified form to aid the readability, in fact more conditions are checked, i.e. to protect from processing invalid data, division by 0, etc.
The moving models which leave the picture frame are filtered and only those considered as reliable are counted. The model is considered reliable if it comes from a blob that has been moving smoothly for some time (depending on the quality q B of the movement path) or it has been often overlapping with blobs moving with similar size, speed and direction. The traffic flow data are constantly updated, the moving vehicles are classified according to their speed and direction—this information is periodically transmitted by the radio to a nearby node which is closer to the data sink.
3 Data Transfer in the Sensor Network
To avoid the overheads of the standard wireless communication protocols such as 802.11 or 802.15.4, the authors decided to develop and implement the proprietary protocol optimized for this application. The proposed protocol has shorter lengths of the headers (Table 5), moreover, the header already contains information about the state of the node (battery state, data buffer occupancy), which is important for neighboring nodes in routing the traffic. Also, data length is suited to carry only important information on car traffic. The simple comparison of the proposed protocol to the standard 802.15.4 is shown in Table 6. The sink nodes have a simple serial interface with text commands, so connecting the network to a PC is very easy.
The access to the transmission medium is based on decentralized Time Division Multiple Access (TDMA). Each node is equipped with a radio transceiver module using ISM 869.5 MHz band utilizing data rate of 38.4 kbps with Manchester encoding. The nodes’ transmitters have the power of 500 mW (transmitter duty cycle <10 %), which helps in providing longer communication range in the harsh environment. The transmitter’s power can be remotely changed by the software to the values 40 mW, 125 mW, 250 mW and 500 mW, if needed. The dedicated operating system of the node, running on the main 32-bit on-chip processor, controls all the operations of the node and collects data from the image-processing part and the PP processors.
-
A.
Transmission Time t tx
Each node has its own unique address and a table for storing data about its neighbors. All the nodes use the same period T t of the transmission schedule, basing on their local clocks. Each node selects its own transmission start time t tx in the TDMA transmission period T t , providing:
where t tx,i is the transmission start time of the other previously discovered node i (this can be also an indirect neighbor, i.e. the neighbor of the neighbor), T MIN is a constant defined globally for the network, providing the required time distance between the transmissions. The times t tx and t tx,i are relative to the node’s local time source. The time distances of all the transmission start times t tx,i of the neighboring nodes are continuously monitored. When a node detects that among the neighboring nodes the condition (7) is not satisfied, it asks the problematic node to change its t tx .
-
B.
Beacon
The beacon contains basic information about the node, such as the node’s address, the distance of the node from the sink (measured in transmission hops), the node’s battery condition, the node’s data buffer occupancy and data about the node’s neighbors. Beacon is transmitted at t tx every N B data transmission periods, typical value of N B is from 3 to 10. Beacons are also used for establishing data links between the nodes. Each node infers from its neighbors’ beacons the existence of the indirect neighboring nodes, which helps to prevent the hidden terminal problem [2].
-
C.
Startup and Discovery of the Neighbors
At the startup, each node enters the discovery mode, when it continuously listens to the transmitted beacons, identifies neighbors and saves neighbors’ transmission times t tx,i . To find new nodes and accommodate the network topology changes, the discovery mode is periodically repeated. This is similar to the procedure used in S-MAC protocol [35].
There are four types of frames that can be transmitted through the network: beacon, data, acknowledge and configuration packets. All the frames have a 64-bit header of a similar structure. The beacon frame may contain information for the neighboring nodes needed for organizing the network (Table 7). The data frame carries the encrypted 128-bit packets (PKT) with car traffic information (Table 8), which can be aggregated from several nodes and transmitted together in a single frame. The acknowledge frame, transmitted just after the successful reception of the data frame, is used to confirm the reception of data, so the data transmitting node can free its buffer.
Each node switches on its receiver at the time t tx,i to listen to the possible transmission of the i-th neighbor. If the receiving node is not an addressee of the data transmission, it switches off its receiver just after receiving the header, in order to avoid overhearing.
Car traffic data in the sensor network are regularly transmitted from the nodes to the data sinks using data frames. To provide the ability of sending information in the reverse direction, i.e. from the sink to distributed nodes, the configuration frame is used. The configuration frame is transmitted without acknowledgement. Each node, after receiving a new configuration packet, simply retransmits it several times. This simple mechanism causes a heavy load of radio links but it is used rarely and only for service purposes, such as configuring remote nodes’ parameters, updating the firmware (FPGA nodes only) or the software.
Each data packet (PKT) contains the time stamp coming from the node’s real clock. The time stamp is used for checking the validity of data at the host computer. The sensor network utilizes global real clock synchronization scheme. The sync packets are periodically generated by the main sink node and distributed through the network by each node using the beacons.
-
D.
Data Protection
Each packet header is protected with 16-bit CRC, so the receiver can quickly verify the integrity of the header and the addressee of the packet without receiving the whole packet. Additionally, the payload has its own 32-bit CRC.
The header is transmitted using plain text, while the payload (PKT_DATA) is encrypted using AES with symmetric key. The hardware encryption, compared to the software encryption, has better energy efficiency measured in energy/bit, thus each node has the hardware encryption block. Decryption is used occasionally by the nodes (except the sink nodes), so it has been implemented in the software.
The 3 prioritized AES keys have been used: KEY1 and KEY2 are the same for all the nodes, the KEY3 is different for each node. During regular transmission, the KEY1 is used. To change the KEY1, the KEY2 is required. At a regular time interval, the change of the KEY1 (protected by KEY2) is commanded by the sink node. The KEY3 enables changes to the KEY1 and KEY2 and it should be used in case of a node’s takeover.
4 Hardware Realization of the Sensor Network
-
A.
Block Diagram of the Sensor Network Nodes
The block diagram of the node’s hardware, common for FPGA and ASIC prototype, is presented in Fig. 9. The main part of the hardware has been integrated into a Sensor Network Processing Module (SNPM), containing the custom microelectronic system with 32-bit processor BA12 from Beyond Semiconductor (the same class as Arm’s ARM9™) and the peripherals connected to the Wishbone bus. The hardware moving object detection system, described in section II, has also been integrated, together with the additional hardware blocks providing quick AES encryption and control of the low level operations of the transceiver (Fig. 10).
-
B.
Hardware Implementation
The first prototype sensor network node has been developed using Xilinx’s XC4VLX60 FPGA. The main board contains all the elements from Fig. 10 with the camera MT9V111 from Micron and the transceiver ARF29 from Adeunis. The power supply has been implemented on additional boards using 12 V 6 Ah gel cell sealed lead acid battery. The resources used in FPGA implementation are listed in Table 9 and the pictures of the prototypes with the FPGA are shown in Fig. 11.
After the successful startup with the FPGA, the ASIC has been designed in 130 nm UMC CMOS process and manufactured through Europractice (Fig. 12), using the RTL code from the FPGA prototype. Both FPGA and ASIC provide almost the same functionality. The design of ASIC has been made using Cadence SoC software with Faraday L130FSG (LP and HS) library of digital gates. The code consisting of ~95 000 lines of VHDL and Verilog has been synthesized using Cadence RTL Compiler with clock gating optimization. DFT flip-flops and JTAG controller have been added to enable future tests. For the implementation, Cadence SOC Encounter GXL 6.2 has been used, with crosstalk and signal integrity analysis and power supply analysis (electromigration, IR drop). The parameters of the designed ASIC are listed in Table 10. Comparing Tables 9 and 10, a large difference in the number of Flip Flops can be observed. The reason is that small memories and shift registers in FPGA are implemented using LUTs, while ASIC implementation uses FFs for that purpose. Moreover, in the FPGA all the memory is composed of 18Kbit block memories, resulting in wasted bits. Each memory block used in ASIC is designed to exactly fit the required size.
The manufactured ASIC has been used to build the low-power version of the node shown in Fig. 13, which works with Li-Ion 3.7 V 3.5 Ah single cell battery and can be supplied by a solar panel of area of 0.5 m2 (50 W peak power).
The set of the nodes has been installed on the street lamp-poles on several streets, as shown in Fig. 14. The detailed plan of the deployment is presented in the next section of the paper. The nodes have been using their batteries during the day; at night the batteries have been charging from the lamps’ power supply. The ASIC node consumes less power than the FPGA counterpart and therefore it is capable of working with a solar panel, instead of using the lamp’s power supply.
5 Simulation and Test Results
-
A.
Object Detection
The vehicles detected by the system have been verified on-line by the human operator. The road selected for traffic detection tests had various traffic conditions: from traffic jams to speeding vehicles. The sensor network node installed above the road was detecting and counting passing vehicles. Concurrently, the human operator was manually counting the cars. The comparison of the detection for 100 cars for various conditions has been presented in Table 11. More results, including the object segmentation simulations and videos, are available on-line at http://www.ue.eti.pg.gda.pl/sn.
As can be seen from Table 11, the image detection system recognized 63–93 % of the moving vehicles registered by the human. There were typically 1–9 % additional false detections of non-existent vehicles. During the sunny day, the most important problem are the shadows, resulting in joining the blobs from different cars. The basic shadow detection used in this system is not able to detect all the shadows. On a cloudy day, the most errors come from the dark cars of gray level similar to the color of the road. In this situation, the edge detection blocks are very helpful in increasing the detected pixel rate. The detection quality is significantly lower in the night, where mostly the car lights are detected. The achieved accuracy is the result of the compromises made during the design, such as a processing of monochrome and low resolution images and 4-bit representation of the pixel values, to obtain a low-power operation and simpler hardware. The detection rates are satisfactory for the collecting of the statistical data on average traffic flow and getting the overall general picture of the traffic. However, the authors feel that improvements in accuracy should be the next step in the further development of the presented idea.
-
B.
Network Data Transmission
The sensor network has been simulated to estimate the traffic parameters. The proposed sensor network is based on non-standard communication protocol and the use of popular simulation packages such as ns2 would require the creation of the exact communication model of the node. Instead of this, the authors decided to embed the exact copy of the node’s software into the custom simulator written in C++. In this way there is no need to create the separate simulation model of the node and the simulation results are very close to reality.
In the simulator, ideal radio links have been assumed, so all transmissions are successful. Assuming that each node regularly generates on average 9 bps of data stream containing information about the car traffic and basic node’s information such as battery condition or node’s neighbors, the maximal throughput for the data packets at the single radio link is 620 bps. Other parameters are listed in Table. 12.
Two basic network configurations of Fig. 15 have been simulated. For the network configuration shown in Fig. 15(a), the delay and radio activity have been analyzed. Both source nodes were generating the test messages consisting of single 128-bit packet. The total average delay of packets was 2.7 s, when the source nodes generate packets every t gen = 0.5–2 s (Fig. 16). The activity of the radio receiver and transmitter is shown in Fig. 17(a) and (b), respectively. The radio activity is highest for the middle node, as it has the largest number of neighbors. The average delay from the node 1 to the node n in the network configuration from Fig 15(b) is shown in Fig. 18. As can be seen, the average delay of packets per single hop in this configuration is almost constant and equal to approx. 1 s.
The effective radio range of the transceivers installed in real environment was about 900 m at the direct visibility of the antennas. The direct visibility can be easily achieved when the nodes are mounted on the street lamp-poles. After the installation of the nodes, the network organized itself and was correctly transmitting data to the sink, in the same way as during the simulation, except for sporadically occurring radio interference. The nodes have large data buffers for collecting unsent data (for at least 30 minutes); in the case of transmission problems, data is retransmitted at a later time. All the transmission times have been verified in practice and conform to the simulations. Apart from regular data about the car traffic, the operator can ask a node to send a single frame of the picture (Fig. 19), and uploading new firmware or software to the selected node, or all the nodes, is also possible.
-
C.
Power Consumption
The comparison of the power consumption of FPGA and ASIC prototypes is shown in Fig. 20. As could be expected, the main power savings are at the core in ASIC version. The other blocks (SDRAM, camera, I/O) have also slightly lower power consumption in ASIC prototype, but it is only due to the fact that they work with lower supply voltage (3 V instead of 3.3 V used at FPGA prototype).
-
D.
The Sensor Network Installation
The sensor network consisting of 26 nodes (22 FPGA-based and 4 ASIC-based) has been tested in real conditions. Data from the remote nodes are transferred to the sink node. The sink node is connected via RS-232 to the PC computer, which plays the role of operator’s console, where the collected data are visualized. The map of the installation is shown in Fig. 21, the installed nodes are configured to measure only the traffic along the street, as indicated by the arrows on the map. For the deployment of the nodes, the following criteria have been chosen: the vicinity of the university, the possibility to test multi-hop radio transmission, one-way and two-way streets and the various congestion of the nodes’ positions.
Each node accumulates information about the traffic and transmits it every 60 s. Finally, this information is presented at the console as simple histograms (Fig. 22), showing the number of vehicles detected for each direction. Each node maintains statistics about average tracks of the moving vehicles, so standing vehicles, for which the moving direction cannot be determined, shall be classified according to their location on the street. The project had a strictly scientific purpose, therefore the collected data were used only by the researchers, the controlling of the city lights was not introduced at this stage.
Apart from the current state of the traffic, the history of the traffic for each node and each direction can be displayed in the form of a time graph. The operator can also: configure a single parameter of each node, reset the node, update its software or download logs or single camera pictures, however these operations work slowly due to the low throughput of the radio network. Each node is also equipped with on-board temperature sensor and A/D converters for measuring the solar panel and battery voltages and currents. The proposed network protocol has dedicated data slots for transmitting 8-bit values of the measured temperature, noise, sun intensity and node’s internal battery state.
6 Conclusions
This work represents an attempt to solve the problem of measuring street traffic using a sensor network. For the purpose of video detection, dedicated multi-processor hardware and algorithms have been developed. The system has been realized as a relatively low-power device, with low hardware and software resource usage.
The video detection of the vehicles is realized using low-resolution images and simplified algorithms, thus its accuracy is not high, but it seems appropriate for a rough estimation of the traffic flow. The reduction of video data from 8 bits to 4 bits resulted in a decreased sensitivity for low-light scenes.
The sensor network nodes acquire data about car traffic and environmental parameters, such as temperature or the voltage from the solar panel, and they transmit them by the radio in the compressed form to the node closer to the sink. Other measured parameters, such as acoustic noise level, can be easily added, giving the possibility to monitor the noise in the area of the city. For that purpose, a self-organizing multi-hop radio network protocol has been designed, responsible for downloading data from the remote nodes to the sinks. Data are aggregated at the nodes to decrease the number of radio transmissions. The radio protocol also enables the uploading of data to configure the remote nodes. Despite using 500 mW transceivers, the transmission range was only 900 m, which requires dense location of the nodes. In some situations, it might be more convenient to install GSM transceivers for the remote nodes.
The sensor network node has been practically realized in two versions: FPGA and ASIC and compared with each other. The network consisting of 26 nodes has been tested in real conditions for more than 12 months. The achieved detection rate (as shown in Table 11) was enough to report the traffic flows in the monitored area. The radio transmission performance was very close to the simulation results. In the case of transmission problems, the large nodes’ data buffers were able to buffer data locally for up to 30 min, waiting for the radio link improvement. The installation was successfully collecting data, which were recorded and visualized on a PC, proving the usefulness of this idea.
References
Akyildiz, I. F., Su, W., Sankarasubramaniam, Y., & Cayirci, E. (2002). “A survey on sensor networks”. Communications Magazine IEEE, 40(8), 102–114.
Karl, H., Willig, A. (2007) Protocol and architecture for wireless sensor networks, John Wiley and Sons, Ltd.
Stojmenovic, I. (2005). Handbook of sensor networks, John Wiley and Sons, Inc.
Yick, J., Mukherjee, B., & Ghosal, D. (2008). Wireless sensor network survey. Computer Networks, 52(12), 2292–2330.
Leduc, G. (2008). “Road traffic data: Collection methods and applications”, JCR technical notes. European commission, joint research centre, institute for prospective technological studies
“Traffic Detector Handbook: 3rd edition”, vol I, II, FHWA-HRT-06-139, FHWA-HRT-06-108, US Department of Transportation, Federal Highway Administration 2006.
Eng-Han Ng, Su-Lim Tan, Guzman, J.G. (2009). “Road traffic monitoring using a wireless vehicle sensor network”, Int. Symposium on Intelligent Signal Processing and Communications Systems ISPACS 2008, pp. 1–4, 8–11
Yang, S.S., Kim, Y. G., Choi, H. (2007). “Vehicle identification using wireless sensor networks”, Proc. IEEE SoutheastCon, 2007, pp. 41–46, 22–25.
Kastrinaki, V., Zervakis, M., & Kalaitzakis, K. (2003). A survey of video processing techniques for traffic applications. Image And Vision Computing, 21(4), 359–381.
Kumar, P., Ranganath, S., Weimin, H., & Sengupta, K. (2005). “Framework for real-time behavior interpretation from traffic video”. IEEE Transactions on Intelligent Transportation Systems, 6(1), 43–53.
Friedman, N., Russell, S. (1997). “Image segmentation in video sequences: A probabilistic approach”, Proc. Thirteenth Conf. on Uncertainty in Artificial Intelligence (UAI 97).
Stauffer, C., Grimson, W. (1999). “Adaptive background mixture models for real-time tracking”, Proc. IEEE Conf. Compu. Vis. Pattern Recognit., pp. 246–252.
Veeraraghavan, H., Masoud, O., Papanikolopoulos, N. (2003). “Computer algorithms for intersection monitor”, IEEE Trans. Intell. Transp. Syst., no. 2, pp. 78–89.
Chiu, C., Ku, M., & Liang, L. (2010). A robust object segmentation system using a probability-based background extraction algorithm. IEEE Trans. Circuits Syst. Video Technol., 20(4), 518–528.
Cucchiara, R., Piccardi, M., & Mello, P. (2000). Image analysis and rule-based reasoning for a traffic monitoring system. IEEE Transactions on Intelligent Transportation Systems, 1(2), 119–130.
Ferrier, N.J., Rowe, S. M., Blake, A. (1994). “Real-time traffic monitoring”, in Proc. 2nd IEEE Workshop on Applications of Computer Vision, Sarasota, FL, pp. 81–88.
Koller, D., Daniilidis, K., & Nagel, H. H. (1993). Model-based object tracking in monocular image sequences of road traffic scenes. Int. J. Comput. Vision, 10(3), 257–281.
Zhou, J., Gao, D., & Zhang, D. (2007). Moving vehicle detection for automatic traffic monitoring. IEEE Trans. Vehicular Technology, 56(1), 51–59.
Gorgon, M., Pawlik, P., Jablonski, M., Przybylo, J. (2007). “FPGA-based road traffic videodetector”, in Proc. 12th Euromicro Conf. on Digital System Design, Architectures, Methods and Tools (DSD '09), Lubeck, Germany.
Pamula, W. (2009). “Vehicle detection algorithm for FPGA based implementation”, Computer Recognition Systems 3, Springer Berlin/Heidelberg, 57: 585–592.
Chen, P., Ahammad, P., Boyer, C., Shih-I Huang, Lin, L., Lobaton, E., Meingast, M., Oh, S., Wang, S., Yan, P., Yang, A. Y., Yeo, C., Chang, L-C., Tygar, J. D., Sastry, S. S. (2008). “CITRIC: A low-bandwidth wireless camera network platform”, Second ACM/IEEE International Conference on Distributed Smart Cameras ICDSC 2008, pp.1–10.
Xu, L., Bu, W. (2011) “Traffic flow detection method based on fusion of frames differencing and background differencing”, Second Int. Conf. Mechanic Automation and Control Engineering (MACE), pp.1847–1850.
Kwon, C.H., Park, H.J. (2004). “A Study on Camera-Detector System for Traffic Control in Korea”, Lecture Notes in Computer Science, Conceptual Modeling for Advanced Application Domains 3289: 566–576, Springer.
Wang, C., & Lien, J.-J. J. (2008). Automatic vehicle detection using local features—A statistical approach. IEEE Transactions on Intelligent Transportation Systems, 9(1), 83–96.
Junior, O. L., & Nunes, U. (2008). “Improving the generalization properties of neural networks: An application to vehicle detection” (pp. 310–315). Oct: Proc IEEE Conf. Intell. Transp. Syst.
Sun, Z., Bebis, G., & Miller, R. (2006). Monocular precrash vehicle detection: Features and classifiers. IEEE Transactions on Image Processing, 15(7), 2019–2034.
Arrospide, J., Salgado, L., Nieto, M. and Jaureguizar, F. (2008). “On-board robust vehicle detection and tracking using adaptive quality evaluation,” in Proc.IEEE Int. Conf. Image Process. pp. 2008–2011.
Chan, Y., Huang, S., Fu, L., & Hsiao, P. (2007). “Vehicle detection under various lighting conditions by incorporating particle filter” (pp. 534–539). Oct: Proc IEEE Conf. Intell. Transp. Syst.
Wójcikowski, M., Żaglewski, R., & Pankiewicz, B. (2012). FPGA-based real-time implementation of detection algorithm for automatic traffic surveillance sensor network. Journal of Signal Processing Systems, 68(1), 1–8.
Cucchiara, R., Grana, C., Piccardi, M., Prati, A. (2000). “Statistic and knowledge-based moving object detection in traffic scenes”, in IEEE Proc. Intell. Transp. Syst., Dearborn, MI, pp. 27–32.
Elgammal, A., Harwood, D., Davis, L.S. (2000). “Non-parametric model for background subtraction” in European Conf. Computer Vision, Dublin, Ireland, vol. II, pp. 751–767.
Duque, D., Santos, H., Cortez, P. (2005). “Moving Object Detection Unaffected by Cast Shadows, Highlights and Ghosts”, in Proc. IEEE Int. Conf. Image Processing, pp. 413–416.
Porikli, F., Ivanov, Y., Haga, T. (2008). “Robust abandoned object detection using dual foregrounds”, EURASIP J. Adv. Signal Process.
Cucchiara, R., Granna, C., Piccardi, M., Prati, A., Sirotti, S. (2001). “Improving Shadow Suppression in Moving Object Detection with HSV Color Information”, in Proc. IEEE Intell. Transp. Syst. Conf., Oakland, CA, pp. 334–339.
Ye, W., Heidemann, J., & Estrin, D. (2002). “An energy-efficient mac protocol for wireless sensor networks”. New York: Proc INFOCOM 2002 IEEE Press.
Acknowledgments
This work was supported in part by the Polish Ministry of Science and Higher Education under R&D grant no. R02 014 01.
Open Access
This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
Author information
Authors and Affiliations
Corresponding author
Additional information
Index Terms—wireless sensor networks, sensor systems, object detection
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Wójcikowski, M., Żaglewski, R., Pankiewicz, B. et al. Hardware-Software Implementation of a Sensor Network for City Traffic Monitoring Using the FPGA- and ASIC-Based Sensor Nodes. J Sign Process Syst 71, 57–73 (2013). https://doi.org/10.1007/s11265-012-0681-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-012-0681-7