Automatic Sound Recognition of Urban Environment Events

Over the last years, the recognition of person’s communication activities (mostly phenomena with cognitive index like speech, gesture and node) through an increasing number of areas of interest drove the automatic processing tools to incorporate analysis about person’s surrounding. One of the common audio surroundings is the urban environment. The researchers of article [20] divide the urban events according to their origin (humane, biological and geophysical) and host conclusions about their affect on peoples’ health, psychology, economic and lifestyle behavior, cognitive needs and etc.

An analytic taxonomy of the urban soundscape is presented in article [18]. Reviewing the bibliography, many tasks have investigating the audio events occurring in urban environments [2, 5, 8, 15, 21, 23]. Article [12] examines the landscapes as an importance parameter of study. Event of urban origin can also be sought in other studies [4, 6, 9, 10, 11, 14, 16, 17, 22]. As can be seen, events related to the transportation system of a city [8, 10, 14, 15, 18, 23] and to weather phenomena [9, 10, 14, 15, 18] are quite common in recordings. Typical for an urban environment (in addition to rural and non-industrial environments) is the occurring construction activity [18] which includes interesting events of the urban soundscape. Moreover, people ability to be sound as crowd noise (in terms of acoustics) [10, 14, 15, 17, 23] and the vocalizations of domestic animals and urban wildlife [4, 11, 16, 18, 23] can easily be captured in audio recordings. Local customs, traditions and the technology (like horn/siren, phone ring and bell) [4, 11, 18, 22] are also related to acoustic phenomena of the urban soundscape.

There are several works that proposed architectures that address unexpected audio events occurring on urban surrounding in speech or communication audio signals. In those architectures sound recognition precedes all other stages, and thus sound recognition stands as cornerstone of the audio processing. Typical sound recognition consists of short-time analysis in time and frequency domain, dividing the audio sequence into intervals. Audio descriptors are used to represent those intervals and pattern recognition algorithms build models to categorize those intervals with sound label from the set of events of interest.

Seeking for audio descriptors to distinguish the characteristics of urban events, several parameters of both time and frequency domain have been proposed. The zero crossing rate [9, 14, 17] and the Mel Frequency Cepstral Coefficients [10, 15, 18, 22] are correspondingly typical examples of time and frequency domain that commonly have been used. Meanwhile, several studies presented a variety of descriptors, like the MPEG-7 descriptors [6, 11, 15], the Perceptual Wavelet Packets [15], the Linear Prediction Code [8, 10], the Matching Pursuit [10], the pitch [17] and the spectral statistics [9, 14, 17].

A variety of deterministic and probabilistic machine learning algorithms have been selected to evaluate experimental frameworks of urban environment events. Some of the well known and commonly used algorithms are the Neural Networks [8, 10, 21], the Support Vector Machine [6, 17, 18, 22, 23], the decision trees [17, 18], the k-nearest neighbors [10, 17] and the hidden markov models [4, 6, 11].

In this work, we present a sound recognition methodology for distinguishing audio events of the urban environment based on widely known and commonly used audio descriptors. We combine them with a data-driven ranking algorithm for investigating their necessity and relevance to our audio task. Consequently, we were concentrating on our framework hypotheses about the coping of our classifiers with the dimensionality redundancy and changes in classifier’s effectiveness while irrelevant to the task descriptors haven’t been discarded.

The rest of the article is organized as follows. In Sect. 2 we present an analytic architecture. In Sect. 3 we introduce the experimental framework and derive details about the database, the audio descriptor and recognizer selection and the ranking technique. In Sect. 4 we described the experimental results. Finally, Sect. 5 follows with the conclusions.

In the proposed scheme the architecture for distinguishing audio events, found in urban environment recordings, relies on short-time analysis in both time and frequency domain. A ranking technique is applied to the audio descriptors to score their discrimination ability with respect to the events of interest in this task. The scheme of this architecture appears in Fig. 1. As it seems, the architecture is divided into training and testing phase.

Open image in new window

During the training phase, the training set of recordings \(X=\{ X^r \}, r\in [1,R]\), which is previously be annotated with label tags and includes the whole and close set of events of interest of urban environment origin, are sequentially driven though the stages of short time analysis. Initially, the preprocessing stage frame blocks the recordings into sequences of overlapping frames with constant length and time-shift \(O=\{ O^r \}\),. Afterwards these frame sequences are decomposed using a close set of audio descriptors, provided in the feature extraction stage. The outcome feature vector sequence \(V=\{ V^r \}\), consists of N feature in frame level organized in feature vectors \(V^r=\{ V_i^r \}, i\in [1,N]\),. Thereafter, the ranking score measures the discrimination ability of each feature. The output rankings \(S_i^r\) are used to create feature subspaces. Within the sequence is discarded from those features with less significance. The threshold D that defines the boundary of necessity of a feature is manually defined or determined by data-driven criteria. The new sequences \(P^r=g(V^r,S_i^r,D)\) are driven into the training steps of the classification. Within model \(M^D\) is trained from the sequence \(P^r\). During the testing phase, the examining audio file \(X^Y\) is pre-processed with the frame block procedure to be similar to the one of the training phase producing the frame sequence \(O^Y\). Afterward the selection of features which was done during the training phase is feeding the feature extraction block in order to decompose the frame sequence with only those descriptors attached to the working subspace. The extracted sequence \(P^Y=g(O^Y,S_i^r,D)\) is finally driven to the classifier, where the results are constructed \(w=f(P^Y,M^D)\) based on frame level classification according to the label tags previously annotated during the training phase. Further post processing of the results is performed for fine-tuning (either on decision level or on classification scores). This architecture allows exploiting feature subspaces, which contribute to accurate discrimination of events of interest with simultaneously dimensionality redundancy by examining and discarding non-preferable descriptors.

As can be seen in Table 2, the best performing algorithm was the support vector machines with radial basis function kernel, which achieved classification performance equal to \(92.73\,\%\). The second best performing was the MLP neural network with \(90.59\,\%\) accuracy. The results show that the two evaluated discriminative algorithms outperformed the rest of algorithms.

In terms of sound types, the most misclassified sound type pairs were the wind sound and the motor engines sounds, which for the best performing SVM algorithm were found to be misclassified by approximately \(5\,\%\). All the rest misclassified sounds were found in less than \(2\,\%\) for all cases.

The increasing influence of speaker’s surroundings drove the interest of the scientific research interest towards to sound recognition of events from person’s environment. Since urban environment is a quite common environment, it becomes into cornerstone of environments of interest. In the present work, we studied a methodology of automatic sound recognition with a short-time analysis framework using urban environment events. After computing a large set of audio descriptors we applied an ranking algorithm to score the descriptors’ discrimination ability, in terms of necessity and relevance on the current task. With some well known and commonly used machine learning algorithms we evaluate this framework. Our results point out that SVMs managed to outperform all other algorithms with accuracy equal to \(92.73\,\%\). Also the sequential expansions of the working descriptor space with unnecessary dimensions don’t significantly affect the effectiveness of the machine learning algorithms.