A Novel Way to Measure and Predict Development: A Heuristic Approach to Facilitate the Early Detection of Neurodevelopmental Disorders


Purpose of Review

Substantial research exists focusing on the various aspects and domains of early human development. However, there is a clear blind spot in early postnatal development when dealing with neurodevelopmental disorders, especially those that manifest themselves clinically only in late infancy or even in childhood.

Recent Findings

This early developmental period may represent an important timeframe to study these disorders but has historically received far less research attention. We believe that only a comprehensive interdisciplinary approach will enable us to detect and delineate specific parameters for specific neurodevelopmental disorders at a very early age to improve early detection/diagnosis, enable prospective studies and eventually facilitate randomised trials of early intervention.


In this article, we propose a dynamic framework for characterising neurofunctional biomarkers associated with specific disorders in the development of infants and children. We have named this automated detection ‘Fingerprint Model’, suggesting one possible approach to accurately and early identify neurodevelopmental disorders.


Early human development has attracted increasing attention from researchers across scientific disciplines in recent years. This expansion has been driven, in part, by the acknowledgement of the great diversity of (neuro)developmental disorders, their large genetic and phenotypic heterogeneity and the clear need to better understand similarities and differences across syndromes, disorders and disease processes. A continuing series of new paradigms on the functional development of the young nervous system has resulted in progressive changes in approaching typical and atypical early development.

Research on early development during the last decade, in particular, has benefitted greatly from adapting approaches and theories from not only closely related disciplines, such as developmental neuroscience, but also from novel technical and computational fields (e.g. computer vision, machine learning, signal processing, speech and voice analysis). These recent trends are moving towards the disappearance of classic discipline boundaries in investigating early human development and the specificities of various disorders [1, 2].

In this overview—and perspective—paper, we propose a research model aimed at providing (i) a framework for age-specific cross-syndrome comparisons and (ii) eventual prediction of neurodevelopmental outcome. This model may allow for the recognition of syndrome-specific developmental traits and provide an opportunity for earlier identification of disorders, often recognisedFootnote 1 at toddlerhood or even later. These particular conditions shall henceforth be referred to as ‘conditions of interest’ (COI), and include for example autism spectrum disorder (ASD), attention deficit hyperactivity disorder (ADHD), Rett syndrome (RTT) and fragile X syndrome (FXS). COI will also apply to infants who have suffered potential deleterious environmental exposures such as, in utero exposure to mosquito-borne diseases like Zika virus (ZIKV), malaria or exposure to sexually transmitted maternal diseases, teratogenic compounds and/or maternal substance abuse.

Ontogenetic Adaptation

The concept of ontogenetic adaptation and continuity of neural functions [1, 3, 4] plays a central role in our interdisciplinary scientific approach to study infant development. Physiologically speaking, life ex utero as compared to in utero needs to adapt to dramatically different environments and requirements. For survival, it is essential that newborns immediately adapt to their new environment and be embedded within a system that efficiently meets their needs (endogenous or exogenous system perturbations/adaptations; e.g. respiration, nutrition). From our perspective, the most significant changes during this period of development occur in the nervous system, which undergoes the most dramatic, almost permanent adaptation and optimisation during the early postnatal years [1, 3, 5].

Compared to non-human primates, and factoring out vital-functions, we are far less equipped for instant adaptation to the extra uterine life [5,6,7]. As a consequence, the newborn human is ‘by no means the competent individual which has been sometimes proposed’ [1: 837]. A series of studies on early human postnatal development have indicated that the first 2 months after term are, to a certain extent, a continuity of foetal behaviour [1, 4, 8,9,10]. Around the end of the second month of life, a major transformation sets in (the 3-month-transformation [4, 5]) and many neural functions change or (gradually) occur leading to a neurobehavioural adaptation to the requirements of extra uterine life. This seems to be specific to the human species and even though we know a great deal about development, the period of the first few months of life is not yet fully understood. However, what we do understand is that this transformation can be characterised by a number of neurofunctional changes, such as an increase in muscle power, postural changes [11, 12], a change in the sucking pattern [13], development of focused visual attention and binocular vision [14, 15] and the beginning of social smiling and cooing vocalisations [16,17,18].

Neurological Underpinnings of Behavioural Changes During Infancy

Focusing initially on brain development, the early postnatal period is an intense phase of structured growth and expansion [19,20,21]. Anatomical changes follow a predictable course in the typically developing brain, when measured using non-invasive cerebral magnetic resonance imaging (MRI), and seem to follow a sigmoidal growth pattern in terms of brain volume [22] as well as white matter structure and content [23, 24]. Functional neuroimaging techniques, such as positron emission tomography (PET), demonstrate concomitant increases in glucose utilisation at around 2 to 3 months, particularly in the parietal, temporal and primary visual cortex, basal ganglia and cerebellar hemispheres. These escalations in metabolism coincide with the emergence of behaviours involving visuospatial and visuo-sensorimotor integration, disappearance or reorganisation of subcortical neonatal behaviours and evidence of increasing cortical activities [25].

At typical term birth, inter-neuronal axonal connections are largely unmyelinated and dendritic sprouting and synaptogenesis is still ongoing. Though the white matter architecture underlying brain connectivity is largely established by term [26], in the first year of life and especially the first 8 months [27], there is a rapid expansion in the spatial distribution and extent of myelination [23, 28] and dendritic sprouting [29]. Linked to this, ongoing proliferation of synapses leads to peaks of synaptic density at the ages of 6–18 months, after which the rate of synaptic pruning overtakes proliferation and a protracted decline begins [30]. Importantly, the rate of myelination, synaptogenesis and synaptic pruning is regionally specific. Generally, the subcortical, cerebellar and primary sensory areas peak in terms of synaptic density first, their connections are myelinated first, and synaptic pruning begins earlier, when compared with other cortical association areas and in particular with the frontal cortex.

What remains unclear is the extent to how these changes associate with cognitive and neurofunctional/behavioural development. It is tempting to assume that behavioural development maps easily onto the maturation profile of the brain itself (e.g. motor and sensory abilities first, then higher-order cognitive functions). Longitudinal studies evaluating MRI at term age and follow-up with neuromotor, behavioural and cognitive assessments during later childhood have also shown links between anatomical features from MRI and later outcome [31, 32]. Furthermore, individual differences in brain growth trajectories have been associated with individual differences in cognitive development at the same age [33]. In all three studies, using three different MRI modalities, features in the subcortical and especially thalamic-associated white matter were the consistent areas that predicted later, or current, language ability and developmental quotient (e.g. areas that mature earlier). It should be noted that only referring to synaptogenesis and myelination is an oversimplification of the complex neurodevelopmental processes, given the developmental changes of neurotransmitter modulation, neuronal differentiation, cortico-cortical connectivity, glia development, etc. that are also taking place.

Given these overlapping early brain processes, it is perhaps not surprising that studying the early brain (before 12 months) has become a larger focus for understanding COI [34,35,36].

Developmental Domains of Interest

Development can be seen as a complex autopoietic system, but to better understand the functionality and deviances within it, we need to highlight specific features of this system. Aspects of two major developmental domains (motor- and speech-language) during the first months of life are outlined. Within the scope of this paper, general movements (GMs) and early vocalisations are discussed with respect to both precursors/prerequisites for further development and potential early indicators of neurodevelopmental disorders.

Central Pattern Generated Spontaneous Motor Behaviour: The General Movements

Embryonic, foetal and neonatal movement patterns all share characteristics of being endogenously generated. Without being triggered by a specific sensory input, the foetal and neonatal nervous system generates a variety of motor patterns such as startles, general movements, breathing movements, stretching, yawning, sucking, side-to-side movements of the head (rooting) or eye movements [9]. These movement patterns are generated by specific neural networks, the central pattern generators (CPGs), which are located in the brain stem [1, 8, 9, 37]. Some CPGs operate continuously (e.g. respiration), whereas others are activated to perform specific tasks (e.g. sucking, locomotion). In order to lend variability to the motor output, supraspinal projections activate, inhibit and most importantly modulate the CPG activity as does sensory feedback [38•, 39].

The fact that CPG activity was, and partly still is, either overlooked or misinterpreted as reflexes (or even worse as ‘primitive’ reflexes although there is nothing primitive in the developing nervous system) stems from classical neurophysiology. At the end of the 19th century, foetal and neonatal movements were recognised as spontaneously generated; however, reflexology and particularly behaviouristic interpretations tended to ignore these observations [1, 9]. Sir Charles Sherrington studied the contact between the afferent and efferent arch in the spinal cord and introduced experimental lesions to the nervous system in order to eliminate the ‘nuisance’ of spontaneous neural activity. In this way, the relation between stimulus and reflex was extremely consistent as it was not interfered with fluctuations caused by spontaneously generated activity. Although Sherrington [40] himself was fully aware of the artificial nature of his findings and even mentioned that the simple reflex is a fiction, his followers seemed to have ignored this cautionary note and made the reflex pattern the crucial element of neural functions [1, 8, 9].

During the last 20 years of infant studies, attention has shifted from exclusively testing reflexes to additionally assessing spontaneous movements. From the rich repertoire of distinct spontaneous movement patterns to emerge during infancy, the so-called GMs are the most frequently occurring and most complex. During preterm and term age, GMs involve the entire body and manifest themselves in a variable sequence of arm, leg, neck and trunk movements. At a post-term age of 3–5 months, GMs appear as fidgety movements, which are small movements of the neck, trunk and limbs in all directions with variable acceleration [8, 10, 37, 41]. The presence of normal GMs is indicative of normal neurological development, whereas abnormal, monotonous GMs point to neurological deficits. Specifically, the absence of fidgety movements (at 3–5 months) is typically associated with the development of cerebral palsy [37, 41, 42]. In addition to its application in infants with perinatal brain injury, the general movement assessment (GMA) has also been applied to—for example—intrauterine HIV-exposed and/or HIV-infected newborns and young infants [43], infants of mothers with ZIKV infection [44], infants with metabolic disorders [45], infants with genetic disorders [46,47,48,49] and infants with ASD [50•, 51].

GMA is not only non-intrusive and cost effective but also has repeatedly proven to be an accurate and reliable assessment tool [41]. In a recent review, Bosanquet and colleagues [42] reported summary estimates of sensitivity and specificity between 98 and 91%, respectively, for the prediction of cerebral palsy. Currently GMA is being used by an increasing number of health professionals around the world for assessment and identification of infants at high-risk for neurological impairments. With this in mind, we are developing a mobile solution allowing for broader application (the GMApp; www.gmapp.idn-research.org; [52]). More recently, complementary efforts have been taken to augment classic GMA, based on visual Gestalt perception, with computer-based movement assessment tools to perform quantitative analysis of GMs [53,54,55,56,57, 58•].

Early Vocal Development

In addition to the production of vegetative sounds, infant vocal development during the first weeks of life is mainly characterised by the generation of distress vocalisations, i.e. crying or fussing sounds [59]. At the same time, though a relatively small proportion, the first vowel-like/quasi-vowel non-distress sounds emerge. Most of these sounds do not excite the vocal tract’s full resonance yet and phonation does not involve a distinct systematic mouth opening [17]. These very first vocalisations have also been discussed to be CPG-generated behaviour [60]. Basic understanding of the CPG circuitry for mouth movements and respiration suggests multiple foci in the brain stem [61]. The increasing level of cortical control over sound production leads to the emergence of a more complex type of vocalisation which is representative for the period around 3 months of age, the cooing sound. Often, cooing sounds develop alongside the onset of social smiling in situations of face-to-face interactions [59, 62]. Cooing sounds are (often velar) consonant-like elements, such as voiced fricatives, optionally combined with vowel-like segments [17, 18] with distinct melodic contour [63] representing the first forms of syllables [64]. These sounds vary substantially in structure, quality and temporal organisation as infants certainly have not yet reached full phonetic competence [17, 62]. The production of cooing sounds represents an important step in early vocal development, as for the first time, the emergence of discernible tongue movements, required for typical phonation, becomes apparent [62].

Around the 4th–5th months of age the infant starts to explore the full potential of his/her vocal apparatus and to expand the vocal repertoire by generating sounds with modulations/variations in melody/pitch, loudness and vocal register. These sounds include raspberry vocalisations, squealing, vowel-like elements and more complex marginal babbling (i.e. first slow and shaky transitions between consonant-like and vowel-like sounds). [17, 59, 63]

Deviations from typical early vocal development have been discussed as potential early signs of developmental disorders with a mean age of diagnosis at or beyond toddlerhood (i.e. COI introduced previously). For instance, Patten et al. [65] reported a late onset and low volubility of canonical babbling in infants later diagnosed with ASD. A limited number of studies have focussed on the transition from marginal babbling to canonical or variegated babbling and first word production in neurodevelopmental disorders, with the majority of these studies focussed on volubility measures. Fewer studies have analysed early atypicalities in vocalisation quality measurable by means of acoustic signal level parameters. Furthermore, the multitude of these investigations has been limited to specific vocalisation types, in many cases crying vocalisations, and to a small number of acoustic parameters such as fundamental frequency or (cry) duration [66,67,68]. In promising pilot work, we demonstrated the potential of more detailed acoustic early vocalisation information retrieval for the identification of infants later diagnosed with RTT [69, 70, 71•, 72].

Developmental Disabilities with a Late Manifestation or Clinical Onset: A Blind Spot

To study early human development and anticipate neurodevelopmental outcomes, key aspects such as the presence and development of distinctive physical and neurological features and potential neurobehavioural and psychopathological abnormalities are in the focus of interest. Intensive research on genetic disorders, for example, have revealed that certain physical features are associated with specific disorders. In some disorders, these features can be apparent before or immediately after birth (e.g. facial dysmorphia in Down syndrome) and they may also be accompanied by early functional abnormalities that enable an early and accurate diagnosis (e.g. atypical vocalisations in Cri du Chat syndrome). As previously mentioned, there are a number of conditions with no apparent physical features at birth (e.g. ASD, FXS, RTT) and it is only during later development that they reach a saliency threshold leading to a characteristic appearance which contributes to accurate diagnosis [48, 51, 70, 73,74,75,76]. When dealing with COI, such as ASD, FXS or RTT for example [73, 77, 78], a formal diagnosis is often given at or beyond toddlerhood. However, parents often raise concerns relating to their child’s delayed or atypical development long before diagnosis [79, 80]. This can lead to substantial difficulties due to long periods of uncertainty where they often follow the path of a diagnostic odyssey.

Let us take FXS as an example. It is the most prevalent form of inherited intellectual disability and one of the most widespread genetic causes of ASD [81, 82]: The majority of clinical features may not be detected or not associated with FXS at an early stage, and thus genetic testing and formal diagnoses are often delayed until, on average, the preschool years [83]. This delay in diagnosis has significant implications for access to early intervention programmes, and results in additional costs and frustration for families. Further, it is not uncommon for additional children to be born with FXS during this time [83]. To remain undetected beyond toddlerhood is not unique to FXS; the same is true for a number of COI that need to be studied in more detail.

A late diagnosis reduces the possibility to study early development and thus hampers research on very young individuals with ‘late’ recognised conditions. Whilst a large number of studies have relied on retrospective parental questionnaires, there has been increasing doubt about the validity of these investigations [79, 84•]. For certain assessments, retrospective video analysis (RVA) is considered to be a more objective method to circumvent the risk of memory bias [84•, 85, 86]. Besides, there has been a lot of effort into prospective studies analysing high-risk populations, e.g. studying infants with an older sibling with ASD or ADHD and prospective studies of infants with single or multiple exposure to intrauterine hazards [87, 88•, 89•, 90,91,92]. Prospective studies of infants at high risk for ASD, for example, use serial longitudinal direct assessments of behaviour to investigate timing and patterns of symptom onset. These studies have demonstrated that neurofunctional abnormalities may occur even earlier than parents report, in the later half of the first year of life, and may follow a declining pattern, in which infants transition from typical to atypical developmental trajectories over the course of several months.

It goes without saying that a transition from purely CPG-generated activities (e.g. GMs, early vocalisations) into more intentionally controlled behaviours (e.g. movements towards the midline and reaching; modulation in pleasure vocalisations increasingly associated with looking towards the caregiver) gradually evolves with the coexistence of both early spontaneous behaviours and intentionally driven behaviours [4, 10].

A Novel Approach: The Fingerprint Model

Constantly adapting and optimising neurological functions need to be seen and understood in their development and complexity, i.e. we need to aim for the big picture whilst avoiding the hazards of oversimplification. We need to account for age-specificity when examining children, keeping in mind that this can account for different ‘vulnerabilities’ and the consequential effects for early intervention efficacy [1]. Building upon a concept devised by Prechtl to assess the integrity of the nervous system, it is crucial to (i) take into account the age-specific properties of the nervous system; (ii) to avoid the artificial fragmentation of performances or signs; (iii) to indicate defects but also delays and (iv) take into account the behavioural state of the foetus/infant/child [93].

The ultimate aim of our proposed approach—combined with other biological methods and markers—is to earlier detect suboptimal and pathological development, and, as mentioned at the beginning, achieve a ‘find-early-and-intervene-early’ reality to improve developmental outcomes and avoid diagnostic odyssey.

To achieve this, we propose a heuristic and probabilistic ‘Fingerprint Model’ (Fig. 1). Such a model requires large age-specific datasets from various developmental domains of typically developing infants and infants with COI. One may however consider this model in the light of neuroconstructivism as an autopoietic model that is already functional whilst it is ‘growing’ [99]. It specifically adopts a maximum likelihood approach whereby age-specific parameters of interest (POI, such as fidgety general movements, one aspect of the motor domain at around 3 months of age; Fig. 2) are used to reliably differentiate between specific COI. Although optimal performance of the model relies upon large and accurate datasets spanning specific age ranges and neurofunctional domains, it can still be effective in providing accurate developmental outcome predictions using incomplete POI datasets. Furthermore, this model has the potential for progressive development in the use of additional POI or newly identified biomarkers which may help to improve accuracy levels of detection. Thus, this model has the potential to develop and expand alongside with new developments across various fields in early human development. The proposed model will encompass analyses at multiple age ranges and include atypical trajectories to detect developmental delays in addition to developmental deviations from typical development [100].

Fig. 1

Proposed ‘iDN Fingerprint Model’ for the earlier detection of COI. Our goal is to unravel the early fingerprint of various COI as age-specific POI constellations in an extensive knowledge tensor. Thereby, fingerprint information is modelled in terms of atypicality in objective parameters from POI-related approaches (e.g. GMA in motor development, or AVA of cooing sounds in speech-language development) and underlying levels of representation (e.g. state-of-the-art signal attributes used in audio/video analysis, such as optical flow [94,95,96], zero-crossings rate, harmonics-to-noise ratio, jitter [97]). Finally, we propose the implementation of a probabilistic model to automatically detect COI from multidimensional data for future clinical application, e.g. by means of logistic regression [98]. AVA acoustic vocalisation analysis, COI condition of interest, GM general movement, GMA general movement assessment, GUARDIAN Graz University Audiovisual Research Database for the Interdisciplinary Analysis of Neurodevelopment, mo month, POI parameter of interest, TD typical development, Colour code: green optimal/normal, orange suboptimal, but within the range of normality, red atypical)

Fig. 2

Illustration of a multi-device infant recording setup (schematic on the left; sensor view on the right): two HD video recordings from different angles (top right), two Kinect recordings from different angles (middle right), motion tracking (bottom left) and contact pressure distribution measurement (bottom right) when lying in supine position

It remains open whether we will be able to define syndrome specific constellations in early infancy or rather identify general signs of atypical development. It is unlikely there will be one signature behaviour in early infancy but rather symptom constellations that pinpoint a certain disorder or subset of disorder (i.e. a fingerprint, an age-specific POI constellation). This approach need not be restricted to neurofunctional assessments but rather be combined with classical biomarker research (most likely a multimarker panel of biomarkers generated from different levels of biological analysis; [101•, 102, 103]). This approach provides a likelihood model that precedes clinical and genetic testing to verify a given suspicion (derived from the model). Our proposed Fingerprint Model should be seen as suggestion or one possibility of an input-output system that allows the detection of certain COI. For the time being, it is as a combined retrospective and prospective approach, with the future aim to prospectively apply certain measures routinely.

Multidimensional Assessment of CPG-Related Neurofunctions: Preliminary Studies

In order to better understand the 3-month-transformation, in particular, and to define a set of POI for the Fingerprint Model, we are currently conducting a prospective longitudinal study with low-risk infants. Each infant is assessed seven times within their first 4 months of life, with the first assessment taking place at 28 ± 2 days post-term age and subsequent evaluations every 2 weeks, with the last one at 112 ± 2 days. Inclusion criteria of the study are as follows: uneventful pregnancy, uneventful delivery at term age, singleton birth, appropriate birth weight and uneventful neonatal period.

Each assessment session consists of the following three modules: (i) multi-device recordings of endogenously generated neurofunctions; (ii) eye-tracking assessment of visual attention and (iii) video recording of non-nutritive sucking patterns. Here, we highlight the assessment of age-specific spontaneous behaviour in the motor and speech-language domains.

In the assessment module—previously introduced as (i)—the infant is placed in supine position in a standard cot and recorded for 5 min, using the following equipment: (a) camera system: two standard HD camcorders and two Microsoft Kinects; (b) audio recording system: one stereo audio recording device with an additional external studio microphone; (c) motion sensors: six MTw motion sensors attached to the upper arms, upper legs and feet and (d) pressure sensor: one pressure-sensitive mat consisting of a 32 × 32 array of pressure sensels. Figure 2 illustrates the proposed setup. For synchronisation, a clapperboard is placed on the pressure-sensitive mat, with one motion sensor attached to the clapperboard’s clapstick. This generates an easily detectable signal on all employed devices. We perform the synchronisation signal directly before and after the assessment; these two synchronisation signals allow to define a common offset and to compensate a linear clock drift during the measurement.

For these multidimensional recordings, we apply automatic assessments using machine learning. The proposed approach is based on the automated detection of variability of CPG-related functions at signal level. The collected datasets serve as a starting point for our exploratory work on machine learning-based approaches for assessment of spontaneous neurofunctions. Preliminary data obtained (single case differentiation outlined in Figs. 3 and 4) demonstrate the feasibility of our approach. Two specific options, assessment of GMs and assessment of cooing vocalisations, are outlined in the following subsections.

Fig. 3

Top image frames extracted from video recordings of a male infant, together with the large-displacement optical flow [94, 95]. Left infant at 69 days post-term, showing not yet fidgety movements. Right corrected age of 84 days post-term age, performing typical fidgety movements.

Bottom acceleration measured at the right upper arm of the infant within a time window of 5 s around the frames showing the optical flow. The acceleration vector is represented in spherical coordinates, i.e. radius (r), azimuth (phi) and polar angle (theta). For better readability, the means of r, phi and theta have been removed

Fig. 4

Waveforms, spectrograms and visualised Mel-frequency cepstral coefficients (MFCCs) 1–12 for (top left) a vowel-like, low-resonant vocalisation (no cooing), and (top right) a typical cooing vocalisation of a female infant at 56 days post-term. The 3D scatter plot (bottom) shows the distribution of three exemplarily selected acoustic parameters (mean zero-crossings rate [ZCR], mean logarithmic harmonics-to-noise ratio [HNR], mean local jitter) over 20 frames of 0.01 s extracted from the voiced periods (marked with rectangular boxes in upper plots) of either vocalisation (no cooing: 0.35–0.55 s, grey dots; cooing: 0.29–0.49 s, black dots)

Automatic GM Assessment

Researching automatic assessment of GMs has received considerable attention in recent years, including computer vision- and motion sensor-based approaches [53,54,55,56,57, 58•].

Within computer vision-based approaches, the use of Kinect sensors is a particularly popular avenue, as they provide in-depth data at a reasonable price. Besides RGB and depth streams (RGB-D), the Kinect devices can also deliver a kinematic stick model of persons, potentially useful for the automatic assessment of infants’ spontaneous motor behaviour. For instance, the motion features that distinguish children at risk for cerebral palsy, as computed by Meinecke and colleagues [104], could be directly applied without the need to use an expensive and complex motion-tracking system. Moreover, the temporal kinematic information could serve as input to advanced machine learning methods, e.g. recurrent neural networks and dynamic Bayesian networks, in order to assess the infants’ motion patterns. However, the kinematic body tracker cannot be applied to individuals shorter than 1 m, since the implemented algorithm was trained on synthetic depth images of avatars ranging from preschool children to adults [105]. Therefore, there is a need to develop kinematic detection and tracking algorithms applicable to infants. Olsen et al. [106] proposed to fit an infant stick model by searching the extreme points in the point cloud corresponding to the depth data.

We aim to follow two alternative directions. The first is to use a similar approach as the original Kinect algorithm [105], i.e. to perform pixel-wise classification of the depth image. For this task, we will need to produce a dataset of synthetic depth images of animated infant body models. Whilst Shotton et al. [105] had a large database of simulated data available, which was rendered with a variety of 3D avatars, our challenge will be to animate the infant models sufficiently in order to cover the whole range of movement. The original Kinect data will serve as test data, which will be manually labelled. The alternative approach will be to use this manually labelled data as training data. For automatic movement assessment, we lack strict real-time requirements that allow us to use more powerful models than the random forest classifier employed by Shotton and colleagues [105], such as convolutional neural networks [107].

In addition to using depth measurements, the state-of-the-art tool for motion analysis in RGB data is based on the optical flow [94, 95], a vector field which measures the movement of image points between two consecutive video frames. The optical flow is often used as input for more advanced motion analysis techniques, e.g. motion-based image segmentation [108], and has also been applied to movement analysis in infants [54].

The alternative to vision-based approaches is the use of motion sensors, comprising accelerometers, gyroscopes and electromagnetic motion trackers. These systems have been applied to GMA in previous studies [53,54,55, 57, 58•]. Similar to vision-based approaches, motion sensors can also be used to derive a kinematic motion model. A promising avenue might be to combine a vision-based approach with motions sensors, with the possibility of incorporating a pressure device, as suggested previously.

For example, Fig. 3 illustrates the large-displacement optical flows [94, 95], extracted from videos of an infant at 69 days post-term showing no fidgety movements, and later at 84 days post-term performing typical fidgety movements. The current optical flow is illustrated using black arrows. The optical flow extracted 100 ms before this frame is illustrated as grey arrows; this depicts recent changes in 2D velocity, i.e. acceleration. The optical flow in the non-fidgety movement sequence changes rather smoothly over the whole image, whereas in the fidgety movement sequence the optical flow changes its orientation rapidly within different areas of the image. This coincides with the description of fidgety movements as unpredictable and spontaneous movements in all directions [37, 109].

At the bottom of Fig. 3, the acceleration measured at the right upper arm of the infant within a time window of 5 s around the frames in the top of Fig. 3 is illustrated. In both figures, the acceleration vector is represented in spherical coordinates, i.e. radius (r), azimuth (phi) and polar angle (theta). The radius r corresponds to the magnitude of the acceleration, whereas phi and theta correspond to the direction of the acceleration. Compared to the non-fidgety movement sequence, the fidgety movement sequence shows (i) higher short-term variance in r, (ii) larger number of small-to-medium-sized pulses in r and (iii) higher short-term variance in theta and phi, causing many spontaneous changes in the movement direction. Also these characteristics reliably match with Gestalt-based fidgety movements descriptions [37, 109].

Furthermore, instead of following an indirect approach to movement assessment, i.e. estimating a kinematic model first, we can also follow an end-to-end machine learning approach. In end-to-end learning, the recorded data, either from vision systems, motion sensors, or both, is used directly in a machine learning method finding a suitable representation of the data to train the final predictor (classifier or regression model). One current widely used paradigm in machine learning is labelled ‘deep learning’, which is based on (deep) neural networks, stacked restricted Boltzman machines (deep belief networks), stacked auto-encoders, convolutional networks and recurrent neural networks, to name but a few. State-of-the-art feature-based models include support vector machines, decision trees, random forests and probabilistic graphical models [110,111,112,113,114,115]. When applied to GMA, this will require the manual labelling of the recorded data, for instance the presence, absence or quality of fidgety movements, or the body parts where the movement occurs. Identifying the best combination of sensors to be used and particular learning algorithms appropriate for GMA will be a challenging task in future research.

Intelligent Early Vocalisation Analysis

In the context of engineering sciences, ‘intelligent’ analysis of data implies a combination of signal processing and machine learning techniques. Applying intelligent audio analysis methodology for early human vocalisation assessment allows the objective identification of early acoustic vocalisation characteristics and potential atypicalities a human listener is incapable of detecting.

Our audio data are segmented manually for infant vocalisations using the multimedia coding system Noldus Observer XT (www.noldus.com). In addition, we are currently developing a system for the automatic detection of infant vocalisations [116]. The segmentation process relies on the definition of infant vocalisations within distinct vocal breathing groups [117]. Pre-linguistic vocalisation types are transcribed according to an annotation scheme adapted from the Stark Assessment of Early Vocal Development-Revised (SAEVD-R; [18]). According to current methods in audio/speech processing [97, 118], a set of acoustic parameters is extracted from each segmented vocalisation by means of the open-source toolkit openSMILE [119, 120]; (www.audeering.com).

Following the ComParE-set of the 2013 to 2017 INTERSPEECH Computational Paralinguistics Challenges, we generate 6373 parameters representing statistical functionals (e.g. arithmetic mean, standard deviation, higher order moments, quartiles) computed for the trajectories of a broad range of acoustic time-, spectral- and/or energy-based short-term low-level descriptors (e.g. spectral band energy, Mel-frequency cepstral coefficients [MFCCs], zero-crossings rate [ZCR], harmonics-to-noise ratio [HNR], jitter) and their derivatives [121, 122]. The acoustic information thereby deduced from the audio recordings constitutes the input to an objective model of early vocal development for successive classification/pattern recognition approaches.

The multidimensional acoustic characterisation of different pre-linguistic vocalisation types (e.g. cooing) facilitates the automatic identification of potential speech-language delays or atypicalities in early vocal development (see Fig. 4).


Amongst scientists there is a degree of optimism that recent and upcoming (ante portas) technical advancements may shortly reveal new insights into early human development and may provide some clarification with regard to atypical development. Amongst clinicians and parents of children with disabilities however, there is both hope and also scepticism about the foundation of this optimistic view.

The goal of the methodology outlined is to enable the early detection of COI and the opportunity for early intervention. A paradigm shift from a ‘wait-and-see-approach’ to an approach that instead focusses on early identification with the aim to intervene on target (skill) deficits as they emerge is warranted.

We believe that only a comprehensive interdisciplinary approach, combining genetic risk factors with both classical and neurofunctional biomarkers, will enable us to detect and delineate specific parameters for identifying specific neurodevelopmental disorders at a very early age. Cross-syndrome comparisons, including neurotypical subjects, may reveal that some signs are present early in development but have not been detected with current methods. Future will tell whether certain feature constellations pinpoint certain disorders or prove that some disorders naturally unfold beyond this early time window. Besides its limitations and potential shortcomings (e.g. the lack of homogeneous datasets), the proposed ‘iDN Fingerprint Model’ may accurately differentiate abnormal development and specific conditions, as well as reliably predict developmental outcomes.


  1. 1.

    ‘Recognised’ refers to a phenotypical onset, clinical manifestation or an accurate (mean age of) diagnosis.



attention deficit hyperactivity disorder


autism spectrum disorder


acoustic vocalisation analysis


Brain, Ears & Eyes—Pattern Recognition Initiative


condition/conditions of interest


central pattern generator


fragile X syndrome


general movement assessment


general movements


Graz University Audiovisual Research Database for the Interdisciplinary Analysis of Neurodevelopment


harmonics-to-noise ratio


interdisciplinary Developmental Neuroscience


Mel-frequency cepstral coefficient


wireless motion tracker/trackers


network-attached storage


parameter/parameters of interest


Rett syndrome


red-green-blue colour model


retrospective video analysis


Stark Assessment of Early Vocal Development-Revised


Zika virus


zero-crossings rate


Papers of particular interest, published recently, have been highlighted as: • Of importance

  1. 1.

    Prechtl HFR. General movement assessment as a method of developmental neurology: new paradigms and their consequences. The 1999 Ronnie MacKeith lecture. Dev Med Child Neurol. 2001;43(12):836–42.

    CAS  PubMed  Article  Google Scholar 

  2. 2.

    Karmiloff-Smith A, Casey BJ, Massand E, Tomalski P, Thomas MS. Environmental and genetic influences on neurocognitive development: the importance of multiple methodologies and time-dependent intervention. Clin Psychol Sci. 2014;2(5):628–37.

    PubMed  PubMed Central  Article  Google Scholar 

  3. 3.

    Oppenheim RW. Ontogenetic adaptations in neural development; towards a more ecological developmental psychobiology. In: Prechtl HFR, editor. Continuity of neural functions from prenatal to postnatal life. Clinics in Developmental Medicine, 94. Oxford: Blackwell Scientific Publications; 1984:16–30.

  4. 4.

    Prechtl HFR (editor). Continuity of neural functions from prenatal to postnatal life. Clinics in Developmental Medicine, 94. Oxford: Blackwell Scientific Publications; 1984.

  5. 5.

    Prechtl HFR. New perspectives in early human development. Eur J Obstet Gynecol Reprod Biol. 1986;21(5–6):347–55.

    CAS  PubMed  Article  Google Scholar 

  6. 6.

    Stewart DB. The pelvis as a passageway. I. Evolution and adaptations. Br J Obstet Gynaecol. 1984;91(7):611–7.

    CAS  PubMed  Article  Google Scholar 

  7. 7.

    Dunsworth HM, Warrener AG, Deacon T, Ellison PT, Pontzer H. Metabolic hypothesis for human altriciality. Proc Natl Acad Sci U S A. 2012;109(38):15212–6.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  8. 8.

    Einspieler C, Prechtl HFR, Bos AF, Ferrari F, Cioni G. Prechtl’s method on the qualitative assessment of general movements in preterm, term and young infants. London: Mac Keith Press; 2004.

    Google Scholar 

  9. 9.

    Einspieler C, Prayer D, Prechtl HFR. Fetal behaviour: a neurodevelopmental approach. Clinics in Developmental Medicine, 189. London: Mac Keith Press; 2012.

    Google Scholar 

  10. 10.

    Einspieler C, Marschik PB, Prechtl HFR. Human motor behavior—prenatal origin and early postnatal development. Z Psychol. 2008;216(3):147–53.

    Article  Google Scholar 

  11. 11.

    Cioni G, Ferrari F, Prechtl HFR. Posture and spontaneous motility in fullterm infants. Early Hum Dev. 1989;18(4):247–62.

    CAS  PubMed  Article  Google Scholar 

  12. 12.

    Cioni G, Prechtl HF. Preterm and early postterm motor behaviour in low-risk premature infants. Early Hum Dev. 1990;23(3):159–91.

    CAS  PubMed  Article  Google Scholar 

  13. 13.

    Iwayama K, Eishima M. Neonatal sucking behaviour and its development until 14 months. Early Hum Dev. 1997;47(1):1–9.

    CAS  PubMed  Article  Google Scholar 

  14. 14.

    Braddick O, Atkinson J. Development of human visual function. Vis Res. 2011;51(13):1588–609.

    PubMed  Article  Google Scholar 

  15. 15.

    Atkinson J. Human visual development over the first 6 months of life. A review and a hypothesis. Hum Neurobiol. 1984;3(2):61–74.

    CAS  PubMed  Google Scholar 

  16. 16.

    van Wulfften-Palthe T, Hopkins B. Development of the infant’s social competence during early face to face interaction. A longitudinal study. In: Prechtl HFR, editor. Continuity of neural functions from prenatal to postnatal life. Clinics in developmental Medicine, 94. Oxford: Blackwell Scientific Publications. 1984:198–219.

    Google Scholar 

  17. 17.

    Oller DK. The emergence of the sounds of speech in infancy. In: Yeni-Komshian GH, Kavanagh JF, Ferguson CA, editors. Child Phonology. Vol 1. New York: Academic Press; 1980:93–112.

  18. 18.

    Nathani S, Ertmer DJ, Stark RE. Assessing vocal development in infants and toddlers. Clin Linguist Phon. 2006;20(5):351–69.

    PubMed  PubMed Central  Article  Google Scholar 

  19. 19.

    Dehaene-Lambertz G, Spelke ES. The infancy of the human brain. Neuron. 2015;88(1):93–109.

    CAS  PubMed  Article  Google Scholar 

  20. 20.

    Blüml S, Wisnowski JL, Nelson Jr MD, Paquette L, Gilles FH, Kinney HC, Panigrahy A. Metabolic maturation of the human brain from birth through adolescence: insights from in vivo magnetic resonance spectroscopy. Cereb Cortex. 2013;23(12):2944–55.

    PubMed  Article  Google Scholar 

  21. 21.

    Gilmore JH, Shi F, Woolson SL, Knickmeyer RC, Short SJ, Lin W, Zhu H, Hamer RM, Styner M, Shen D. Longitudinal development of cortical and subcortical gray matter from birth to 2 years. Cereb Cortex. 2012;22(11):2478–85.

    PubMed  Article  Google Scholar 

  22. 22.

    Makropoulos A, Gousias IS, Ledig C, Aljabar P, Serag A, Hajnal JV, Edwards AD, Counsell SJ, Rueckert D. Automatic whole brain MRI segmentation of the developing neonatal brain. IEEE Trans Med Imaging. 2014;33(9):1818–31.

    PubMed  Article  Google Scholar 

  23. 23.

    Dean 3rd DC, O'Muircheartaigh J, Dirks H, Waskiewicz N, Walker L, Doernberg E, Piryatinsky I, Deoni SC. Characterizing longitudinal white matter development during early childhood. Brain Struct Funct. 2015;220(4):1921–33.

    PubMed  Article  Google Scholar 

  24. 24.

    Sadeghi N, Prastawa M, Fletcher PT, Wolff J, Gilmore JH, Gerig G. Regional characterization of longitudinal DT-MRI to study white matter maturation of the early developing brain. NeuroImage. 2013;68:236–47.

    PubMed  Article  Google Scholar 

  25. 25.

    Chugani HT. Biological basis of emotions: brain systems and brain development. Pediatrics. 1998;102(5 Suppl E):1225–9.

    CAS  PubMed  Google Scholar 

  26. 26.

    Ball G, Aljabar P, Zebari S, Tusor N, Arichi T, Merchant N, Robinson EC, Ogundipe E, Rueckert D, Edwards AD, Counsell SJ. Rich-club organization of the newborn human brain. Proc Natl Acad Sci U S A. 2014;111(20):7456–61.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  27. 27.

    Kinney HC, Brody BA, Kloman AS, Gilles FH. Sequence of central nervous system myelination in human infancy. II. Patterns of myelination in autopsied infants. J Neuropathol Exp Neurol. 1988;47(3):217–34.

    CAS  PubMed  Article  Google Scholar 

  28. 28.

    Flechsig PE. Anatomie des menschlichen Gehirns und Rückenmarks auf myelogenetischer Grundlage. Leipzig: G. Thieme; 1920.

    Google Scholar 

  29. 29.

    de Graaf-Peters VB, Hadders-Algra M. Ontogeny of the human central nervous system: what is happening when? Early Hum Dev. 2006;82(4):257–66.

    PubMed  Article  Google Scholar 

  30. 30.

    Huttenlocher PR, Dabholkar AS. Regional differences in synaptogenesis in human cerebral cortex. J Comp Neurol. 1997;387(2):167–78.

    CAS  PubMed  Article  Google Scholar 

  31. 31.

    Ball G, Pazderova L, Chew A, Tusor N, Merchant N, Arichi T, Allsop JM, Cowan FM, Edwards AD, Counsell SJ. Thalamocortical connectivity predicts cognition in children born preterm. Cereb Cortex. 2015;25(11):4310–8.

    PubMed  PubMed Central  Article  Google Scholar 

  32. 32.

    Deniz Can D, Richards T, Kuhl PK. Early gray-matter and white-matter concentration in infancy predict later language skills: a whole brain voxel-based morphometry study. Brain Lang. 2013;124(1):34–44.

    PubMed  Article  Google Scholar 

  33. 33.

    Deoni SC, Dean 3rd DC, Walker L, Dirks H, O'Muircheartaigh J. Nutritional influences on early white matter development: response to Anderson and Burggren. NeuroImage. 2014;100:703–5.

    PubMed  Article  Google Scholar 

  34. 34.

    Koyama MS, Di Martino A, Castellanos FX, Ho EJ, Marcelle E, Leventhal B, Milham MP. Imaging the “at-risk” brain: future directions. J Int Neuropsychol Soc. 2016;22(2):164–79.

    PubMed  Article  Google Scholar 

  35. 35.

    Courchesne E, Pierce K, Schumann CM, Redcay E, Buckwalter JA, Kennedy DP, Morgan J. Mapping early brain development in autism. Neuron. 2007;56(2):399–413.

    CAS  PubMed  Article  Google Scholar 

  36. 36.

    Courchesne E, Campbell K, Solso S. Brain growth across the life span in autism: age-specific changes in anatomical pathology. Brain Res. 2011;1380:138–45.

    CAS  PubMed  Article  Google Scholar 

  37. 37.

    Prechtl HFR, Einspieler C, Cioni G, Bos AF, Ferrari F, Sontheimer D. An early marker for neurological deficits after perinatal brain lesions. Lancet. 1997;349(9062):1361–3.

    CAS  PubMed  Article  Google Scholar 

  38. 38.

    • Einspieler C, Bos AF, Libertus ME, Marschik PB. The general movement assessment helps us to identify preterm infants at risk for cognitive dysfunction. Front Psychol. 2016:7–406. A review article highlighting in 428 individuals the association between early spontaneous movements, the so-called general movements, and cognitive outcome at 7 to 10 years of age. Normal GMs along with a normal motor repertoire during the first months after term are markers for normal cognitive development until at least age 10

  39. 39.

    Einspieler C, Marschik PB. Central pattern generators and their significance for the foetal motor function. Klin Neurophysiol. 2012;43(1):16–21.

    Article  Google Scholar 

  40. 40.

    Sherrington CS. The integrative action of the nervous system. New Haven: Yale University Press; 1906.

    Google Scholar 

  41. 41.

    Einspieler C, Prechtl HFR. Prechtl’s assessment of general movements: a diagnostic tool for the functional assessment of the young nervous system. Ment Retard Dev Disabil Res Rev. 2005;11(1):61–7.

    PubMed  Article  Google Scholar 

  42. 42.

    Bosanquet M, Copeland L, Ware R, Boyd R. A systematic review of tests to predict cerebral palsy in young children. Dev Med Child Neurol. 2013;55(5):418–26.

    PubMed  Article  Google Scholar 

  43. 43.

    Palchik AB, Einspieler C, Evstafeyeva IV, Talisa VB, Marschik PB. Intra-uterine exposure to maternal opiate abuse and HIV: the impact on the developing nervous system. Early Hum Dev. 2013;89(4):229–35.

    CAS  PubMed  Article  Google Scholar 

  44. 44.

    Brasil P, Pereira JP, Moreira ME, Ribeiro Nogueira RM, Damasceno L, Wakimoto M, Rabello RS, Valderramos SG, Halai UA, Salles TS, Zin AA, Horovitz D, Daltro P, Boechat M, Raja Gabaglia C, de Carvalho Sequeira P, Pilotto JH, Medialdea-Carrera R, da Cotrim Cunha D, de Abreu Carvalho LM, Pone M, Machado Siqueira A, Calvet GA, Rodrigues Baião AE, Neves ES, de Nassar Carvalho PR, Hasue RH, Marschik PB, Einspieler C, Janzen C, Cherry JD, de Bispo Filippis AM, Nielsen-Saines K. Zika virus infection in pregnant women in Rio de Janeiro. N Engl J Med. 2016;375(24):2321–34.

    PubMed  Article  Google Scholar 

  45. 45.

    Bruggink JL, van Spronsen FJ, Wijnberg-Williams BJ, Bos AF. Pilot use of the early motor repertoire in infants with inborn errors of metabolism: outcomes in early and middle childhood. Early Hum Dev. 2009;85(7):461–5.

    CAS  PubMed  Article  Google Scholar 

  46. 46.

    Marschik PB, Soloveichick M, Windpassinger C, Einspieler C. General movements in genetic disorders: a first look into Cornelia de Lange syndrome. Dev Neurorehabil. 2015;18(4):280–2.

    PubMed  Article  Google Scholar 

  47. 47.

    Einspieler C, Hirota H, Yuge M, Dejima S, Marschik PB. Early behavioural manifestation of Smith-Magenis syndrome (del 17p11.2) in a 4-month-old boy. Dev Neurorehabil. 2012;15(4):313–6.

    PubMed  Article  Google Scholar 

  48. 48.

    Einspieler C, Kerr AM, Prechtl HF. Abnormal general movements in girls with Rett disorder: the first four months of life. Brain and Development. 2005;27(Suppl 1):S8–S13.

    PubMed  Article  Google Scholar 

  49. 49.

    Mazzone L, Mugno D, Mazzone D. The general movements in children with Down syndrome. Early Hum Dev. 2004;79(2):119–30.

    PubMed  Article  Google Scholar 

  50. 50.

    • Einspieler C, Sigafoos J, Bartl-Pokorny KD, Landa R, Marschik PB, Bölte S. Highlighting the first 5 months of life: general movements in infants later diagnosed with autism spectrum disorder or Rett syndrome. Res Autism Spectr Disord. 2014;8(3):286–91. Reviews the association between central pattern generated motor behaviour in the first months of life and autism spectrum disorder or Rett syndrome

    Article  Google Scholar 

  51. 51.

    Zappella M, Einspieler C, Bartl-Pokorny KD, Krieber M, Coleman M, Bölte S, Marschik PB. What do home videos tell us about early motor and socio-communicative behaviours in children with autistic features during the second year of life—an exploratory study. Early Hum Dev. 2015;91(10):569–75.

    PubMed  Article  Google Scholar 

  52. 52.

    Spittle AJ, Olsen J, Kwong A, Doyle LW, Marschik PB, Einspieler C, Cheong JLY. The baby moves prospective cohort study—using a smart-phone application with the general movements assessment to predict neurodevelopmental outcomes at age 2 years for extremely preterm or extremely low birth weight infants. BMJ Open. 2016;6(10):e013446.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  53. 53.

    Adde L, Helbostad JL, Jensenius AR, Taraldsen G, Grunewaldt KH, Stoen R. Early prediction of cerebral palsy by computer-based video analysis of general movements: a feasibility study. Dev Med Child Neurol. 2010;52(8):773–8.

    PubMed  Article  Google Scholar 

  54. 54.

    Stahl A, Schellewald C, Stavdahl O, Aamo OM, Adde L, Kirkerod H. An optical flow-based method to predict infantile cerebral palsy. IEEE Trans Neural Syst Rehabil Eng. 2012;20(4):605–14.

    PubMed  Article  Google Scholar 

  55. 55.

    Karch D, Wochner K, Kim K, Philippi H, Hadders-Algra M, Pietz J, Dickhaus H. Quantitative score for the evaluation of kinematic recordings in neuropediatric diagnostics. Detection of complex patterns in spontaneous limb movements. Methods Inf Med. 2010;49(5):526–30.

    CAS  PubMed  Article  Google Scholar 

  56. 56.

    Einspieler C, Marschik PB. Complementary thinking: future perspectives on the assessment of general movements. Dev Med Child Neurol. 2013;55(8):682–3.

    PubMed  Google Scholar 

  57. 57.

    Marcroft C, Khan A, Embleton ND, Trenell M, Plotz T. Movement recognition technology as a method of assessing spontaneous general movements in high risk infants. Front Neurol. 2014;5:284.

    PubMed  Google Scholar 

  58. 58.

    • Orlandi S, Guzzetta A, Bandini A, Belmonti V, Barbagallo SD, Tealdi G, Mazzotti S, Scattoni ML, Manfredi C. AVIM-A contactless system for infant data acquisition and analysis: software architecture and first results. Biomed Signal Process Control. 2015;20:85–99. Orlandi and colleagues report on a novel ‘contactless’ approach to neurologically assess young infants

    Article  Google Scholar 

  59. 59.

    Stark RE. Stages of speech development in the first year of life. In: Yeni-Komshian GH, Kavanagh JF, Ferguson CA, editors. Child Phonology. Vol. 1. New York: Academic Press; 1980:73–92.

  60. 60.

    Barlow SM, Radder JPL, Radder ME, Radder AK. Central pattern generators for orofacial movements and speech. In: Brudzynski SM, editor. Handbook of behavioral neuroscience. London: Academic Press. 2010:351–69.

    Google Scholar 

  61. 61.

    Barlow SM, Estep M. Central pattern generation and the motor infrastructure for suck, respiration, and speech. J Commun Disord. 2006;39(5):366–80.

    PubMed  Article  Google Scholar 

  62. 62.

    Oller DK. The emergence of the speech capacity. Mahwah: Lawrence Erlbaum Associates; 2000.

    Google Scholar 

  63. 63.

    Papoušek M. Vom ersten Schrei zum ersten Wort: Anfänge der Sprachentwicklung in der vorsprachlichen Kommunikation. Bern: Hans Huber; 1994.

    Google Scholar 

  64. 64.

    Locke JL. The child’s path to spoken language. Cambridge: Harvard University Press; 1995.

    Google Scholar 

  65. 65.

    Patten E, Belardi K, Baranek GT, Watson LR, Labban JD, Oller DK. Vocal patterns in infants with autism spectrum disorder: canonical babbling status and vocalization frequency. J Autism Dev Disord. 2014;44(10):2413–28.

    PubMed  PubMed Central  Article  Google Scholar 

  66. 66.

    Esposito G, Venuti P. Developmental changes in the fundamental frequency (f0) of infants’ cries: a study of children with autism spectrum disorder. Early Child Dev Care. 2010;180(8):1093–102.

    Article  Google Scholar 

  67. 67.

    Esposito G, del Carmen RM, Venuti P, Haltigan JD, Messinger DS. Brief report: atypical expression of distress during the separation phase of the strange situation procedure in infant siblings at high risk for ASD. J Autism Dev Disord. 2014;44(4):975–80.

    PubMed  PubMed Central  Article  Google Scholar 

  68. 68.

    Sheinkopf SJ, Iverson JM, Rinaldi ML, Lester BM. Atypical cry acoustics in 6-month-old infants at risk for autism spectrum disorder. Autism Res. 2012;5(5):331–9.

    PubMed  PubMed Central  Article  Google Scholar 

  69. 69.

    Marschik PB, Einspieler C, Sigafoos J. Contributing to the early detection of Rett syndrome: the potential role of auditory Gestalt perception. Res Dev Disabil. 2012;33(2):461–6.

    PubMed  PubMed Central  Article  Google Scholar 

  70. 70.

    Marschik PB, Pini G, Bartl-Pokorny KD, Duckworth M, Gugatschka M, Vollmann R, Zappella M, Einspieler C. Early speech-language development in females with Rett syndrome: focusing on the preserved speech variant. Dev Med Child Neurol. 2012;54(5):451–6.

    PubMed  Article  Google Scholar 

  71. 71.

    • Marschik PB, Kaufmann WE, Bölte S, Sigafoos J, Einspieler C. En route to disentangle the impact and neurobiological substrates of early vocalizations: learning from Rett syndrome. Behav Brain Sci. 2014;37(6):562–3. This article emphasizes the analysis of early vocalizations as neurofunctional biomarkers pinpointing neurodevelopmental disorders

    PubMed  Article  Google Scholar 

  72. 72.

    Pokorny FB, Marschik PB, Einspieler C, Schuller BW. Does she speak RTT? Towards an earlier identification of Rett syndrome through intelligent pre-linguistic vocalisation analysis. In: Morgan N, editor. Proceedings Interspeech 2016. San Francisco; 2016:1953–7.

  73. 73.

    Tarquinio DC, Hou W, Neul JL, Lane JB, Barnes KV, O'Leary HM, Bruck NM, Kaufmann WE, Motil KJ, Glaze DG, Skinner SA, Annese F, Baggett L, Barrish JO, Geerts SP, Percy AK. Age of diagnosis in Rett syndrome: patterns of recognition among diagnosticians and risk factors for late diagnosis. Pediatr Neurol. 2015;52(6):585–91.e2.

    PubMed  PubMed Central  Article  Google Scholar 

  74. 74.

    Marschik PB, Einspieler C, Prechtl HF, Oberle A, Laccone F. Relabelling the preserved speech variant of Rett syndrome? Dev Med Child Neurol. 2010;52(2):218.

    PubMed  Article  Google Scholar 

  75. 75.

    Marschik PB, Kaufmann WE, Sigafoos J, Wolin T, Zhang D, Bartl-Pokorny KD, Pini G, Zappella M, Tager-Flusberg H, Einspieler C, Johnston MV. Changing the perspective on early development of Rett syndrome. Res Dev Disabil. 2013;34(4):1236–9.

    PubMed  PubMed Central  Article  Google Scholar 

  76. 76.

    Bölte S, Bartl-Pokorny KD, Jonsson U, Berggren S, Zhang D, Kostrzewa E, Falck-Ytter T, Einspieler C, Pokorny FB, Jones EJ, Roeyers H, Charman T, Marschik PB. How can clinicians detect and treat autism early? Methodological trends of technology use in research. Acta Paediatr. 2016;105(2):137–44.

    PubMed  Article  Google Scholar 

  77. 77.

    Brett D, Warnell F, McConachie H, Parr JR. Factors affecting age at ASD diagnosis in UK: no evidence that diagnosis age has decreased between 2004 and 2014. J Autism Dev Disord. 2016;46(6):1974–84.

    PubMed  PubMed Central  Article  Google Scholar 

  78. 78.

    Shattuck PT, Durkin M, Maenner M, Newschaffer C, Mandell DS, Wiggins L, Lee LC, Rice C, Giarelli E, Kirby R, Baio J, Pinto-Martin J, Cuniff C. Timing of identification among children with an autism spectrum disorder: findings from a population-based surveillance study. J Am Acad Child Adolesc Psychiatry. 2009;48(5):474–83.

    PubMed  PubMed Central  Article  Google Scholar 

  79. 79.

    Zhang D, Kaufmann WE, Sigafoos J, Bartl-Pokorny KD, Krieber M, Marschik PB, Einspieler C. Parents’ initial concerns about the development of their children later diagnosed with fragile X syndrome. J Intellect Develop Disabil. 2016; doi:10.3109/13668250.2016.1228858.

    Google Scholar 

  80. 80.

    Goldberg WA, Thorsen KL, Osann K, Spence MA. Use of home videotapes to confirm parental reports of regression in autism. J Autism Dev Disord. 2008;38(6):1136–46.

    PubMed  Article  Google Scholar 

  81. 81.

    Hagerman RJ. The physical and behavioral phenotype. In: Hagerman RJ, Hagerman PJ, editors. Fragile X syndrome: diagnosis, treatment, and research. Baltimore: Johns Hopkins University Press. 2002:3–109.

    Google Scholar 

  82. 82.

    Cohen D, Pichard N, Tordjman S, Baumann C, Burglen L, Excoffier E, Lazar G, Mazet P, Pinquier C, Verloes A, Heron D. Specific genetic disorders and autism: clinical contribution towards their identification. J Autism Dev Disord. 2005;35(1):103–16.

    PubMed  Article  Google Scholar 

  83. 83.

    Bailey Jr DB, Raspa M, Bishop E, Holiday D. No change in the age of diagnosis for fragile X syndrome: findings from a national parent survey. Pediatrics. 2009;124(2):527–33.

    PubMed  Article  Google Scholar 

  84. 84.

    • Zwaigenbaum L, Bryson S, Garon N. Early identification of autism spectrum disorders. Behav Brain Res. 2013;251:133–46. In this article, different methods and approaches contributing to the early identification of autism spectrum disorders are reviewed and discussed

    PubMed  Article  Google Scholar 

  85. 85.

    Ozonoff S, Iosif AM, Young GS, Hepburn S, Thompson M, Colombi C, Cook IC, Werner E, Goldring S, Baguio F, Rogers SJ. Onset patterns in autism: correspondence between home video and parent report. J Am Acad Child Adolesc Psychiatry. 2011;50(8):796–806.e1.

    PubMed  PubMed Central  Article  Google Scholar 

  86. 86.

    Marschik PB, Einspieler C. Methodological note: video analysis of the early development of Rett syndrome—one method for many disciplines. Dev Neurorehabil. 2011;14(6):355–7.

    PubMed  Article  Google Scholar 

  87. 87.

    Bölte S, Marschik PB, Falck-Ytter T, Charman T, Roeyers H, Elsabbagh M. Infants at risk for autism: a European perspective on current status, challenges and opportunities. Eur Child Adolesc Psychiatry. 2013;22(6):341–8.

    PubMed  PubMed Central  Article  Google Scholar 

  88. 88.

    • Johnson MH, Gliga T, Jones E, Charman T. Annual research review: infant development, autism, and ADHD—early pathways to emerging disorders. J Child Psychol Psychiatry. 2015;56(3):228–47. This annual research review highlights important aspects and cross-syndrome comparisons of early developmental traits and disease pathways in autism and attention deficit hyperactivity disorder

    PubMed  Article  Google Scholar 

  89. 89.

    • Jones EJ, Gliga T, Bedford R, Charman T, Johnson MH. Developmental pathways to autism: a review of prospective studies of infants at risk. Neurosci Biobehav Rev. 2014;39:1–33. Jones and colleagues describe the prodrome of autism spectrum disorders and emphasize the importance of high-risk sibling studies

    PubMed  PubMed Central  Article  Google Scholar 

  90. 90.

    Miller M, Iosif AM, Young GS, Hill MM, Ozonoff S. Early detection of ADHD: insights from infant siblings of children with autism. J Clin Child Adolesc Psychol. 2016; doi:10.1080/15374416.2016.1220314.

    PubMed Central  Google Scholar 

  91. 91.

    Costanzo V, Chericoni N, Amendola FA, Casula L, Muratori F, Scattoni ML, Apicella F. Early detection of autism spectrum disorders: from retrospective home video studies to prospective ‘high risk’ sibling studies. Neurosci Biobehav Rev. 2015;55:627–35.

    PubMed  Article  Google Scholar 

  92. 92.

    Ozonoff S, Iosif A, Baguio F, Cook IC, Hill MM, Hutman T, Rogers SJ, Rozga A, Sangha S, Sigman M, Steinfeld MB, Young GS. A prospective study of the emergence of early behavioral signs of autism. J Am Acad Child Adolesc Psychiatry. 2010;49(3):258–68.

    Google Scholar 

  93. 93.

    Prechtl HFR. Hazards of oversimplification. Dev Med Child Neurol. 1970;12(4):522–4.

    CAS  PubMed  Article  Google Scholar 

  94. 94.

    Brox T, Bruhn A, Papenberg N, Weickert J. High accuracy optical flow estimation based on a theory for warping. In: Pajdla T, Matas J, editors. Computer Vision - ECCV 2004: 8th European Conference on Computer Vision, Prague, Czech Republic, May 11–14, 2004. Proceedings, Part IV. Berlin, Heidelberg: Springer; 2004:25–36.

  95. 95.

    Brox T, Malik J. Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans Pattern Anal Mach Intell. 2011;33(3):500–13.

    PubMed  Article  Google Scholar 

  96. 96.

    Bruhn A, Weickert J. Towards ultimate motion estimation: combining highest accuracy with real-time performance. Proceedings of the tenth IEEE International Conference on Computer Vision (ICCV'05). 2005;12005:749–55.

  97. 97.

    Schuller BW, Batliner AM. Computational paralinguistics: emotion, affect and personality in speech and language processing. New York: John Wiley & Sons; 2013.

    Book  Google Scholar 

  98. 98.

    Bishop CM. Pattern recognition and machine learning. Secaucus: Springer; 2006.

    Google Scholar 

  99. 99.

    Mareschal D, Sirois S, Westermann G, Johnson M (editors). Neuroconstructivism: perspectives and prospects. Vol. 2. Oxford: Oxford University Press; 2007.

  100. 100.

    Wolff JJ, Botteron KN, Dager SR, Elison JT, Estes AM, Gu HB, Hazlett HC, Pandey J, Paterson SJ, Schultz RT, Zwaigenbaum L, Piven J, The IBIS Network. Longitudinal patterns of repetitive behavior in toddlers with autism. J Child Psychol Psychiatry. 2014;55(8):945–53.

    PubMed  PubMed Central  Article  Google Scholar 

  101. 101.

    • Szatmari P, Chawarska K, Dawson G, Georgiades S, Landa R, Lord C, Messinger DS, Thurm A, Halladay A. Prospective longitudinal studies of infant siblings of children with autism: lessons learned and future directions. J Am Acad Child Adolesc Psychiatry. 2016;55(3):179–87. Highlights the first decade of high-risk infant sibling studies in autism and reviews the achieved insights and future perspectives for studying autism

    PubMed  PubMed Central  Article  Google Scholar 

  102. 102.

    Ruggeri B, Sarkans U, Schumann G, Persico AM. Biomarkers in autism spectrum disorder: the old and the new. Psychopharmacology. 2014;231(6):1201–16.

    CAS  PubMed  Article  Google Scholar 

  103. 103.

    Mandy W, Lai MC. Annual research review: the role of the environment in the developmental psychopathology of autism spectrum condition. J Child Psychol Psychiatry. 2016;57(3):271–92.

    PubMed  Article  Google Scholar 

  104. 104.

    Meinecke L, Breitbach-Faller N, Bartz C, Damen R, Rau G, Disselhorst-Klug C. Movement analysis in the early detection of newborns at risk for developing spasticity due to infantile cerebral palsy. Hum Mov Sci. 2006;25(2):125–44.

    CAS  PubMed  Article  Google Scholar 

  105. 105.

    Shotton J, Girshick R, Fitzgibbon A, Sharp T, Cook M, Finocchio M, Moore R, Kohli P, Criminisi A, Kipman A. Efficient human pose estimation from single depth images. IEEE Trans Pattern Anal Mach Intell. 2013;35(12):2821–40.

    PubMed  Article  Google Scholar 

  106. 106.

    Olsen MD, Herskind A, Nielsen JB, Paulsen RR. Model-based motion tracking of infants. In: Agapito L, Bronstein MM, Rother C, editors. Computer Vision - ECCV 2014 Workshops: Zurich, Switzerland, September 6–7 and 12, 2014, Proceedings, Part III. Cham: Springer; 2015:673–85.

  107. 107.

    Le Cun Y, Boser B, Denker JS, Howard RE, Habbard W, Jackel LD, Henderson D. Handwritten digit recognition with a back-propagation network. In: Touretzky DS, editor. Advances in neural information processing systems 2. San Francisco: Morgan Kaufmann Publishers. 1990:598–605.

    Google Scholar 

  108. 108.

    Fragkiadaki K, Salas M, Arbeláez P, Malik J. Grouping-based low-rank trajectory completion and 3D reconstruction. Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS’14). 2014:55–63, Montréal.

  109. 109.

    Einspieler C, Peharz R, Marschik PB. Fidgety movements—tiny in appearance, but huge in impact. J Pediatr. 2016;92(3 Suppl 1):S64–70.

    Article  Google Scholar 

  110. 110.

    Breiman L. Random forests. Mach Learn. 2001;45(1):5–32.

    Article  Google Scholar 

  111. 111.

    Hinton GE, Osindero S, Teh YW. A fast learning algorithm for deep belief nets. Neural Comput. 2006;18(7):1527–54.

    PubMed  Article  Google Scholar 

  112. 112.

    Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol P-A. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res. 2010;11:3371–408.

    Google Scholar 

  113. 113.

    Pascanu R, Mikolov T, Bengio Y. On the difficulty of training recurrent neural networks. Proceedings of the 30th International Conference on Machine Learning (ICML’13). 2013;28(3):1310–8.

  114. 114.

    Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20(3):273–97.

    Google Scholar 

  115. 115.

    Ghahramani Z. Learning dynamic Bayesian networks. In: Giles CL, Gori M, editors. Adaptive processing of sequences and data structures: International Summer School on Neural Networks; “E.R. Caianiello” Vietri sul Mare, Salerno, Italy; September 6–13, 1997; Tutorial Lectures. Berlin, Heidelberg: Springer; 1998:168–97.

  116. 116.

    Pokorny FB, Peharz R, Roth W, Zohrer M, Pernkopf F, Schuller BW. Manually versus automated: the challenging routine of infant vocalisation segmentation in home videos to study neuro(mal)development. In: Morgan N, editor. Proceedings Interspeech 2016. San Francisco; 2016:2997–3001.

  117. 117.

    Lynch MP, Oller DK, Steffens ML, Buder EH. Phrasing in prelinguistic vocalizations. Dev Psychobiol. 1995;28(1):3–25.

    CAS  PubMed  Article  Google Scholar 

  118. 118.

    Schuller BW. Intelligent audio analysis. Berlin: Springer; 2013.

    Book  Google Scholar 

  119. 119.

    Eyben F, Wöllmer M, Schuller BW. openSMILE: the Munich versatile and fast open-source audio feature extractor. Proceedings of the 18th ACM International Conference on Multimedia. Firenze; 2010:1459–62.

  120. 120.

    Eyben F, Weninger F, Groß F, Schuller B. Recent developments in openSMILE, the Munich open-source multimedia feature extractor. Proceedings of the 21st ACM International Conference on Multimedia. Barcelona; 2013:835–8.

  121. 121.

    Schuller BW, Steidl S, Batliner AM, Vinciarelli A, Scherer K, Ringeval F, Chetouani M, Weninger F, Eyben F, Marchi E. The Interspeech 2013 computational paralinguistics challenge: social signals, conflict, emotion, autism. Proceedings Interspeech 2013. Lyon; 2013:148–52.

  122. 122.

    Schuller BW, Steidl S, Batliner AM, Epps J, Eyben F, Ringeval F, Marchi E, Zhang Y. The Interspeech 2014 computational paralinguistics challenge: cognitive & physical load. Proceedings Interspeech 2014. Singapore; 2014:427–31.

  123. 123.

    Einspieler C, Marschik PB, Bos AF, Ferrari F, Cioni G, Heinz FR. Prechtl, 1927-2014 crossing the borders. Dev Psychobiol. 2014;56(7):1609–11.

    PubMed  Article  Google Scholar 

Download references


Open access funding provided by Medical University of Graz. We would like to dedicate this paper to Heinz FR Prechtl [123], a mentor, friend and critical mind, as a continuation of many insights and ideas he fascinated us with. One may consider this as bringing some ideas of his 1999 Ronnie MacKeith Lecture [1] to a new level applying novel methods to ideas and thoughts he envisaged some 20 years ago but also looking beyond them. Many thanks to Gunter Vogrinec, Miha Tavcar, Laura Langmann, Claudia Zitta and Mathias Egger for their continuous help and assistance, #iDN4ever (bones). This work is supported by the Bill and Melinda Gates Foundation, Seattle, WA (OPP112887), the FWF—Austrian Science Fund (P25241; and TCS 24), the OeNB—Austrian National Bank (P16430), the General Movements Trust, BioTechMed-Graz, the Franz-Lanyar Foundation (P374) and the Country of Styria.

The editors would like to thank Dr. John Brust for taking the time to review this manuscript.

Author information




Corresponding authors

Correspondence to Peter B . Marschik or Christa Einspieler.

Ethics declarations

Conflict of Interest

Florian B. Pokorny, Robert Peharz, Dajie Zhang, Jonathan O’Muircheartaigh, Herbert Roeyers, Sven Bölte, Alicia J. Spittle, Berndt Urlesberger, Björn Schuller, Luise Poustka, Sally Ozonoff, Franz Pernkopf, Thomas Pock, Kristiina Tammimies, Christian Enzinger, Magdalena Krieber, Iris Tomantschger, Katrin D. Bartl-Pokorny, Jeff Sigafoos, Laura Roche, Gianluca Esposito, Markus Gugatschka, Karin Nielsen-Saines, Christa Einspieler, Walter E. Kaufmann and the BEE-PRI study group declare that they have no conflict of interest.

Peter B. Marschik has received grants from FWF—Austrian Science Fund, the Bill and Melinda Gates Foundation, OeNB, the Franz-Lanyar Foundation, the Country of Styria and BioTechMed-Graz.

Human and Animal Rights and Informed Consent

All reported experiments with human subjects performed by the authors have been previously published and complied with all ethical standards. Informed consent was obtained from all individual participants included in the study. Additional informed consent was obtained from all individual participants for whom identifying information is included in this article.

Additional information


- We stress the need to study the early period of postnatal development as an important period for identifying neurodevelopmental disorders usually detected beyond infancy or even toddlerhood.

- We propose a new way to assess the functional integrity of the developing nervous system.

- We suggest a ‘Fingerprint Model’ for the delineation of developmental progress or regression and for an automated detection of neurodevelopmental disorders in infancy.

- With the proposed model, we aim to contribute to the paradigm shift from a ‘wait-and-see-approach’ to a ‘find-early-and-intervene-early-approach’.

Topical Collection on Pediatric Neurology

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Marschik, P.B..., Pokorny, F.B., Peharz, R. et al. A Novel Way to Measure and Predict Development: A Heuristic Approach to Facilitate the Early Detection of Neurodevelopmental Disorders. Curr Neurol Neurosci Rep 17, 43 (2017). https://doi.org/10.1007/s11910-017-0748-8

Download citation


  • Computer vision
  • Diagnosis
  • Early human development
  • Intelligent vocalisation analysis
  • Multidimensional assessment
  • Neurodevelopmental disorders