Automated LED tracking to measure instantaneous velocities in swimming

In this paper a video-based method to automatically track instantaneous velocities of a swimmer is presented. Single cameras were used to follow a marker (LED) attached to the body. The method is inspired by particle tracking techniques, traditionally used in the field of fluid dynamics, to measure local velocities of a fluid flow. During the validation experiment, a white LED was attached to the hip of a swimmer together with a speedometer. A swimmer performed four different stroke types. The velocity profiles using LED tracking were captured and showed less noise than the speedometer measurements. Only at times when the marker disappeared above the water surface due to body role in front crawl and backstroke swimming did the LED tracking fail to capture the athlete’s motion. The algorithm was tested in a 2D case with a single LED to illustrate the proof of principle, but should be suitable for implementation in a 3D analysis or multiple LED analysis.


Introduction
The need for measuring the intra-cyclic velocity variations (IVVs) and velocity profiles in an experiment including many swimming trials [1] forms the framework of this study.
The most widely used method to access the instantaneous velocity of a swimmer is a speedometer, also known as a tachometer [2][3][4][5][6][7]. A thin, non-stretching cable is attached to the waist of a swimmer. The rotation speed within the instrument is measured when pulling the cable while swimming. Although the use of a speedometer is straightforward, the method just offers 1D information and limits the swimmer's motion. Other techniques have been developed as well , for example, [8] measured the instantaneous velocity by a three-dimensional reconstruction of the location of the centre of mass of the swimmer using multiple cameras and anthropometric data calculations, which is associated with a high computational effort. During the last decade, the use of inertial sensors (i.e., accelerometers) to measure kinematic variables in swimming has increased [9]. However, inertial sensors have noise complicating the calculation of velocity by integration. When applied in real time, the use of inertial sensors suffers from problems with data transfer through the water (surface) and integration to obtain velocity.
The use of video recordings is a common method in sport practice to provide analyses of the performance in both training and competition. The use of video recordings is attractive, since it is a versatile tool that serves as input for postprocessing for a variety of analyses. Since a visual record is obtained, validation and troubleshooting of methods are simplified. With increasing digital capacities and knowledge of image processing, it is possible to improve the videobased data collection. In swimming practice, video recordings are used in a variety of ways. They are used for race analysis (software: SwimWatch, SwimOptimum), qualitative technique analysis and quantitative 2D or 3D analysis of kinematic parameters (software: Qualisis, Simi [10][11][12][13][14][15][16][17][18]) of strokes, starts and turns [19]. The use of markers (on body landmarks) is a useful addition to video analysis to obtain quantitative information. However, detecting the marker is often performed manually or only partially automated, making the analysis costly and time consuming and less feasible for practical usage [20]. Some automated tracking approaches exist (also markerless) ( [20][21][22][23] and software for this purpose: Simi, Qualisis), but they are either time consuming or not easily applicable in an aquatic environment. Other areas in which (3D) motion analysis is frequently used and developed are the gaming and film industry. Demands for such an application intiated the development of a realtime 3D motion analysis system (PRIMAS) by Furne et al. [24]. Fast and accurate marker tracking algorithms are essential for the real-time application of this system.
The aim of this study was to determine the instantaneous position and velocity of the swimmer. A tracking algorithm is presented, and the application for tracking a single LED in a two-dimensional space is shown to illustrate the potential of the method. The velocity time series for the four competitive stroke types (butterfly, backstroke, breaststroke and front crawl) were compared with the results of velocity measurements with a speedometer.

Setup and participant
The experiment was conducted in the 50 m indoor training pool of the Pieter van den Hoogenband Swimming Stadium at Innosportlab in Eindhoven, the Netherlands, which has a camera system along the length of the pool. One swimmer, competent in all four competitive strokes, participated voluntarily and gave written informed consent. The ethical officer of the University of Technology Eindhoven has approved the design of this study. The swimmer wore a swimming trunk, with a small socket on the hip containing a waterproof white LED. It was assumed that the hip velocity provides a good representation of the forward velocity profile (measured at the center of gravity), although small differences between the two have been reported [28,29]. The motion of the swimmer (including the LED) in the sagittal plane was captured by four cameras (Basler, sc1400gc, 50 fps, resolution: 788 × 524 pixels), located in the side wall of the pool at a depth of 0.55 m below the water surface, on positions 2.5, 5, 10 and 15 m from the beginning of the lane (see Fig. 1). The swimmer was instructed to follow the line on the bottom of the pool at 3.75 m from the side wall. At that level, the total recording range was approximately 17 m and the images of the different cameras partly overlap. Two-dimensional tracking was used, since lateral motion was assumed to be negligible. The calibration for translating pixels to meters, coupling and post-processing the recordings of the different cameras was provided by Innosportlab de Tongelreep (Eindhoven, the Netherlands). The selection of a sagittal plane of interest was required to apply the calibration, here chosen at 3.55 m, approximately coinciding with the distance to the LED. The raw video recordings were collected in a binary sequence (.seq) file. This bulk file was read in Matlab and single frames were extracted during the tracking procedure. No preprocessing of the single frames was required for the analysis. Positions of the LED and the instantaneous horizontal (forward) and vertical velocity of the LED were obtained using an automated LED-tracking algorithm, which was programmed in Matlab (R2015b). The algorithm is presented in Sect. 2.2.
The swimmer was instructed to swim 25 m for each stroke type twice (butterfly, backstroke, breaststroke, and front crawl) during which the motion was captured with the cameras. To enable comparison of the velocity profile obtained by the LED tracking, the velocity of the swimmer was also measured by a speedometer (Swimsportec, length: 25 m, velocity range: 0-3 m/s captured linearly on a 5 V range) attached to the waist of the swimmer with a cord. The speedometer data was captured with a sample rate of 32.5 Hz using the Swim Analysis Software 3.0.4. developed by Swimsportec.

LED tracking algorithm
To track the LED throughout the video recordings, an algorithm consisting of different cases was used. In advance of the tracking, a template of the typical light spot emitted by the white LED was created by selecting a small area in a typical image including the LED. A single standard template T ( 13 × 17 pixels) was used to perform the tracking on all recordings.
The tracking was started by manually selecting the position of the LED in the first frame that captured the LED (case 1 in Fig. 2). In the second frame an area of 100 × 100 pixels around the position of the first point was used to search for the new position of the LED (case 2 in Fig. 2). The area was restricted, because too many pixels is computationally expensive. For too small a window the LED might move out of the search area. Three normalized cross-correlations between the area of interest and the template of the LED were performed on the separate RGB (red-green-blue) colour components denoted by i = R, G, B . The normalized cross-correlation was defined as [26] : with T the template of the LED with size (K × L) , T the mean of the template, I the area of interest on the image, Ī x,y the mean of I i in the region under T, i n w h i c h k = x − 1 2 K, x − 1 2 K + 1, ..., x + 1 2 K a n d l = y − 1 2 L, y − 1 2 L + 1, ..., y + 1 2 L , and c i (x, y) the correlation coefficient on the position (x, y) relative to the predicted position [30]. The three resulting cross-correlation coefficients were multiplied, giving c(x, y) = c R × c G × c B . The peak in the correlation corresponds to the displacement of the LED. To further increase accuracy, a Gaussian peak detection was applied to find the peak in the correlation. To this end, a Gaussian curve, with a the height of the peak, (x 0 , y 0 ) the position of the maximum and x,y the characteristic width of the Gaussian curve, was fitted through the results using a non-linear least-square algorithm available in Matlab (lsqcurvefit). This improved the algorithm to find the maximum with sub-pixel accuracy [26]. The peak corresponds to the displacement of the LED and thus to the new position. When two previous positions are known, a prediction of the next position of the LED can be made, n+1 = 2 n − n−1 (case 3 in Fig. 2), with n+1 the predicted position and n and n−1 the two previous positions. The same cross-correlation technique and peak detection was applied, but now an area of interest around n+1 with the size of the template image was used for the calculation.
The main part of the tracking relies on the following procedure (case 4 in Fig. 2). The last part of the track was smoothed up to nine positions using a moving average filter. Then, an approximate location of the LED in the next frame was estimated based on this smoothed track. The prediction was made with: with n the coordinates of the last point in the track, = n−1 − n−2 and = n − n−1 the vectors between the last three data points in the track and a directional vector which contains information about the curvature of the track. A more regular approach of a prediction based on acceleration directly gave errors due to the large fluctuations in swimming. A weighted average of and gave better results. Similar to case 3 an area of interest with the size of the template image was selected around the predicted position in the new frame. The new position was found again by applying the cross-correlation, the peak detection fit and subsequent detection.
Finally, the noise in the track was corrected by smoothing the velocities (in pixels/s) using a moving average filter (standard Matlab function nanfastsmooth) on three subsequent velocity data points: The new positions were calculated by integrating the velocity.

Constraints
As long as three previous positions are known, the next position was determined using case 4. However, in reality this procedure was not straightforward. Additional difficulties might arise such as the LED disappearing due to body roll, the view being blocked by passing body parts, or reflections of the LED appearing in the water surface compromising identification of the correct LED position in the image. Therefore, some constraints must be added to overcome these difficulties in automated tracking adding to the robustness of the algorithm. In this 2D experiment, with an LED on the hip, no difficulties were expected with breaststroke swimming. In all other strokes, the hand could block the LED, which could lead to an incomplete velocity profile. In general, this was of such a short duration ( ∼ 1 frame) that no hindrance was observed due to the speed of the hand. In backstroke and front crawl swimming, the LED disappeared for a longer period of time ( ∼ 15 frames) when the LED reached the water surface during the body roll, which caused some interruption in the acquired velocity signal.
To quantify these mishaps in detecting, the LED location the peak to correlation energy (PCE) of the cross-correlation was used: with c(x, y) the result of the normalized cross-correlation, c max the peak value and the denominator the correlation energy [31]. When the PCE was below a threshold (set to 0.08), c max was low compared to the noise, it was assumed that the LED was not properly detected and the code returned to case 2. As a new starting point (prediction), the last predicted position summed with the average horizontal displacement until that point was used. In the vertical, a fixed minor displacement in the negative vertical direction was added, to favour the tracking of the LED instead of its reflection in the water surface. As long as the PCE was below the threshold value, the tracking was continued with case 2. Note that the threshold in case 2 (0.025) must be chosen differently from case 3 and 4, since the window size was larger and the characteristics of the enlarged image, and thus c(x, y), were completely different.
Another issue arose when multiple peaks appeared in c(x, y), and the maximum of c(x, y) satisfied the PCE threshold, but did not necessarily correspond to the peak of interest. The presence of a second peak was captured when R max ∕c max > 0.8 , with R the residual of the Gaussian peak detection. The location of this second peak was determined with a Gaussian peak fit. When the location of the second peak was closer to the predicted position, this location was set to the new position in the track.

Data analysis
The output of the tracking algorithm was a ( 3 × n ) array containing the frame numbers and horizontal and vertical positions in pixel coordinates (sub-pixel accurate) for a single camera. These arrays were converted to positions in realworld coordinates (m) using the calibration. The velocities were calculated using a first-order backward finite difference method: with f the frame rate (50 fps).
To obtain a single data array for each trial, the individual arrays of each of the four cameras were coupled. The cameras are synchronized and had an overlapping view at the level of the swimmer, which was useful for coupling. The cut-off (and coupling) of the arrays was chosen at the frame number halfway in the overlap region. A possibly missing data point in this part of the signal was replaced by the data point of the other camera.
The velocity data of the speedometer contained a significant amount of noise as could be observed in Fig. 3 and was therefore filtered before further analysis. A traditional fourth-order Butterworth filter (cut-off frequency 5 Hz) was applied for filtering [32]. The measurements with the speedometer and cameras were synchronized, by shifting the signals such that the characteristic peaks for each of the four strokes coincided.
For comparison of the data, an averaged velocity profile of a stroke was created by manually selecting the area around characteristic peaks within the velocity profile to distinguish single stroke cycles. The location of the local minimum was then automatically obtained by fitting a second-order polynomial through the selected areas and determining the minimum by differentiating; then the stroke cycles' duration was resampled on 100 sample points and averaged.

Results
The settings (template image LED, thresholds, window size) of the LED tracking algorithm were kept constant throughout the analysis of different recordings. During the analysis, the LED was tracked automatically, only the starting position was selected manually. Note that this can be automated, but this was not implemented in this study. After periods in which the LED was lost, the LED was captured again automatically. In Table 1 the typical computation times are shown. The calculations were performed within the Matlab interface (tic toc function) on a single processor. The duration of analysing the swimmer's passage through all frames of a single camera was on the order of 30 s.
In Fig. 3 a typical velocity profile of a breaststroke trial (a) and front crawl trial (b) is shown for both the LED tracking and the speedometer data. For the tracking results of the breaststroke the acceleration profile in the x direction is shown (derivative of the velocity profile). Looking at the peaks of maximum velocity and maximum acceleration, there is a time shift of ∼ 0.15 s. When the LED is lost from the camera view the algorithm turns back to case 2 and the computational time is slightly increased. In these trials the push-off start, the underwater phase and ∼ 7 stroke cycles for the breaststroke and front crawl were included. During the LED tracking a horizontal ( v x ) and vertical ( v y ) velocity component were obtained, whereas the speedometer measures one component, which was approximately in the forward direction ( ∼ v x ). As can be observed, the raw speedometer signal contains more noise. Although some minor parts of the velocity profile are missing in the LED tracking of the front crawl due to body roll, a clear stroke cycle pattern is visible, partly due to the availability of the additional v y component, where the stroke pattern is very distinguishable. Difficulties were experienced with the analysis of the velocity profile obtained with the speedometer: the signal degrades, i.e., becomes more noisy, when the distance to the speedometer increases. It is of added value to have the corresponding LED tracking data, which could be incorporated in the analysis of the speedometer data.
In Fig. 4 the averaged time-normalized velocity profiles for each stroke obtained with the LED tracking are presented. For the averaging procedure a number of complete   (Fig. 4a), backstroke (Fig. 4b), breaststroke (Fig. 4c), and front crawl (Fig. 4d), respectively. In the velocity profiles of the backstroke and the front crawl, a gap in the velocity data can be observed due to the body roll whereby the LED disappears above the water surface. This results in larger inaccuracies around that gap. In butterfly and breaststroke swimming the single stroke cycles are selected from v x . Typically, during front crawl and backstroke swimming, the forward velocity variations are considerably reduced, and render the selection of stroke cycles based on the horizontal velocity v x alone more difficult. Moreover, fluctuations like noise might appear more pronounced in v x . Therefore, characteristic points in the vertical velocity v y are used for the identification of single strokes.
Looking at the velocity profiles given in Fig. 4 and the video recordings attached as supplementary material, some actions of the swimmer are identified, which are indicated by the numbered, dashed vertical lines. However, the actual onset of an action is first reflected in the peaks of the acceleration profile, which have a time shift with the peaks in the velocity profile. The observed effects in the velocity profile might be misplaced over time ( ∼ 0.15 s based on comparison with the acceleration profile (Fig. 3a)). The v x profile of the butterfly stroke (Fig. 4a) shows three characteristic peaks, which correspond to the first kick, sweeping in of the hands, and the second kick in combination with the sweeping up of the arms, respectively [33]. The action of legs and arms in breaststroke swimming is represented by the first and second peak in v x (Fig. 4c), respectively. The steap deceleration after the second peak corresponds to the high drag experienced in the arm and leg recovery phase [6,33]. With some effort and considering that we are looking at cyclic motion, two broad peaks corresponding to the asymmetric action of the arms can be distinguished in the velocity profiles v x of the backstroke and front crawl (Fig. 4b, d) [33]. However, the size of these intra-cyclic velocity variations in v x is almost similar to the variations related to inter-cyclic velocity fluctuations. In v y the variations are larger and stroke actions are more distinguishable. The minima before the acceleration around 0-0.2 (normalized times) in v y for both backstroke and front crawl, coincides with the start of the left arm stroke and right arm stroke, respectively. In Fig. 5 the average velocity profiles of the speedometer measurements are shown. In the back, the average velocity profile of the LED tracking measurements is added in grey. The averaging is performed on the same strokes as in Fig. 4. In Table 2, an overview of means and standard deviations of the two different measurements methods is given. In general, the averaged velocity of the LED tracking was slightly higher and the maximum and averaged deviations were smaller. Two striking differences between the results of the LED tracking and speedometer are the missing second peak in the velocity profile of the butterfly and the lower second peak within the breaststroke signal of the speedometer. The ratio between the intra-cyclic velocity variations and the intercyclic fluctuations (captured in the standard deviation) in v x of the backstroke and front crawl is too low to identify clear differences. Meanwhile, v y contains large variations during the stroke cycle, which, fortunately, are captured with the LED tracking.

Discussion
The advantages of LED tracking over a speedometer include a visual record that is inherently obtained, the distinction between different velocity components v x and  v y , the collection of position and acceleration data, and that swimmers are not disrupted by the measurement device itself. The only requirement is that the pool must be accommodated with a (calibrated) camera system, a fast computer and suitable LEDs. The approach in this study was inspired by particle tracking techniques (PTV) used in the field of experimental fluid dynamics, which is a Lagrangian method to measure the displacement of submerged small particles through subsequent frames to determine the particle velocities ( ( , t) = ∕ t , with the velocity vector at position , = n+1 − n the displacement and t the time between frames n and n + 1 ). PTV methods can yield results at a sub-pixel level ( ∼ 0.3 pixel for PTV). This gives a high potential for making measurements of around (1) accuracy [26,27]. Keeping in mind the successes of the PTV technique in the field of experimental fluid dynamics, the application of these ideas to (LED) marker tracking of human motion is interesting. It is possible to expand this technique towards multiple LED tracking for a full 3D analysis. This would simplify video-based 3D motion and coordination analysis [34] to understand the relationship between body actions and acceleration. In combination with augmented reality tools automatic LED tracking may become a powerful interactive method to optimize interventions in stroke training.
The LED tracking shows similarities with the results (especially for the butterfly and breaststroke profiles) found in theliterature [5,6,29,33]. However, compared to the literature and the speedometer data in this experiment, it is striking that the second peak in breaststroke is larger than the first peak. Analysis of the video footage shows that the LED analysis is correct, and the observed differences with the speedometer are due to inaccuracy of the speedometer. Also, the second peak in butterfly swimming is more pronounced than described in the literature and is completely missing from the speedometer data in this experiment. Presumably, the speedometer system is not sensitive to sudden accelerations when the cord has slack, causing not all variations in horizontal speed to be observed, or even the appearance of additional fluctuations due to unwanted extra degrees of freedom of the cord.
In previous studies that investigate the velocity profile of backstroke and front crawl, two broad peaks are indentified that correspond to the actions of the arms, with several smaller peaks on top, probably due to the action of the legs [29,33]. These peaks were not clearly identifiable in this study, but the recordings (see Online Resource 1-4) demonstrate that the LED tracking was not compromised. Of course, the technique of the swimmer might give a distorted view of the capabilities of selecting individual strokes in the backstroke and front crawl. For example, it is assumed that an experienced swimmer, is swimming more efficiently with less velocity variations [1,33,35]. Therefore, it would be useful to perform similar measurements for different swimmers.
It was observed that the mean velocity obtained with the LED tracking is slightly higher (up to ∼ 0.1 s) for all strokes compared to the speedometer. A minor deviation might be explained by the assumptions that (i) the participant is swimming at the center of the lane (3.75 m), and (ii) the LED is just moving in the sagittal plane.
From the butterfly and breaststroke results, it can be concluded that the standard deviation in LED tracking was mainly determined by the inter-cyclic differences of the athlete. The additional inaccuracies due to noise were larger for the speedometer. Especially in the speedometer measurement of the backstroke and front crawl trials there were difficulties in selecting proper periods. In all likelihood, small errors arise when choosing the start of a single stroke, which could result in a small shift of the time-normalized velocity profile.
In this LED tracking approach, the only signal loss occurred with the body roll. Therefore, this technique seems useful for the analysis of breaststroke and butterfly swimming. Perhaps such a loss of signal can be avoided by performing a 3D analysis with multiple cameras.
Concerning the computational times of the LED tracking (Table 1) at this stage, the algorithm is suitable for analysing velocity profiles in experiments quickly. To perform realtime analysis at the pool side for application in day-to-day training or in competition, the algorithm must be further optimized and a suitable interface containing the video capture, tracking computations and visualization is advisable for user-friendliness. Much time can be saved by converting the Matlab language to an imperative programming language (such as C/C++) and parallelizing the computation on multiple processors. A fast laptop PC must then be sufficient to perform the analysis. In fact, it has been successfully used in an experiment about breaststroke, in which the intra-cyclic velocity variations and velocity profiles were the focus of investigation [1].

Conclusion
In this study the potential of a technique for measuring the instantaneous velocity of a swimmer by automatically tracking a LED marker was described and compared with speedometer measurements. Although some velocity profiles were incomplete due to the body role within the backstroke and front crawl stroke cycles, the LED tracking technique seems to be much more convenient given that a visual record is obtained, which can be used to check the accuracy. In general the LED tracking shows little noise and individual strokes can be better distinguished, also by the additional information of the vertical velocity component. Since the settings were kept constant throughout the analysis of all recordings, it is shown that this technique is robust. Moreover, without much optimization this technique is close to real time, which makes the technique attractive for practical usage. Extension towards 3D LED tracking is straightforward.

Compliance with ethical standards
Conflict of interest The authors declare that there are no conflicts of interest.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creat iveco mmons .org/licen ses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.