Signal, Image and Video Processing

, Volume 11, Issue 7, pp 1197–1204 | Cite as

Automatic spin measurements for pitched Baseballs via consumer-grade high-speed cameras

  • Takashi Ijiri
  • Atsushi Nakamura
  • Akira Hirabayashi
  • Wataru Sakai
  • Takeshi Miyazaki
  • Ryutaro Himeno
Original Paper

Abstract

Controlling the spin of a ball is important in a variety of sports, especially baseball. For an athlete to properly train, it is necessary to know spin information immediately after throwing or hitting a baseball. This paper presents a fully automatic and marker-free technique to measure both the spin rate and spin axis of a pitched baseball using a consumer-grade high-speed camera. After tracking a ball from the high-speed video, our technique measures spin rate by detecting periods in which similar ball images appear, and then estimates spin axis by performing rigid registration that considers three-dimensional rotation. By separating the spin rate measurement and spin axis estimation processes, we achieve reasonable computational efficiency and robustness for small blurred baseball images extracted from video. We evaluated the accuracy of our presented technique by using synthesized videos. To illustrate the feasibility of our technique, we applied it to a variety of breaking ball pitches captured under normal outdoor lighting conditions.

Keywords

High-speed video Spin analysis Baseball 

1 Introduction

Techniques for measuring the spin of a ball have a variety of applications in sports training and broadcasting. The spin of a ball causes variations in trajectories [1], thus controlling spin is an important skill in most sports that use a ball. For example, in baseball, a pitcher throws a ball with different grips and snaps to effect specific spinning patterns that result in various types of pitches, including the breaking ball [2, 3]. To practice and perfect one’s spin control in various sports, knowing spin information immediately after throwing or hitting a ball is a fundamental need. Further, for sports broadcasting and scouting, it would be possible to characterize and classify players based on their skills in spin control.
Fig. 1

Overview of the proposed technique (see also our supplementary video)

Numerous studies have attempted to estimate the spin of a flying ball from a video; however, such studies usually assume specific ball textures (e.g., additional markers or a logo), specific lighting conditions, or manual interaction in the tracking process. Estimation of spin under natural conditions is difficult for such studies. This paper presents a robust and efficient technique for automatically estimating both the spin rate and spin axis of a pitched baseball from high-speed video. Our technique does not assume specific textures nor additional markers on a ball and works well for videos captured under outdoor ambient lighting conditions.

Although imaging devices with high frame rates had previously been too expensive and therefore difficult for non-professional users to acquire and use, several easy-to-use consumer-grade high-speed cameras are now available [4]. These newer cameras provide many potential applications, because they allow a large number of casual users to deal with high-speed videos; however, to date, only a few applications using casual high-speed cameras have been explored. We focus on sports and provide an innovate example for casual applications using consumer-grade high-speed cameras.

Our technique consists of three steps, i.e., ball tracking, spin rate measurement, and spin axis estimation, as illustrated in Fig. 1. We first detect and track a baseball in an input video (a) and extract a smaller video clip that centers on the tracked ball; we call this the ball clip (b). We then obtain the spin rate by detecting the period of time in which similar frames repetitively appear in the ball clip (c). Next, we estimate spin axis by acquiring the rotation axis that fits adjacent frames of the ball clip most coherently (d). Separating the rate measurement and axis estimation processes achieves an efficient and robust procedure even for small or blurred ball images. Our technique visualizes the estimated spin rate and spin axis as in (e).

To evaluate the accuracy of our technique, we applied it to computer-synthesized videos. To illustrate the feasibility of our technique, we analyzed many breaking balls captured in different outdoor scenes, confirming that our approach worked well for actual pitching examples. We also implemented a rapid feedback system with a consumer-grade high-speed camera, enabling a player to know the spin information approximately 20 s after the pitch.

2 Related work

Trajectory tracking In sports broadcasting and training, numerous studies have been published on tracking the trajectory of a ball in flight in a variety of sports, including baseball [5, 6], tennis [7], soccer [8], table tennis [9], and basketball [10] (see also survey [11] and references therein). In particular, the KZone system [5] and PITCHf/x system [6] measure trajectories of a pitched baseball by using multiple cameras built into a ballpark. Based on aerodynamics [1], it is possible to estimate drag coefficients and spin from the trajectories [9, 12]; however, we are more interested in a direct measurement of spin, because trajectories are impacted by other factors such as wind and smoothness of the ball surface. Furthermore, we prefer to measure spin with simple equipment to support sports training at many levels.

Spin estimation In the field of computer vision, tracking a rigid motion from a single video is an important topic. Various approaches based on edge, optical flow, or local features have already been presented [13, 14]; however, images of a flying ball captured from a sports video are usually too small and blurry to obtain reliable optical flow or local feature points. Boracchi et al. [15] estimated spin from a single blurred image by analyzing its blur direction; however, their method requires high-resolution images of the ball, which is usually difficult to obtain.

The spin of a ball can be estimated by observing it under specific conditions. For example, Szep tracked corners of a specific texture pattern painted on a ball [16]. Zhang et al. [17] traced the position of a logo on a ping-pong ball by using two cameras placed on both sides of a table. Other image registration-based methods also rely on additional markers located on a ball [18, 19, 20]. Theobalt et al. [21] tracked a ball by using strobe lights and cameras with long exposure times. In contrast, our goal is to measure spin from images captured in a natural sports scene without any of these additional assumptions or limitations.

Adopting a high-speed camera is a promising means of measuring spin, capturing video in sports scenes far less invasively. Nagami et al. [2] manually calibrated the spin of a ball at each frame; however, such full manual analysis is labor-intensive. Some researchers have presented semiautomatic estimation methods based on image registration [18, 19, 20]. These methods simultaneously search for the best rotation angle and axis. In contrast, we estimate the spin rate and spin axis separately, thus reducing the search space and achieving computational efficiency. While Tamaki et al. [18, 20] used only successive pairs of frames for spin estimation, we assume the spin rate and spin axis are constant during a short period of time and estimate spin using a set of sequential frames. Through this, we achieve robust measurements even for small and blurry ball images. In addition, previous methods using high-speed cameras [18, 19, 20] assume manual specifications of ball positions; in contrast, we propose a fully automatic algorithm.

3 Capturing environment

According to a recent report [22], the maximum spin rate of top player’s pitches reaches 50 revolutions per second (rps). To measure such fast spin motion of a small ball, we need a camera with enough temporal and spatial resolution. Thanks to recent advancements in imaging technologies, several consumer-grade high-speed cameras now exist. In this study, we used Sony RX10II [4] on a tripod (Fig. 2a). This camera captures videos at 480 fps or 960 fps, which is fast enough for analyzing human pitches. More specifically, we may obtain 9.6 and 19.2 frames for one revolution of a 50 rps spinning ball with a 480 and 960 fps video, respectively. Although this camera is able to provide color frames, we only used intensity values, because many high-speed cameras are still limited to only grayscale, and we would like to maintain as wide a level of applicability as possible.

We assume that the camera is statically mounted behind the pitcher and a ball is not occluded by the pitcher during flight. Before the estimation, four parameters—region of interest (ROI), \(R^{r}\), \(R^{c}\), and \(\mathbf{d}\)—are specified by the user. The ROI represents a rectangular region in which a flying ball may exist (Fig. 1a) and is used to trim the video. Parameters \(R^r\) and \(R^c\) are the ball radii in the video at release and catch (Fig. 2b, c), respectively; these values are used for tracking (see Sect. 4.1). The parameter d is the highlight direction on the video plane used to define a weighting mask for axis estimation (see Sect. 4.3). Note that the same set of parameters is applicable to multiple videos captured within the same environment; thus, parameter specification is usually required only once after the camera position is fixed.
Fig. 2

Capturing device and video frames. The pitching scenes are captured using Sony RX10II (a), obtaining videos with high frame rates during from release (b) to catch (c)

4 Spin measurement

4.1 Ball tracking for extracting a ball clip

Given an input video, our technique automatically detects and tracks a ball to extract a ball clip (Fig. 3). We binarize the video via background subtraction and apply an opening operation to reduce noise. In this paper, an average image of the video is used as a background. Next, we apply template matching to the binarized video frames, using a white circle image as a template, as shown in Fig. 3c. Since the ball travels away from the camera, its radius decreases in successive frames. We therefore set the circle radius of the template for the ith frame as \(R_i = R^r + \frac{(R^c-R^r ) \times i}{M-1}\), where M is the number of the video frames. This provides the best matching positions, i.e., ball centers, for all frames as \(\{\mathbf{c}_0, \mathbf{c}_1, \ldots , \mathbf{c}_{M-1}\}\).

Because the input video may contain frames before releasing or after catching the ball, it is necessary to extract a center sequence that purely captures the flying ball. To do this, we first find the frame that has the highest template matching score; we denote the index of such a frame as \(i^*\). Next, we iteratively check centers \(\mathbf{c}_i\) in ascending order from \(i^*\), adding each to the center sequence until we find discontinuous pair \(\mathbf{c}_i\) and \(\mathbf{c}_{i+1}\) such that \(||\mathbf{c}_i - \mathbf{c}_{i+1} || < threshold\). A similar process is performed in descending order from \(i^*\) to obtain final center sequence \(\{\mathbf{c}_k, \ldots , \mathbf{c}_{i^*}, \ldots , \mathbf{c}_{k+N-1} \}\).
Fig. 3

Ball tracking. The input video (a) is binarized via background subtraction (b). Template matching and a circular Hough transform are applied to the binarized frame to detect the ball (b, c)

The acquired center points are sensitive to noise and do not contain radius information. We adopt a circle Hough transform [23] to detect the center and radius of the ball region for each frame; for computational efficiency, we limit the search domain of the center and radius to around \(\mathbf{c}_i\) and \(R_i\) for the ith frame.

Ball clip construction Given sequential center points and radii, we cutout the corresponding ball regions and uniformly stretch them to obtain a small square-shaped video clip, i.e., a ball clip. We denote the ball clip as \({\bar{I}}(\mathbf{x}, t) \in [0,255]\), where \(\mathbf{x} \in R^2\) is pixel position and \(t \in [0, N-1]\) is frame position. To remove noise, we apply Gaussian convolution with standard deviation \(\sigma _b\). It is also necessary to remove the lighting effect from the ball clip, because the video is captured under outdoor ambient conditions. Similarly to [19], we subtract an average image of ball clip \(c(\mathbf{x})\) from original frames to obtain \(I( \mathbf{x},t) = {\bar{I}}( \mathbf{x}, t) - c(\mathbf{x})\).

4.2 Spin rate estimation

The spin rate and axis of a flying ball can be assumed as constant in a short time period. If the spin of a ball is constant, similar frames must appear within the ball clip in a fixed interval. This interval indicates a single spin period. To detect it, we calculate the difference between arbitrary pairs of frames in the ball clip as:
$$\begin{aligned} D( i, j) = \sum _\mathbf{x} M(\mathbf{x})( I(\mathbf{x}, i) - I( \mathbf{x}, j ) )^2, \end{aligned}$$
(1)
where \(I( \mathbf{x}, i)\) represents the ith frame and \(M(\mathbf{x})\) is a weighting function (see Sect 4.3). Figure 4b shows an example of D(ij); black and white colors indicate lower and higher differences, respectively. In our experiments, D(ij) provides stripe patterns in almost all cases. The interval of the stripe pattern indicates a spin period. Further, the intervals were usually constant, indicating the uniformity of spin rate in a short period.
Next, we sum D(ij) in a diagonal direction to convert to a one-dimensional signal as \(v(i) = \sum _{k=0}^{N/3}D(k, k+i)\) for \(i \in [0, \frac{2}{3}N]\) (Fig. 4c). We then calculate the average magnitude difference function (AMDF) of v(i) (Fig. 4d), which is commonly used for sound pitch detection [24, 25],
$$\begin{aligned} \textit{AMDF}(k) = \sum _{i=0}^{2N/3}|v(i)-v(i-k)|, \end{aligned}$$
(2)
where we extend v(i) as an even function to negative domain \(v(-i) = v(i)\). The minima of AMDF(k) provides spin period, \(T^*= \mathop {\mathrm{arg~min}}\nolimits _k{ \textit{AMDF}(k) }\).
The above-mentioned method is limited to providing an integer spin period. We apply a cubic interpolation to v(i) to obtain a period with sub-integer accuracy. In addition, AMDF-based period detection may cause a double pitch error discussed in [25]. To solve this problem, we recode the first five local minima as candidates, \(T_i \ (i = 0, \ldots , 4)\), and evaluate them in the subsequent step.
Fig. 4

Spin rate estimation. Given a ball clip (a), we compute D(ij); its (ij) entry represents the difference between the ith and jth frames (b). We diagonally sum D(ij) to obtain v(i) (c) and compute its AMDF. The minima of AMDF provides the spin period (d)

Fig. 5

Axis estimation. A frame of the ball clip is warped by assuming orthogonal projection and 3D rotation (a). To avoid highlight on the ball, weighting mask \(M(\mathbf{x})\) is specified (b)

4.3 Spin axis estimation

Given ball clip \(I( \mathbf{x}, t)\) and spin period candidates \(T_i\), we estimate the spin axis by performing texture registration:
$$\begin{aligned} \mathop {\mathrm{arg~min}}\limits _{\mathbf{a}, T_i} \frac{\sum _{t = 0}^{N_f-1} {\sum _\mathbf{x} M(\mathbf{x}) M(\mathbf{x'}) ( I( \mathbf{x}, t) - I(\mathbf{x'}, t + 1))^2}}{\sum _{t = 0}^{N_f-1} {\sum _\mathbf{x} M(\mathbf{x}) M(\mathbf{x'}) } },\nonumber \\ \end{aligned}$$
(3)
where \(N_f\) is the user-specified frame number at which spin axis is approximated to be constant and \(\mathbf{x'}\) is a pixel position warped from \(\mathbf{x}\). We warp pixel position \(\mathbf{x} = (x_u, x_v )\) to \(\mathbf{x'}\) by assuming an orthogonal projection and three-dimensional (3D) rotation:
$$\begin{aligned} \mathbf{x'} = \begin{pmatrix} 1 &{}0 &{}0 \\ 0 &{}1 &{}0 \end{pmatrix} \mathbf{R}( \mathbf{a}, 2\pi /T_i ) \begin{pmatrix} x_u \\ x_v \\ z(\mathbf{x}) \end{pmatrix}, \end{aligned}$$
(4)
where \(\mathbf{R}( \mathbf{a}, \theta ) \in R^{3 \times 3}\) is a rotation matrix along axis \(\mathbf{a} \in R^3\) with angle \(\theta \), \(z(\mathbf{x}) = \sqrt{ r^2 - x_u^2 - x_v^2}\) is a depth of \((x_u, x_v )\), and r is a radius of a ball clip (Fig. 5a).
Although shading is removed from the ball clip by subtracting the average image, highlight saturation may remain when the video is captured under bright sunlight. In addition, since the ball is spherical, an area close to the center provides a clearer texture and is more reliable than an area close to the edge. We use weighting function \(M(\mathbf{x})\) to incorporate these effects to our optimization (1) and (3) as
$$\begin{aligned} M(\mathbf{x}) = {\left\{ \begin{array}{ll} 0 &{} \text {if}\ (\mathbf{x}-\mathbf{c})\cdot \mathbf{d}> 0 \; \text {or} \; |\mathbf{x}-\mathbf{c}| > r \\ \text {e}^{- \frac{(\mathbf{x} - \mathbf{c})^2}{ 2\sigma _m^2}} &{} \text {otherwise} \end{array}\right. } \end{aligned}$$
(5)
where \(\mathbf{c}\) and r represent the center and radius of the ball clip, respectively, \(\sigma _m\) is the standard deviation of a Gaussian, and \(\mathbf{d}\) is a user-specified highlight direction in the ball clip. The half of \(M(\mathbf{x})\) corresponding to highlight direction \(\mathbf{d}\) is masked as zero (Fig. 5b). When highlight does not exist, we simply specify \(\mathbf{d} = \mathbf{0}\) to consider the entire area.
We perform an exhaustive search to solve optimization (3). We prepare a unit sphere model by subdividing an icosahedron and use its vertices as axis candidates \(\mathbf{a}\). Figure 6 shows our used unit sphere model that has 2562 vertices. Although exhaustive search is adopted, this optimization took about 0.2 s in our experiments, primarily because our spin rate estimation step substantially reduces the number of spin period candidates.
Fig. 6

Unit sphere for axis candidates (a). We visualize the registration cost of Eq. (3) at each orientation \(\mathbf{a}\) for an example ball clip (b)

5 Results and discussion

5.1 Accuracy evaluation

To evaluate the accuracy of our technique, we applied it to synthesized videos and compared estimation results with actual spin rate and spin axis values. We generated \(5\times 37\times 19=3515\) videos of spinning balls by varying the spin rate from 10 to 50 rps with an interval of 10 rps, the azimuth angle of the spin axis from \(0^\circ \) to \(360^\circ \) with an interval of \(10^\circ \), and the elevation of the spin axis from \(-90^\circ \) to \(90^\circ \) with an interval of \(10^\circ \). All videos emulated 480 fps, and the ball radius is about 25 pixel. Figure 7(top left) shows three representative frames of a synthesized video. The position of the ball was fixed at the center of the video and we omitted the tracking process, because our focus was on evaluating our spin estimation algorithm.

Figure 7 and Table 1 summarize estimation errors; we measured an absolute difference between estimated rate and ground truth as a rate error and an angle between the estimated axis and the ground truth as an axis error. Spin rates were almost perfectly estimated for all videos. Even for videos with a 50 rps spin rate, the average error for spin rate estimation was less than 0.5 rps.
Fig. 7

Accuracy evaluation. We synthesized multiple videos of a spinning ball with various spin rates and spin axes (top left). We estimated their spin axes and measured differences from the ground truths. The panes show two-dimensional error maps for videos with from 10 to 50 rps spin. Their horizontal and vertical axes represent azimuth and elevation angles of the spin axis for synthesizing each video

Table 1

Average and maximum errors of spin estimation

Fig. 7

10 (rps)

20 (rps)

30 (rps)

40 (rps)

50 (rps)

Avg. rate error (rps)

0.00

0.00

0.00

0.00

0.37

Max. rate error (rps)

0.00

0.00

0.00

0.00

1.52

Avg. axis error (\(^\circ \))

3.7

2.8

2.3

2.3

2.0

Max. axis error (\(^\circ \))

18.1

8.9

6.9

6.1

5.6

Fig. 8

Balls with four different textures for evaluation

Table 2

Average and maximum errors for balls in Fig. 8

 

10 (rps)

20 (rps)

30 (rps)

40 (rps)

50 (rps)

Fig. 8a

   Avg. rate error (rps)

0.00

0.00

0.00

0.00

0.28

   Max. rate error (rps)

0.00

0.00

0.00

0.00

1.52

   Avg. axis error (\(^\circ \))

3.5

2.7

2.4

2.2

2.0

   Max. axis error (\(^\circ \))

15.6

6.1

5.5

6.1

5.8

Fig. 8b

   Avg. rate error (rps)

0.00

0.00

0.00

0.00

0.23

   Max. rate error (rps)

0.02

0.00

0.00

0.00

0.52

   Avg. axis error (\(^\circ \))

5.1

4.4

4.1

4.2

3.8

   Max. axis error (\(^\circ \))

16.9

9.4

10.1

9.8

7.4

Fig. 8c

   Avg. rate error (rps)

0.00

0.00

0.00

0.00

2.15

   Max. rate error (rps)

0.00

0.00

0.00

0.00

9.26

   Avg. axis error (\(^\circ \))

3.3

2.6

2.1

2.1

1.9

   Max. axis error (\(^\circ \))

18.1

5.5

6.1

6.1

17.0

Fig. 8d

   Avg. rate error (rps)

0.00

0.00

0.00

0.00

0.46

   Max. rate error (rps)

0.00

0.00

0.00

0.00

2.00

   Avg. axis error (\(^\circ \))

3.8

2.6

2.2

2.1

1.9

   Max. axis error (\(^\circ \))

16.4

6.2

6.1

6.2

6.2

Regarding spin axis, our technique also achieved accurate estimates. The average errors of axis estimation were less than \(4^\circ \). We observed slightly higher errors for videos in which azimuth angles are close to \(0^\circ \) or \(180^\circ \) or in which elevation angles are close to \(-90^\circ \) or \(90^\circ \). This indicates that slightly higher errors may appear when the spin axis is nearly parallel to the video screen. Note that the average errors of axis estimation were less than \(4^\circ \) and even the maximum error was \(18.1^\circ \), which we conclude as accurate enough for sports training.

To examine the effect of texture, we performed evaluation similar to Fig. 7 by using a ball with different textures, such as a baseball with a scribble noise, basketball, football, and volleyball (Fig. 8). Table 2 summarizes estimation errors. Regarding spin rate, we found slightly higher error only for the football texture with 50 rps. This was caused by the fact that highly symmetric texture pattern of the football causes many local minima in AMDF. For the other all cases, spin rates were estimated almost perfectly. With respect to spin axis, although we found slightly higher errors for videos with 10 rps, the average estimation errors were not greater than \(5.1^\circ \) and were small enough for our purpose. Because our algorithm does not assume specific textures on the ball, it works well for various balls.

5.2 Measurement and classification of breaking balls

We applied our technique to outdoor pitching scenes. Two participants, undergraduate students with baseball experiences, threw fastballs, curveballs, sliders, and forkballs. We mounted a camera (RX10II) approximately 15 m behind the pitchers as in Fig. 9a, and captured pitching video clips. We specified video resolution as \(1920\times 1080\) and frame rate as 480 fps. The ball size in the video was about \(R^r=28\) pixels at release and \(R^c=14\) pixels at catch. We empirically selected the following parameters, \(N_f=32\), \(\sigma _b = 0.1r\), and \(\sigma _m = 0.5r\), where r is the radius of the ball clip.

Figure 9d, e shows representative measurement results as well as v(i) used for spin rate estimation. From this visualization, spin conditions for each breaking ball or differences between pitchers can be recognized at a glance. It is difficult to obtain actual spin speed and spin axis from the real word data; we thus visually evaluated the results. We captured and measured 134 videos in total (25 for the pitcher 1 and 109 for the pitcher 2). For 130 videos of them, correct estimations were confirmed.
Fig. 9

Spin estimation for outdoor pitching scenes. We mounted a camera as in (a) and captured pitching videos of the pitcher 1 (b) and the pitcher 2 (c). Panes in (d) and (e) visualize estimated spin rates and axes as well as v(i) of representative pitches

On the other hand, our technique failed for 4 videos. We found two common causes for measurement failure. The first was tracking error; the camera was sometimes unintentionally moved by wind, by which our background subtraction failed in extracting a ball accurately. The second was extremely low spin rate; one forkball of the pitcher 2 rotated less than \(360^\circ \) during the flight. For such a ball, our technique failed in obtaining AMDF with clear minima. Note that our approach estimated 97% videos accurately, which we think is high enough for sports training applications.

One promising application of our spin measurements technique is automatic breaking ball classification. Given set of spin data \((s_i, \theta _i, \phi _i )\), where \(s_i\) represents spin rate and \(\theta _i\) and \(\phi _i\) are azimuth and elevation angles of the spin axis, respectively, we plot them in 3D space as \(\mathbf{v}_i = (s_i \cos \theta _i \cos \phi _i, s_i \sin \phi _i, -s_i \sin \theta _i \cos \phi _i)\). We then apply k-means clustering to the set of spin data. For both pitchers, all types of pitches produce small clusters and they were well classified as shown in Fig. 10.

With our breaking balls visualization in Fig. 10, we observed that spin axes differed depending on pitchers even for the same type of breaking ball (e.g., the curveballs). It was also possible to confirm the distribution of axes of a specific breaking ball. Such detailed spin analysis has a potentially large audience in sports training and broadcasting.

Although we could obtain accurate results, we found that k-means clustering was unstable for the data of the pitcher 1. Because the data size was relatively small, its result depended on initial labeling. It is necessary to collect larger data for stable breaking ball classification.

5.3 Performance, rapid feedback, and other cameras

Performance. We performed our experiments on an Intel Core i7 3.50 GHz CPU with 32GB RAM. Table 3 summarizes the average times for measuring our 130 pitching videos. The average frame size of the 130 videos was about 552. After loading high-speed video into RAM, our technique finished the entire process in less than 1.5 s.
Fig. 10

Breaking ball classification. We measured 25 balls from pitcher 1 and 105 from pitcher 2, and classified them by the k-means clustering. Balls in different clusters are depicted with different colors. The red cone (z-axis) indicates the traveling direction of the ball

Table 3

Average times for ball tracking, spin rate estimation, and spin axis estimation

Tracking (s)

Spin rate estim. (s)

Spin axis estim. (s)

Total (s)

1.085

0.103

0.220

1.409

The table does not include time for video loading

Fig. 11

Rapid feedback system. We directly transfer video from the camera to the computer via an HDMI capture device, and then measure spin and visualize the spin information, all within approximately 18 s

Rapid feedback After recording a video, high-speed cameras usually store it in their internal storage (e.g., memory card), and it is therefore necessary to transfer the video to a computer for analysis. Both recording and transferring are common bottlenecks to adopting high-speed cameras for real-time application. Fortunately, newly available cameras support fast video transfer, e.g., RX10II replays high-speed video just after recording it and outputs it via an HDMI interface. In our experiments, our camera took approximately 16 s to replay one second of 480 fps video.

With such functionality, it is possible to implement a system that provides spin information rather quickly. Figure 11 shows the essentials of our system. By directly receiving high-speed video via an HDMI interface from the camera, our technique returns estimation results in approximately 18 s, i.e., 16 s for loading time and 2 s for the estimation algorithm. Note that our estimation algorithm actually takes less than 1.5 s and even more rapid feedback will be achieved if faster video transfer is realized.

Applicability of other cameras The experiments presented above were performed by using a specific camera, our technique works well with other high-speed cameras. We report our experiments with another camera in our supplementary document and video.

6 Conclusions

In this paper, we presented an automatic technique for measuring the spin of a pitched ball by using a single consumer-grade high-speed camera. We first track a ball in input video to extract a ball clip. Next, we detect the interval in which similar frames appear in the ball clip to obtain the spin rate. We perform rigid image registration that uses 3D rotation to determine the spin axis. Separating spin rate detection and spin axis estimation reduces the parameter space and achieves a more efficient estimation process. We performed a detailed evaluation by using synthesized spin videos, thus proving that our technique measures spin accurately under various spin conditions. Experiments with actual pitching scenes also indicated that our technique works well for standard baseballs (i.e., without additional markers) and videos captured in typical outdoor ambient lighting environments.

Thanks to advances in imaging technologies, several high-quality consumer-grade high-speed cameras are available, with more to come in the future. This trend has already enabled non-professional users to access high-speed videos, and thus applications for casual users are now required. We believe that the technique presented in this paper provides an innovative step toward building a casual application for consumer-grade high-speed cameras.

Limitations and future work One limitation of our technique is a ball with a low spin rate, such as a knuckleball. Since our technique detects self-similarity of the ball clip to estimate spin rate, it requires at least two spins in the ball clip for robust estimation. Another limitation is to detect the seam positions on the ball relative to the spin axis. For example, two types of fastballs, two seams and four seams, have similar spin axis but differ in their seam positions. Our system does not detect the seam position and not distinguish these two balls. Robust estimation for low spin rates and detection of seam positions remains as future work.

Notes

Acknowledgements

We appreciate anonymous reviewers for their valuable comments. We thank undergraduate students at Ritsumeikan University and at the University of Electro-Communications for participating the evaluations of our technique.

Supplementary material

Supplementary material 1 (mp4 58199 KB)

11760_2017_1075_MOESM2_ESM.pdf (743 kb)
Supplementary material 2 (pdf 742 KB)

References

  1. 1.
    Mehta, R.D.: Aerodynamics of sports balls. Ann. Rev. Fluid Mech. 17, 151–189 (1985)CrossRefGoogle Scholar
  2. 2.
    Nagami, T., Morohoshi, J., Higuchi, T., Nakata, H., Naito, S., Kanosue, K.: The spin on fastballs thrown by elite baseball pitchers. Med. Sci. Sport Exer. 43(12), 2321–2327 (2011)CrossRefGoogle Scholar
  3. 3.
    Nagami, T., Higuchi, T., Kanosue, K.: How baseball spin influences the performance of a pitcher. J. Sports Med. Phys. Fit. 2(1), 63–68 (2013)CrossRefGoogle Scholar
  4. 4.
    SONY.: Digital Imaging. [Online]. http://www.sony.net/Products/di/en-us/ (2016)
  5. 5.
    Gueziec, A.: Tracking pitches for broadcast television. IEEE Comput. 35(3), 38–43 (2002)CrossRefGoogle Scholar
  6. 6.
    Fast, M.: What the heck is PITCHf/x?. Harball Times, Baseball Annual (2010)Google Scholar
  7. 7.
    Pingali, G., Jean Y., Opalach, A.: Ball tracking and virtual replays for innovative tennis broadcasts. In: Proc. Int. Conf. Pat. Rec., pp. 152–156 (2000)Google Scholar
  8. 8.
    Yu, X., Xu, C., Leong, H.W., Tian, Q., Tang, Q., Wan, K.: Trajectory based ball detection and tracking with applications to semantic analysis of broadcast soccer video. In: Proc. ACM Int. Conf. Multimedia, pp. 11–20 (2003)Google Scholar
  9. 9.
    Huang, Y., Xu, D., Tan M., Su, H.: Trajectory prediction of spinning ball for ping–pong robot. In: Proc. IEEE/RSJ Int. Conf. Intel. Robots and Systems, pp. 3434–3439 (2011)Google Scholar
  10. 10.
    Chen, H.-T., Tien, M.-C., Chen, Y.-W., Tsai, W.-J., Lee, S.-Y.: Physics-based ball tracking and 3D trajectory reconstruction with applications to shooting location estimation in basketball video. J. Vis. Commun. Image Represent. 20(3), 204–216 (2009)CrossRefGoogle Scholar
  11. 11.
    Wang, J.R., Parameswaran, N.: Survey of sports video analysis: research issues and applications. In: Proc. Vis. Inform. Process., pp. 87–90 (2003)Google Scholar
  12. 12.
    Nathan, A.M.: Analysis of pitchf/x pitched baseball trajectories. The Physics of Baseball. [Online]. http://baseball.physics.illinois.edu/pitchtracker.html (2007)
  13. 13.
    Baker, S., Matthews, I.: Lucas–Kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(3), 221–255 (2004)CrossRefGoogle Scholar
  14. 14.
    Lepetit, V., Fua, P.: Monocular model-based 3D tracking of rigid objects: a survey. Found. Trends Comput. Graph. Vis. 1(1), 1–89 (2005)CrossRefGoogle Scholar
  15. 15.
    Boracchi, G., Caglioti, V., Giusti, A.: Single-image 3D reconstruction of ball velocity and spin from motion blur. In: Proc. VISAPP, pp. 22–25 (2008)Google Scholar
  16. 16.
    Szep, A.: Quantifying rotations of spheric objects. In: Proc. IAPR Conf. MVA, pp. 255–258 (2011)Google Scholar
  17. 17.
    Zhang, Y., Xiong, R., Zhao, Y., Wang, J.: Real-time spin estimation of Ping–Pong ball using its natural brand. IEEE Trans. Instrum. Measure. 64(8), 2280–2290 (2015)CrossRefGoogle Scholar
  18. 18.
    Tamaki, T., Sugino, T., Yamamoto, M.: Measuring ball spin by image registration. In: Proc. the 10th Korea-Japan Joint Workshop on FCV, pp. 269–274 (2004)Google Scholar
  19. 19.
    Shum H., Komura, T.: Tracking the translational and rotational movement of the ball using high-speed camera movies. In: Proc. IEEE ICIP, pp. 1084–1087 (2005)Google Scholar
  20. 20.
    Tamaki, T., Wang, H., Raytchev, B., Kaneda, K., Ushiyama, Y.: Estimating the spin of a table tennis ball using inverse compositional image alignment. In: Proc. IEEE ICASSP, pp. 1457–1460 (2012)Google Scholar
  21. 21.
    Theobalt, C., Albrecht, I., Haber, J., Magnor, M., Seidel, H.-P.: Pitching a baseball—tracking high-speed motion with multi-exposure images. ACM Trans. Graph. 23(3), 540–547 (2004)CrossRefGoogle Scholar
  22. 22.
    Passan, J.: 10 Degrees: the next baseball revolution is here, and spin is in. Yahoo sports. [Online]. http://sports.yahoo.com/news/10-degrees--the-next-baseball-revolution-is-here--and-spin-is-in-050433447-mlb.html (2015)
  23. 23.
    Ballard, D.: Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognit. 13(2), 111–122 (1981)CrossRefMATHGoogle Scholar
  24. 24.
    Ross, M.J., Shaffer, H.L., Cohen, A., Freudberg, R., Manley, H.J.: Average magnitude difference function pitch extractor. IEEE Trans. ASSP 22(5), 353–362 (1974)CrossRefGoogle Scholar
  25. 25.
    Muhammad, G.: Extended average magnitude difference function based pitch detection. Int. Arab J. Inf. Technol. 8(2), 197–208 (2015)Google Scholar

Copyright information

© Springer-Verlag London 2017

Authors and Affiliations

  1. 1.College of Information Science and EngineeringRitsumeikan UniversityKusatsuJapan
  2. 2.The University of Electro-CommunicationsChofuJapan
  3. 3.RIKENWakoJapan

Personalised recommendations