In the AdultSize league, the robot must be aware of its surroundings. Two stationary obstacles are avoided while planning, while a moving goalie should be avoided while kicking. The robot also detects a white ball, lines, and goal posts on the green turf field. We tackle these challenges using color classification, bitwise preprocessing operations, and strict shape filters. Our locomotion strategy depends on accurate goal post detection, calibrated locomotion odometry, and some line information.
A single Logitech C920 HD camera provides color pixels for processing. We train a Gaussian mixture model for applying maximum likelihood estimation of the color class of a pixel – white, green, off-white. The maximum likelihood estimate is encoded in a look-up table to categorize pixels into one of three color labels. We use two white color classes, where one class models the lines and one class models the ball and posts. We found that this disambiguates the ball from the lines very well in practice, but required a highly trained human supervisor in the model training stage.
The labeled image from the lookup table is then down-sampled for faster processing before being fed into high level blob detection routines and object classifiers. The down-sampled labeled image is a bitwise OR concatenation of the color classes of the pixels from the higher resolution image in the subsampled \(2\times 2\) block. The robot is able to detect the ball, goal posts, and lines based on their color, size, and shape information. Given the exact dimensions of objects, the distance and angle of detected objects are obtained by projection from the image frame to the robot’s egocentric coordinates.
In the AdultSize League in RoboCup 2015, two black cylinders are randomly placed between the robot and the goal. During a game, the robot must not touch these obstacles. Placed just before the “Play” command, the obstacles are immobile and so detection can be done once without a need for tracking. We do not model explicitly the color black, but only as “uncategorized colors.” Thus, blue or red colors are just as uncategorized as the black color of the obstacles. We apply blob detection used for the similarly shaped posts, but we add additional checks for surrounding green, much larger size. Also, we only consider pixels below the horizon by a certain angle. In this way, uncategorized pixel blobs off the field pose no issues.
The labeled image is scanned horizontally and the position of the lowest black pixel (depth of black) in each column is marked. The columns with black depth difference within a threshold are stitched together to be connected regions, and those are the candidates of obstacles which then go through a filtering process including height check, width check, on-the-ground check, and within-the-field check. A final filtering process is performed after transforming the positions of the detected obstacles from image frame to the robot’s egocentric, polar coordinates. The potential obstacles are clustered into groups based on their angles to the robot, and statistical information is calculated for each group providing the certainty and weighed average position of each obstacle. Finally, the closest two obstacles are registered for later trajectory planning.
5.2 White Objects
When the ball lies on a line, detection becomes difficult as the color classification leads to continuous, indiscriminate, white pixels. We built a pipeline of bitwise region erosion and dilation to eliminates weak pixel connections between a ball and lines/posts. The number of erosion steps and dilation steps are tuned to eliminate background noise – false positive ball and line pixels. We model erosion as bitwise AND on \(2\times 2\) blocks of pixels, while dilation is modeled as a bitwise OR operation on \(2\times 2\) blocks.
Distant balls and close balls require different erosion and dilation steps, but in practice, it is difficult to make these step choices scale invariant. While we used an additional color class for the particular shade of white on the ball, we found that bitwise erosion and dilation is useful for removing noisy color classifications (especially off the field) for field and line colors. With less noisy pixels, object checks for the ball can be tighter. To detect white field lines, we implement a decision tree for color class edges, where \(2\times 2\) blocks of pixels are mapped to a coordinate in Hough transform space. We modulate weights and coordinates for Hough transform counts. This more stringent guideline using edges for line population finds Green/white/neither edges on \(2\times 2\) blocks. We are able to find good line statistics even with a ball on a line far away, as shown in Fig. 8.
In previous years, we used a particle filter to track the three-dimensional robot pose \((x, y, \theta )\) on the field, in which orientation is reset when the robot falls down. The particles are probabilistically updated using a motion model based on the odometry of the robot and yaw angle based on inertial measurement, as well as vision information that takes into account pre-specified positions of landmarks such as goal posts, lines and corners.
This year we could further simplify this framework due to the uniqueness of the AdultSize league: the robot always starts at exactly the same initial position. It is never supposed to fall down, and each trial has quite a short time limit of 90 s. This makes the inertial tracking of the robot heading angle \(\theta \) very precise during the match, so we assume that the current estimate of heading direction is correct. Then we discard the distance information from goalpost observations, which can be quite noisy for distant goalposts, to handle the redundant information. This simplification greatly helped the localization during the whole competition, where the localization error was hardly visible.