Image Feature Extraction

Abstract

In the last chapter we discussed the acquisition and processing of images. We learned that images are simply large arrays of pixel values but for robotic applications images have too much data and not enough information. We need to be able to answer pithy questions such as what is the pose of the object? what type of object is it? how fast is it moving? how fast am I moving? and so on. The answers to such questions are measurements obtained from the image and which we call image features. Features are the gist of the scene and the raw material that we need for robot control.

The image processing operations from the last chapter operated on one or more input images and returned another image. In contrast, feature extraction operates on a single image and returns one or more image features. Features are typically scalars (for example, object area or aspect ratio) or short vectors (for example the coordinate of an object, the parameters of a line, the corners of a bounding box). Image feature extraction is an essential first step in using image data to control a robot. It is an information concentration step that reduces the data rate from 10${}^{6}$–10${}^{8}$ bytes s${}^{-1}$ at the output of a camera to something of the order of tens of features per frame that can be used as input to a robot’s control system.