Object Recognition and Localisation for Item Picking

Abstract

One of the challenges of future retail warehouses is automating the order-picking process. To achieve this, items in an order tote must be automatically detected and grasped under various conditions. A product recognition and localisation system for automated order-picking in retail warehouses was investigated, which is capable of recognising objects that have a descriptor in the warehouse product database containing both 2D and 3D features. The 2D features are derived from normal CMOS camera images and the 3D features from time-of-flight camera images. 2D features perform best when the object is relatively rigid, illuminated uniformly, and has enough texture. They can cope with partial occlusions and are invariant to rotation, translation, scale, and affine transformations up to some level. 3D features can be fruitfully used for the detection and localisation of objects without texture or dominant colour. The 2D system has a performance of 2–3 frames-per-second (fps) at about 400 extracted features, good enough for a pick-and-place robot. Almost all rigid items with enough texture could be recognised. The method can cope with partial occlusions. The 3D system is insensitive to lighting conditions and finds 3D point clouds, from which geometric descriptions of planes and edges are derived as well as their pose in 3D. The 3D system is a welcome addition to the 2D system, mainly for box-shaped objects without much texture or.