Color perception plays an important role in object recognition and scene understanding both for humans and intelligent vision systems. Recent advances in digital color imaging and computer hardware technology have led to an explosion in the use of color images in a variety of applications including medical imaging, content-based image retrieval, biometrics, watermarking, digital inpainting, remote sensing, visual quality inspection, among many others. As a result, automated processing and analysis of color images has become an active area of research, which is witnessed by the large number of publications during the past two decades. The multivariate nature of color image data presents new challenges for researchers and practitioners as the numerous methods developed for single-channel images are often not directly applicable to multichannel images. In addition, common scalar image processing operations often become computationally intractable when performed in high-dimensional spaces. As a result, the development of computationally practical methods for color image processing has become an important research area.

The goals of this special issue are to summarize the state-of-the-art in real-time color image processing and to provide future directions for this exciting subfield of image processing. The intended audience includes researchers and professionals, who are increasingly dealing with the processing and analysis of color images and video.

The special issue opens with “Design of a Shift-and-Add Based Hardware Accelerator for Color Space Conversion” by Li et al. The authors propose a hardware/software co-design architecture for Nios II, a \(32\)-bit embedded processor. The architecture integrates a pipelined color space converter hardware accelerator based on a genetic algorithm and an LCD touch module. When implemented on a system-on-a-programmable-chip, the proposed architecture is capable of converting a \(512 \times 512\) RGB image to YCbCr in \(0.11\) s.

The special issue continues with seven articles on color image enhancement. In “A Simple Gray-Edge Automatic White Balance Method with FPGA Implementation,” Tan et al. propose a real-time automatic white balance method based on the gray-edge hypothesis. To reduce computational time, the method employs horizontal downsampling using the mean filter and gradient computation using horizontal first-order difference operator. The authors demonstrate the practicality of their approach using an FPGA implementation.

In “A Full Linear \(3 \times 3\) Color Correction between Images,” Lecca describes a fast color correction algorithm that is capable of equalizing the colors of two images of the same scene that are acquired under different illuminants and/or by different devices. The author demonstrates the performance of her approach against two state-of-the-art approaches on a variety of synthetic and real-world image databases. The method has linear time complexity in the number of pixels, is invariant to scaling and rotation, and does not require any sensor preprocessing.

In “Novel Multi-scale Retinex with Color Restoration on Graphics Processing Unit,” Jiang et al. proposes an improvement over the Mutli-scale Retinex with Color Restoration algorithm. The authors then describe a GPU accelerated implementation of their improved contrast enhancement algorithm. In their experiments with \(1024 \times 1024\) images, the authors obtain an impressive \(45\times \) speedup over the single-threaded CPU implementation.

In “An Adaptive Dynamic Range Compression with Local Contrast Enhancement Algorithm for Real-Time Color Image Enhancement,” Tsai and Huang propose a fast color image enhancement method that uses dynamic range compression based on an image-dependent nonlinear intensity-transfer function and local contrast enhancement. The authors also present an accelerated version of their method based on two-dimensional lookup table and linear interpolation. Extensive experiments demonstrate that the proposed enhancement method achieves real-time performance on \(1080\)p video streams.

In “High Dynamic Range Imaging Pipeline on the GPU,” Akyüz describes how the high dynamic range imaging (HDRI) pipeline, from HDR image assembly to tone mapping, can be implemented exclusively on the GPU. The author also explains the trade-offs that need to be made for improving efficiency and shows timing comparisons for CPU versus GPU implementations of the HDRI pipeline.

In “Adaptive Rank Weighted Switching Filter for Impulsive Noise Removal in Color Images,” Smolka et al. presents a new approach to the problem of impulsive noise suppression in color images. The proposed switching filter is based on rank-weighted, cumulative dissimilarity measures, which are utilized for the detection of contaminated pixels. This novel design enables the filter to adapt its parameters to the image corruption intensity. Comparisons with existing denoising schemes show that the introduced technique more efficiently removes the impulsive noise, while better preserving image details. The low computational complexity of the proposed design enables its application in real-time computer vision tasks.

In “Fuzzy 3D Filter for Color Video Sequences Contaminated by Impulsive Noise,” Ponomaryov et al. describe a fuzzy filter designed to restore image sequences contaminated by impulsive noise. The new filtering scheme analyzes the fuzzy spatial gradient values calculated in the direction of the neighboring pixels in three successive video frames. The detection of impulse noise is based mainly on the correlations between the frame components and the filter output is calculated using the fuzzy weights assigned to the spatio-temporal local neighborhood. Comparisons of the restoration quality obtained using the new approach with the state-of-the-art designs, exhibit the excellent properties of the proposed video denoising framework.

The special issue continues with two articles on color image segmentation. In “An Effective Real-Time Color Quantization Method Based on Divisive Hierarchical Clustering,” Celebi et al. describe a fast and effective color reduction method based on divisive hierarchical clustering. The authors employ the commonly used binary splitting strategy along with several carefully selected heuristics that ensure a good balance between effectiveness and efficiency. Experiments on a diverse set of publicly available images demonstrate that the proposed method outperforms \(15\) well-known quantizers. Trieu and Maruyama, in their work titled “Real-Time Color Image Segmentation Based on Mean Shift Algorithm Using an FPGA,” describe the details of a fast implementation of a well-known image segmentation algorithm. Computational complexity reduction is achieved using a cache memory that allows accessing pixels belonging to the local image neighborhood. Additionally, the region merging process is also accelerated using the proposed processing pipeline. Performance analysis shows that the described implementation architecture can be used for real-time applications in which segmentation of objects play a significant role.

The special issue continues with three articles on detection. In their paper “Novel Haar Features for Real Time Hand Gesture Recognition using SVM,” Hsieh and Liou propose a robust real-time vision system performing an adaptive face and skin detection and hand gesture recognition. For the classification of dynamic gestures, novel Haar-like features are investigated and an efficient algorithm utilizing a Support Vector Machine classifier is developed. Experimental results reveal the high accuracy of the proposed hand gesture-recognition system.

The problem of skin detection is also investigated in a paper titled “Dynamic Approach for Real-Time Skin Detection,” by Akmeliawati et al. To accurately detect human skin color pixels and to decrease the number of false results, a prior face or hand detection model is developed using Haar-Like features and AdaBoost technique. To decrease the computational cost, a fast search algorithm for skin detection was elaborated. Experimental analyses show that level of performance reached in terms of detection accuracy and processing time allows this approach to be an satisfactory choice for real-time skin blob tracking.

Losson and Macair, in their paper “CFA Local Binary Patterns for Fast Illuminant-Invariant Color Texture Classification,” present an approach to color texture analysis that does not require demosaicing of the CCD data. Their approach relies on rank-based local binary pattern texture descriptors that are calculated from component images of the color filter array data, and is shown to provide better texture recognition compared to features extracted from demosaiced images.

The special issue continues with three articles on tracking. In “Color Tracking with Contextual Switching,” Laguzet et al. propose a tracking algorithm that comprises two complementary approaches based on mean-shift and covariance tracking. While for the mean-shift technique the color space is adapted to provide better foreground–background discrimination, the covariance method is employed in case the mean-shift tracker fails. The combined approach is shown to run in real-time on a quad-core architecture.

A closely related problem, namely background subtraction is addressed by Holtzhausen et al. in their contribution “An Illumination Invariant Framework for Real-time Foreground Detection.” The authors present a generic approach based on the principles of object persistence and reintegration, illumination robustness and resistance to environmental effects, while focusing on the real-time aspects of background modeling.

In “Hierarchical Architecture for Motion and Depth Estimations based on Color Cues,” Barranco et al. describe an FPGA implementation for motion and disparity estimation using color information. Based on a multi-scale approach to color optical flow, the proposed system is shown to run in real time.

An application-oriented article titled “AVScreen: A Real-Time Video Augmentation Method” by Hernandez-Lopez and Rivera completes this special issue. The authors introduce a tool that, as the title suggests, supports augmentation of videos in real time. The authors present a method for stabilizing camera videos as well as an approach to foreground–background segmentation to deal with occlusions, while the system is implemented on CUDA architecture to provide real-time editing.

The editors are grateful to the journal editors-in-chief and the editorial staff for providing technical support, to the authors for taking the time and effort to revise their submissions, and to the reviewers for diligently evaluating these submissions and offering constructive criticism. We hope that this special issue focused on real-time color image processing will demonstrate the significant progress that has occurred in this field during the past few years. We also hope that the developments reported in this issue will motivate further research in this field.