Feature-set characterization for target detection based on artificial color contrast and principal component analysis with robotic tealeaf harvesting applications

Abstract

For the benefits of productivity, efficiency and sustainability, the adoption of machine vision with robotics in applications has become essential in agriculture industry; for example, outdoor tender-tealeaf harvesting, where tasks are highly repetitive and laborious. As machine vision must cope with changing daylight conditions, among the challenges is an efficient technique to accurately detect/locate randomly distributed targets in natural background that has closely similar color as the targets of interests. The success of developing a harvesting machine depends on robust, accurate, and efficient target-detection, which is the first step of the automation. Built upon the concept of an artificial color contrast (ACC) model developed for color-feature classification using principal component analysis (PCA), this paper presents an improved ACC/PCA method to overcome commonly encountered target-detection problems for outdoor agriculture applications where targets in a closely similar background must be identified/located in real time for subsequent robotic handling. Specifically, several methods have been developed to determine an optimal feature-set boundary for effective target detection, which require only a limited set of training data. The effectiveness of the methods is evaluated using experimentally obtained samples in terms of three practical measures (% detection error, % numerical noise and computation time) by comparing results with commonly used methods.

Appendices

Appendix

Principal Component Analysis (PCA)

Given N RGB samples, a covariance matrix [C] can be calculated from (24, 25) where $i,j \in \{ R,\;G,\;B\}$:

$$ \left[ {\mathbf{C}} \right] = \left[ {\begin{array}{*{20}c} {\text{cov} (R{\mathbf{,}}R)} & {\text{cov} (R,G)} & {\text{cov} (R,B)} \\ {\text{cov} (G,R)} & {\text{cov} (G{\mathbf{,}}G)} & {\text{cov} (G,B)} \\ {\text{cov} (B,R)} & {\text{cov} (B,G)} & {\text{cov} (B,B)} \\ \end{array} } \right] $$

(24)

$$ {\text{where }}\text{cov} \left( {i,j} \right) = \frac{1}{{N - 1}}\sum\limits_{{n = 1}}^{N} {\left( {i_{n} - \bar{i}} \right)\left( {j_{n} - \bar{j}} \right)} {\text{; }}\left( {\bar{i},\bar{j}} \right) = \frac{1}{N}\sum\limits_{{n = 1}}^{N} {\left( {i_{n} ,j_{n} } \right)}. {\text{ }} $$

(25)

Using (26) where (v₁ v₂ v₃) are the eigenvectors corresponding to the eigenvalues (λ₁, λ₂, λ₃) of [C] in descending order, image I_RGB in RGB color space can be transformed to I_PCA in the PCA coordinate system:

$$ {\mathbf{I}}_{{{\text{PCA}}}} (k,\ell ) = \left[ {\mathbf{A}} \right]\left( {{\mathbf{I}}_{{{\text{RGB}}}} {\text{ }}(k,\ell ) - {\mathbf{\bar{I}}}_{{{\text{RGB}}}} } \right) $$

(26)

where $\left[ {\mathbf{A}} \right] = \left[ {\begin{array}{*{20}c} {{\mathbf{v}}_{{\text{1}}}^{{\text{T}}} } & {{\mathbf{v}}_{2}^{{\text{T}}} } & {{\mathbf{v}}_{3}^{{\text{T}}} } \\ \end{array} } \right]^{{\text{T}}}$ and $\overline{{\mathbf{I}}} _{{{\text{RGB}}}} = \left[ {\overline{{\text{I}}} _{{\text{R}}} \;\overline{{\text{I}}} _{{\text{G}}} \;\overline{{\text{I}}} _{{\text{B}}} } \right]^{{\text{T}}}$

$$ {\text{In RGB color space}},\quad {\mathbf{I}}_{{{\text{RGB}}}} (k,\ell ) = \left[ {\mathbf{A}} \right]^{{ - 1}} \left[ {{\mathbf{I}}_{{{\text{PCA}}}} (k,\ell )} \right] + {\mathbf{\bar{I}}}_{{{\text{RGB}}}} $$

(27)