A CMOS Vision System On-Chip with Multi-Core, Cellular Sensory-Processing Front-End
This chapter describes a vision-system-on-chip (VSoC) capable of doing: image acquisition, image processing through on-chip embedded structures, and generation of pertinent reaction commands at thousands frame-per-second rate. The chip employs a distributed processing architecture with a pre-processing stage consisting of an array of programmable sensory-processing cells, and a post-processing stage consisting of a digital microprocessor. The pre-processing stage operates as a retina-like sensor front-end. It performs parallel processing of the images captured by the sensors which are embedded together with the processors. This early processing serves to extract image features relevant to the intended tasks. The front-end incorporates also smart read-out structures which are conceived to transmit only these relevant features, thus precluding full gray-scale frames to be coded and transmitted. The chip is capable to close action–reaction loops based on the analysis of visual flow at rates above 1,000 F/s with power budget below 1 W peak. Also, the incorporation of processors close to the sensors enables signal-dependent, local adaptation of the sensor gains and hence high-dynamic range signal acquisition.
KeywordsSingle Instruction Multiple Data High Dynamic Range Image Linear Convolution Digital Processor CMOS Chip
Authors would like to acknowledge fruitful discussions with Dr. Ricardo Carmona, Dr. Gustavo Liñán, Dr. Akos Zarandy, and Dr. Piotr Dudek.
The work of Prof. Rodríguez-Vázquez has been partially supported by the Spanish project 2006-TIC-2352 and the PIMA program of the CICE/JA.
- AnaFocus Ltd., http://www.anafocus.com
- Chih-Chi Cheng et al (2008) iVisual: An intelligent visual sensor SoC with 2790fps CMOS image sensor and 205GOPS/W vision processor. IEEE Int Solid-State Circuits Conf Dig Tech Papers 306–307, San Francisco CAGoogle Scholar
- Delbruck T, Lichsteiner P (2006) Freeing vision from frames. Neuromorphic Eng 3:3–4Google Scholar
- Devaraj G, et al (2008) Applying algorithms. Vis Syst Design 13(11):17–20; 85–87Google Scholar
- Duller A, et al (2003) Parallel processing -the picoChip way! Communicating process architectures – 2003. IOS Press, pp 125–138Google Scholar
- El Gamal A, Eltoukhy H (2005) CMOS image sensors. IEEE Circuits Devices Mag 6–20Google Scholar
- ENIAC working group (2007) Strategic research agenda, 2nd edn. European Technology Platform Initiative, November 2007*.Google Scholar
- Green WE, et al (2004) Flying insect-inspired vision for autonomous aerial robot maneuvers in near-earth environment. Proc IEEE Int Conf Robotics Automat 2347–2352, New Orleans LA, April-MayGoogle Scholar
- International Technology Roadmap for Semiconductors (ITRS) (2007) Edition emerging research devices, http://www.itrs.net/Links/2007ITRS/Home2007.htm
- Kapasi UJ, et al (2003) Programmable stream processors. IEEE Comput 36(8):54–62Google Scholar
- Keckler S, et al (2003) A wire-delay scalable microprocessor architecture for high performance systems. IEEE Int Solid-State Circuits Conf Dig Tech Papers 1:168–169Google Scholar
- Kyo S, et al (2005) An integrated memory array processor architecture for embedded image recognition systems. Proc 32nd Int Symp Comput Arch (ISCA’05) 134–145Google Scholar
- Morris K (2008) A passel of processors: NVIDIA’s Tesla T10P blurs some lines. FPGA Structured ASIC J, June 2008 (on-line: http://www.fpgajournal.com/articles_2008/20080617_nvidia.htm)
- Pham D, et al (2005) The design and implementation of a first-generation CELL processor. IEEE Int Solid-State Circuits Conf Dig Tech Papers 184–185Google Scholar
- Philipp RM, et al (2006) A 128 ×128 33 mW 30 frames/s single-chip stereo imager. IEEE Int Solid-State Circuits Conf (ISSCC 2005) Digest Tech Papers 2050–2059Google Scholar
- Roska T, Rodríguez-Vázquez A (2001) Towards the visual microprocessor. Wiley, Chichecter UKGoogle Scholar
- Russ JC (1992) The image processing handbook. CRC Press, Boca RatonGoogle Scholar
- Seiler L, et al (2008) Larrabee: A many-core x86 architecture for visual computing. ACM Transact Graphics 27(3)Google Scholar
- Vangal S, et al (2007) An 80-tile 1.28TFLOPS network-on-chip in 65nm CMOS. Int Solid-State Circuits Conf Dig Tech Papers 98–99Google Scholar