Analog Integrated Circuits and Signal Processing

, Volume 46, Issue 2, pp 121–138 | Cite as

Analog Integrated 2-D Optical Flow Sensor

Article

Abstract

I present a new focal-plane analog very-large-scale-integrated (aVLSI) sensor that estimates optical flow in two visual dimensions. Its computational architecture consists of a two-layer network of locally connected motion units that collectively estimate the optimal optical flow field. The applied gradient-based optical flow model assumes visual motion to be translational and smooth, and is formulated as a convex optimization problem. The model also guarantees that the estimation problem is well-posed regardless of the visual input by imposing a bias towards a preferred motion under ambiguous or noisy visual conditions. Model parameters can be globally adjusted, leading to a rich output behavior. Varying the smoothness strength, for example, can provide a continuous spectrum of motion estimates, ranging from normal to global optical flow. The non-linear network conductances improve the resulting optical flow estimate because they reduce spatial smoothing across large velocity differences and minimize the bias for reliable stimuli. Extended characterization and recorded optical flow fields from a 30 × 30 array prototype sensor demonstrate the validity of the optical flow model and the robustness and functionality of the computational architecture and its implementation.

Key Word

recurrent network neuromorphic constraint satisfaction regularization parallel computation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    C. Mead, “Neuromorphic electronic systems.” Proceedings of the IEEE, vol. 78, no. 10, pp. 1629–1636, 1990.Google Scholar
  2. 2.
    P. Nesi, F. Innocenti, and P. Pezzati, “Retimac: Real-time motion analysis chip.” IEEE Trans. on Circuits and Systems - 2: Analog and Digital Signal Processing, vol. 45, no. 3, pp. 361–375, 1998.Google Scholar
  3. 3.
    E.C. Hildreth, The Measurment of Visual Motion. MIT Press, 1983.Google Scholar
  4. 4.
    T. Horiuchi, J.P. Lazzaro, A. Moore, and C. Koch, “A delay-line based motion detection chip.” in Advances in Neural Information Processing Systems 3, R. Lippman, J. Moody, and D. Touretzky (Eds.), 1991, vol. 3, pp. 406–412.Google Scholar
  5. 5.
    R. Etienne-Cummings, S. Fernando, N. Takahashi, V. Shotonov, J. Van der Spiegel, and P. Müller, “A new temporal domain optical flow measurement technique for focal plane VLSI implementation.” in Computer Architectures for Machine Perception, December 1993, pp. 241–250.Google Scholar
  6. 6.
    R. Sarpeshkar, W. Bair, and C. Koch, “Visual motion computation in analog VLSI using pulses.” in Advances in Neural Information Processing Systems, vol. 5, pp. 781–788, 1993.Google Scholar
  7. 7.
    J. Kramer, “Compact integrated motion sensor with three-pixel interaction.” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 18, no. 4, pp. 455–460, 1996.CrossRefGoogle Scholar
  8. 8.
    C. Higgins and C. Koch, “Analog CMOS velocity sensors.” Procs. of Electronic Imaging SPIE, vol. 3019, February 1997.Google Scholar
  9. 9.
    C. Higgins, R. Deutschmann, and C. Koch, “Pulse-based 2D motion sensors.” IEEE Trans. on Circuits and Systems 2: Analog and Digital Signal Processing, vol. 46, no. 6, pp. 677–687, 1999.Google Scholar
  10. 10.
    T. Horiuchi, B. Bishofberger, and C. Koch, “An analog VLSI saccadic eye movement system.” in Advances in Neural Information Processing Systems 6, J. Cowan, G. Tesauro, and J. Alspector (Eds.), Morgan Kaufmann, 1994, pp. 582–589.Google Scholar
  11. 11.
    R. Deutschmann and C. Koch, “An analog VLSI velocity sensor using the gradient method,” in Procs. IEEE Intl. Symposium on Circuits and Systems, IEEE, 1998, pp. 649–652.Google Scholar
  12. 12.
    R. Deutschmann and C. Koch, “Compact real-time 2-D gradient-based analog VLSI motion sensor.” in Proc. SPIE Int. Conference on Advanced Focal Plane Arrays and Electronic Cameras, 1998, vol. 3410, pp. 98–108.Google Scholar
  13. 13.
    A. Andreou and K. Strohbehn, “Analog VLSI implementation of the Hassenstein-Reichardt-Poggio models for vision computation,” in Procs. of the 1990 Intl. Conference on Systems, Man and Cybernetics, 1990.Google Scholar
  14. 14.
    R. Benson and T. Delbrück, “Direction-selective silicon retina that uses null inhibition.” in Neural Information Processing Systems 4, D. Touretzky, (Ed.), MIT Press, 1992, pp. 756–763.Google Scholar
  15. 15.
    T. Delbrück, “Silicon retina with correlation-based velocity-tuned pixels.” IEEE Trans. on Neural Networks, vol. 4, no. 3, pp. 529–541, 1993.Google Scholar
  16. 16.
    R. Harrison and C. Koch, “An analog VLSI model of the fly elementary motion detector.” in Advances in Neural Information Processing Systems 10, M. Kearns and S. Solla (Eds.), MIT Press, 1998, pp. 880–886.Google Scholar
  17. 17.
    M. Ohtani, T. Asai, H. Yonezu, and N. Ohshima, “Analog velocity sensing circuits based on bio-inspired correlation neural networks,” in Microelectronics for Neural, Fuzzy and Bio-Inspired Systems, Granada, Spain, 1999, pp. 366–373.Google Scholar
  18. 18.
    S.-C. Liu, “A neuromorphic aVLSI model of global motion processing in the fly.” IEEE Trans. on Circuits and Systems 2, vol. 47, no. 12, pp. 1458–1467, 2000.Google Scholar
  19. 19.
    H.-C. Jiang and C.-Y. Wu, “A 2-D velocity- and direction-selective sensor with BJT-based silicon retina and temporal zero-crossing detector.” IEEE Journal of Solid-State Circuits, vol. 34, no. 2, pp. 241–247, 1999.Google Scholar
  20. 20.
    J. Kramer, R. Sarpeshkar, and C. Koch, “Pulse-based analog VLSI velocity sensors.” IEEE Trans. on Circuits and Systems 2, vol. 44, no. 2, pp. 86–101, 1997.Google Scholar
  21. 21.
    R. Etienne-Cummings, J. Van der Spiegel, and P. Mueller, “A focal plane visual motion measurement sensor.” Trans. on Circuits and Systems I, vol. 44, no. 1, pp. 55–66, 1997.Google Scholar
  22. 22.
    S. Mehta and R. Etienne-Cummings, “Normal optical flow measurement on a CMOS APS imager.” in Int. Symposium on Circuits and Systems, Vancouver: IEEE, May 2004, vol. 4, pp. 848–851.Google Scholar
  23. 23.
    J. Tanner and C. Mead, “An integrated analog optical motion sensor.” in VLSI Signal Processing, 2, S.-Y. Kung, R. Owen, and G. Nash, (Eds) IEEE Press, 1986, p. 59 ff.Google Scholar
  24. 24.
    A. Moore and C. Koch, “A multiplication based analog motion detection chip.” in SPIE Visual Information Processing: From Neurons to Chips, B. Mathur and C. Koch (Eds.), SPIE, 1991, vol. 1473, pp. 66–75.Google Scholar
  25. 25.
    A. Stocker and R. Douglas, “Computation of smooth optical flow in a feedback connected analog network.” in Advances in Neural Information Processing Systems 11, M. Kearns, S. Solla, and D. Cohn (Eds.), MIT Press: Cambridge, MA, 1999, pp. 706–712.Google Scholar
  26. 26.
    M.-H. Lei and T.-D. Chiueh, “An analog motion field detection chip for image segmentation.” IEEE Trans. on Circuits and Systems for Video Technology, vol. 12, no. 5, pp. 299–308, 2002.Google Scholar
  27. 27.
    A. Stocker, “Constraint optimization networks for visual motion perception - analysis and synthesis.” Ph.D. dissertation, Dept. of Physics, Swiss Federal Institute of Technology, Zürich, Switzerland, March 2002.Google Scholar
  28. 28.
    A. Stocker, “Analog VLSI focal-plane array with dynamic connections for the estimation of piece-wise smooth optical flow.” IEEE Transactions on Circuits and Systems-1: Regular Papers, vol. 51, no. 5, May 2004, pp. 963–973.Google Scholar
  29. 29.
    A. Stocker and R. Douglas, “Analog integrated 2-D optical flow sensor with programmable pixels,” in IEEE Int. Symposium on Circuits and Systems, Vancouver: IEEE, May 2004, vol. 3, pp. 9–12.Google Scholar
  30. 30.
    J. Limb and J. Murphy, “Estimating the velocity of moving images in television signals.” Computer Graphics Image Processing, vol. 4, pp. 311–327, 1975.CrossRefGoogle Scholar
  31. 31.
    B. Horn and B. Schunck, “Determining optical flow.” Artificial Intelligence, vol. 17, pp. 185–203, 1981.CrossRefGoogle Scholar
  32. 32.
    Y. Weiss, E. Simoncelli, and E. Adelson, “Motion illusions as optimal percept.” Nature Neuroscience, vol. 5, no. 6, pp. 598–604, 2002.CrossRefGoogle Scholar
  33. 33.
    A. Yuille and N. Grzywacz, “A mathematical analysis of the motion coherence theory.” Intl. Journal of Computer Vision, vol. 3, pp. 155–175, 1989.Google Scholar
  34. 34.
    E. Simoncelli, E. Adelson, and D. Heeger, “Probability distributions of optical flow.” in IEEE Conference on Computer Vision and Pattern Recognition. IEEE, June 1991, pp. 310–313.Google Scholar
  35. 35.
    J. Harris, C. Koch, E. Staats, and J. Luo, “Analog hardware for detecting discontinuities in early vision.” Intl. Journal of Computer Vision, vol. 4, pp. 211–223, 1990.Google Scholar
  36. 36.
    J. Hopfield, “Neurons with graded response have collective computational properties like those of two-state neurons.” Procs. National Academic Sciences U.S.A., vol. 81, pp. 3088–3092, May 1984.Google Scholar
  37. 37.
    J. Hopfield and D. Tank, “Neural computation of decisions in optimization problems.” Biological Cybernetics, no. 52, pp. 141–152, 1985.Google Scholar
  38. 38.
    T. Delbrück and C. Mead, “Analog VLSI phototransduction by continuous-time, adaptive, logarithmic photoreceptor cirucuits.” Caltech Computation and Neural Systems Program, Tech. Rep. 30, 1994.Google Scholar
  39. 39.
    S.-C. Liu, “Silicon retina with adaptive filtering properties.” in Advances in Neural Information Processing Systems 10. MIT Press, November 1998, pp. 712–718.Google Scholar
  40. 40.
    C. Mead, Analog VLSI and Neural Systems. Addison-Wesley: Reading, MA, 1989.MATHGoogle Scholar
  41. 41.
    B. Gilbert, “A precise four-quadrant multiplier with subnanosecond response.” IEEE Journal of Solid-State Circuits, vol. 3, no. 4, pp. 363–373, December 1968.Google Scholar
  42. 42.
    P. Gray and R. Meyer, Analysis and Design of Analog Integrated Circuits, 3rd ed. Wiley and Sons: New York, 1993.Google Scholar
  43. 43.
    A. Stocker and E. Simoncelli, “Constraining a Bayesian model of human visual speed perception,” in Advances in Neural Information Processing and Systems NIPS, vol. 17. MIT Press: Vancouver, 2005, to appear.Google Scholar
  44. 44.
    J. Harris, C. Koch, and J. Luo, “A two-dimensional analog VLSI circuit for detecting discontinuities in early vision.” Science, vol. 248, pp. 1209–1211, June 1990.Google Scholar
  45. 45.
    T. Delbrück, “Investigations of analog VLSI visual transduction and motion processing.” Ph.D. dissertation, Department of Computational and Neural Systems, California Institute of Technology, Pasadena, CA, 1993.Google Scholar
  46. 46.
    J. Heinzle and A. Stocker, “Classifying patterns of visual motion—a neuromorphic approach.” in Advances in Neural Information Processing Systems 15, S. T. S. Becker and K. Obermayer (Eds.), MIT Press: Cambridge, MA, 2003, pp. 1123–1130.Google Scholar
  47. 47.
    V. Becanovic, G. Indiveri, H.-U. Kobialka, Plöger, and A. Stocker, Mechatronics and Machine Vision 2002: Current Practice, ser. Robotics. Research Studies Press, 2002, ch. Silicon Retina Sensing guided by Omni-directional Vision, pp. 13–21.Google Scholar
  48. 48.
    S. Treue and J. Maunsell, “Attentional modulation of visual motion processing in cortical areas MT and MST.” Nature, vol. 382, pp. 539–541, August 1996.CrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, Inc. 2005

Authors and Affiliations

  1. 1.Howard Hughes Medical Institute and Center for Neural ScienceNew York UniversityNew YorkUSA

Personalised recommendations