Advertisement

Towards Real-Time Edge Detection for Event Cameras Based on Lifetime and Dynamic Slicing

  • Sherif A. S. MohamedEmail author
  • Mohammad-Hashem Haghbayan
  • Jukka Heikkonen
  • Hannu Tenhunen
  • Juha Plosila
Conference paper
  • 144 Downloads
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1153)

Abstract

Retinal cameras, such as dynamic vision sensors (DVF), transmit asynchronous events with ultra-low latency (\(\sim \)10 \(\upmu \)s) only at significant luminance changes, unlike traditional CMOS cameras which transmit the absolute brightness of all pixels including redundant backgrounds. Due to these significant characteristics, they offer great potential to obtain efficient localization of high-speed and agile platforms. Moreover, event cameras have a high dynamic range (\({\sim }\)140 dB), which makes them suitable for platforms that operate indoors in low-lighting scenarios and in outdoor environments, where the camera might be pointing at a strong light source, e.g. the sun. In this paper, we propose an algorithm to detect edges in event streams coming from retinal cameras. To do that, an algorithm is developed to extract edges from events by augmenting a batch of events with their lifetimes. The lifetime of each event is computed using a local plane fitting technique. We use a batching technique to increase the frame rate of generated images since events with a high sample rate cause the processing of a single event to be computationally expensive. The size of the batch will be adjusted based on the mean optical flow of the previously generated batch. The obtained experimental results show that our proposed technique can significantly reduce the response time with the same sharpness in generating the edges.

Keywords

Edge detector Event camera Dynamic Vision Sensor (DVS) SLAM Visual odometry 

References

  1. 1.
    Lichtsteiner, P., Posch, C., Delbruck, T.: A \(128\times 128\) 120 dB 15 \(\upmu \)s latency asynchronous temporal contrast vision sensor. IEEE J. Solid-State Circuits 43(2), 566–576 (2008)CrossRefGoogle Scholar
  2. 2.
    Brandli, C., Berner, R., Yang, M., Liu, S., Delbruck, T.: A \(240\times 180\) 130 dB 3 \(\upmu \)s latency global shutter spatiotemporal vision sensor. IEEE J. Solid-State Circuits 49(10), 2333–2341 (2014)CrossRefGoogle Scholar
  3. 3.
    Gallego, G., Delbrück, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A., Conradt, J., Daniilidis, K., Scaramuzza, D.: Event-based vision: a survey. CoRR, vol. abs/1904.08405 (2019)Google Scholar
  4. 4.
    Mohamed, S.A.S., Haghbayan, M., Westerlund, T., Heikkonen, J., Tenhunen, H., Plosila, J.: A survey on odometry for autonomous navigation systems. IEEE Access 7, 97466–97486 (2019)CrossRefGoogle Scholar
  5. 5.
    Liu, M., Delbruck, T.: Adaptive time-slice block-matching optical flow algorithm for dynamic vision sensors, September 2018Google Scholar
  6. 6.
    Rebecq, H., Horstschaefer, T., Gallego, G., Scaramuzza, D.: EVO: a geometric approach to event-based 6-DOF parallel tracking and mapping in real time. IEEE Robot. Autom. Lett. 2(2), 593–600 (2017)CrossRefGoogle Scholar
  7. 7.
    Mohamed, S.A.S., Haghbayan, M., Heikkonen, J., Tenhunen, H., Plosila, J.: Towards dynamic monocular visual odometry based on an event camera and IMU sensor. In: Intelligent Transport Systems, From Research and Development to the Market Uptake (INTSYS 2019). Springer (2020)Google Scholar
  8. 8.
    Mlsna, P., Rodriguez, J.: Gradient and Laplacian edge detection, pp. 495–524. Elsevier Inc. (2009)Google Scholar
  9. 9.
    Marr, D., Hildreth, E.: Theory of edge detection. Proc. R. Soc. Lond. Ser. B 207, 187–217 (1980)Google Scholar
  10. 10.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–8(6), 679–698 (1986)CrossRefGoogle Scholar
  11. 11.
    Leavers, V.: Which hough transform? CVGIP Image Underst. 58(2), 250–264 (1993)CrossRefGoogle Scholar
  12. 12.
    Seifozzakerini, S., Yau, W.-Y., Zhao, B., Mao, K.: Event-based hough transform in a spiking neural network for multiple line detection and tracking using a dynamic vision sensor. In: Proceedings of the British Machine Vision Conference (BMVC), pp. 94.1–94.12. BMVA Press, January 2016Google Scholar
  13. 13.
    von Gioi, R.G., Jakubowicz, J., Morel, J., Randall, G.: LSD: a line segment detector. IPOL J. 2, 35–55 (2012)CrossRefGoogle Scholar
  14. 14.
    Brändli, C., Strubel, J., Keller, S., Scaramuzza, D., Delbruck, T.: ELiSeD – an event-based line segment detector. In: 2016 Second International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP), pp. 1–7, June 2016Google Scholar
  15. 15.
    Barranco, F., Teo, C.L., Fermüller, C., Aloimonos, Y.: Contour detection and characterization for asynchronous event sensors. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 486–494, December 2015Google Scholar
  16. 16.
    Mueggler, E., Forster, C., Baumli, N., Gallego, G., Scaramuzza, D.: Lifetime estimation of events from dynamic vision sensors. In: IEEE International Conference on Robotics and Automation (ICRA 2015), Seattle, WA, USA, 26–30 May 2015, pp. 4874–4881 (2015)Google Scholar
  17. 17.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981). http://doi.acm.org/10.1145/358669.358692MathSciNetCrossRefGoogle Scholar
  18. 18.
    Benosman, R., Clercq, C., Lagorce, X., Ieng, S., Bartolozzi, C.: Event-based visual flow. IEEE Trans. Neural Netw. Learn. Syst. 25(2), 407–417 (2014)CrossRefGoogle Scholar
  19. 19.
    Mueggler, E., Rebecq, H., Gallego, G., Delbrück, T., Scaramuzza, D.: The event-camera dataset and simulator: event-based data for pose estimation, visual odometry, and SLAM. I. J. Robot. Res. 36(2), 142–149 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Sherif A. S. Mohamed
    • 1
    Email author
  • Mohammad-Hashem Haghbayan
    • 1
  • Jukka Heikkonen
    • 1
  • Hannu Tenhunen
    • 1
    • 2
  • Juha Plosila
    • 1
  1. 1.University of Turku (UTU)TurkuFinland
  2. 2.Royal Institute of Technology (KTH)StockholmSweden

Personalised recommendations