Skip to main content
SpringerLink
Log in

Designing an integrated driver assistance system using image sensors

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Road accidents cause a great loss to human lives and assets. Most of the accidents occur due to human errors, such as bad awareness, distraction, drowsiness, low training, and fatigue. Advanced driver assistance system (ADAS) can reduce the human errors by keeping an eye on the driving environment and warning a driver to the upcoming danger. However, these systems come only with modern luxury cars because of their high cost and complexity due to several sensors employed. Therefore, camera-based ADAS are becoming an option due to their lower cost, higher availability, numerous applications and ability to combine with other systems. Targeting at designing a camera-based ADAS, we have conducted an ethnographic study of drivers to know what information about the driving environment would be useful in preventing accidents. It turned out that information on speed, distance, relative position, direction, and size and type of the nearby objects would be useful and enough for implementing most of the ADAS functions. Several camera-based techniques are available for capturing the required information. We propose a novel design of an integrated camera-based ADAS that puts technologies—such as five ordinary CMOS image sensors, a digital image processor, and a thin display—into a smart system to offer a dozen advanced driver assistance functions. A basic prototype is also implemented using MATLAB. Our design and the prototype testify that all the required technologies are now available for implementing a full-fledged camera-based ADAS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abuelela, M., Olariu, S., & Weigle, M. C. (2008). NOTICE: An architecture for the notification of traffic incidents. In Vehicular technology conference, 2008. VTC Spring 2008. IEEE (pp. 3001–3005).

  • Adell E., Varhelyi A., Alonso M., Plaza J. (2008) Developing human-machine interaction components for a driver assistance system for safe speed and safe distance. IET Intelligent Transport Systems 2(1): 1–14

    Article  Google Scholar 

  • Akhlaq, M. (2010). A smart-dashboard: Augmenting safe & smooth driving. Master thesis no. 2010:MUC:01, School of Computing, Blekinge Institute of Technology (BTH), Ronneby, Sweden.

  • Albu, A. B., Widsten, B., Wang, T., Lan, J., & Mah, J. (2008). A computer vision-based system for real-time detection of sleep onset in fatigued drivers. Intelligent vehicles symposium, 2008 IEEE (pp. 25–30).

  • Andrade, L. C. G., Campos, M. F. M., & Carceroni, R. L. (2004). A video-based support system for nighttime navigation in semi-structured environments. In Computer graphics and image processing, XVII Brazilian symposium on (SIBGRAPI’04) (pp. 178–185).

  • Baldauf M., Dustdar S., Rosenberg F. (2007) A Survey on Context-Aware Systems. International Journal of Ad Hoc and Ubiquitous Computing 2(4): 263–277

    Article  Google Scholar 

  • Bergasa L. M., Nuevo J., Sotelo M. A., Barea R., Lopez M. E. (2006) Real-time system for monitoring driver vigilance. Intelligent Transportation Systems, IEEE Transactions on 7(1): 63–77

    Article  Google Scholar 

  • Bergmeier, U. (2008). Augmented reality in vehicle0—technical realisation of a contact analogue Head-up Display under automotive capable aspects; usefulness exemplified through Night Vision Systems. In Proceedings of 32. World Automotive Congress, 2008.

  • Bertozzi M., Broggi A., Fascioli A. (1998) Stereo inverse perspective mapping: Theory and applications. Image and Vision Computing 8(16): 585–590

    Article  Google Scholar 

  • Brauckmann, M. E., Goerick, C., Groß, J., & Zielke, T. (1994). Towards all around automatic visual obstacles sensing for cars. In Proceedinhs of intelligent vehicle symposium (pp. 79–84).

  • Cao X. B., Qiao H., Keane J. (2008) A low-cost pedestrian-detection system with a single optical camera. Intelligent Transportation Systems, IEEE Transactions on 9(1): 58–67

    Article  Google Scholar 

  • Cheng, H., Liu, Z., Zheng, N., & Yang, J. (2007). Enhancing a Driver’s situation awareness using a global view map. In Multimedia and expo, 2007 IEEE international conference on (pp. 1019–1022).

  • Computer Visio System Toolbox. (2011). Video and image processing blockset 2.8. http://www.mathworks.com/products/viprocessing/demos.html. Accessed 22 June 2011.

  • David M. (2006) ESC gives life to smart fabric, smart dashboard. Electronic Design, 54(9): 19–19

    Google Scholar 

  • Delot T., Ilarri S., Cenerario N., Hien T. (2011) Event sharing in vehicular networks using geographic vectors and maps. Mobile Information Systems 7(1): 21–44

    Google Scholar 

  • Devi, M. S., & Bajaj, P. R. (2008). Driver fatigue detection based on eye tracking. Emerging trends in engineering and technology, 2008. In ICETET ’08. First international conference on (pp. 649–652).

  • Ehlgen, T., & Pajdla, T. (2007). Monitoring surrounding areas of truck-trailer combinations. In: Proceedings of the 5th international conference on computer vision systems, Bielefeld, Germany.

  • Ehlgen, T., Thorn, M., & Glaser, M. (2007). Omnidirectional cameras as backing-up aid. Computer vision, 2007. ICCV 2007. In IEEE 11th International Conference on (pp. 1–5).

  • Eriksson, J., Girod, L., Hull, B., Newton, R., Madden, S., & Balakrishnan, H. (2008). The pothole patrol: Using a mobile sensor network for road surface monitoring. In Proceedings of the 6th international conference on mobile systems, applications, and services (Breckenridge, CO, USA, June 17–20, 2008). MobiSys ’08 (pp. 29–39). ACM, New York, NY.

  • Finnefrock, M., Jiang, X., & Motai, Y. (2005). Visual-based assistance for electric vehicle driving. Intelligent vehicles symposium, 2005. Proceedings. IEEE (pp. 656–661).

  • Fletcher, L., Petersson, L., & Zelinsky, A. (2003). Driver assistance systems based on vision in and out of vehicles. In Proceedings of IEEE intelligent vehicles symposium (pp. 322–327).

  • Franke, U., & Kutzbach, I. (1996). Fast stereo based object detection for stop and go traffic. In Proceedings of intelligent vehicles symposium (pp. 339–344).

  • Fritsch, J. , Michalke, T., Gepperth, A., Bone, S., Waibel, F., Kleinehagenbrock, M., et al. (2008). Towards a human-like vision system for driver assistance. In IEEE intelligent vehicles symposium. Eindhoven University of Technology, Eindhoven, The Netherlands.

  • Future Launches by Mobileye. (2010). http://www.mobileye.com/manufacturer-products/product-launches/future. Accessed 22 June 2011.

  • Giachetti A., Campani M., Torre V. (1998) The use of optical flow for road navigation. IEEE Transactions on Robotics and Automation 14(1): 34–48

    Article  Google Scholar 

  • Handmann U., Kalinke T., Tzomakas C., Werner M., Seelen W. V. (2000) An image processing system for driver assistance. Image and Vision Computing 18(5): 367–376

    Article  Google Scholar 

  • Hertel, D., Betts, A., Hicks, R., & ten Brinke, M., (2008). An adaptive multiple-reset CMOS wide dynamic range imager for automotive vision applications. Intelligent vehicles symposium, 2008 IEEE (pp. 614–619).

  • Hoffmann, C., Dang, T., & Stiller, C. (2004). Vehicle detection fusing 2D visual features. IEEE intelligent vehicles symposium (pp. 280–285).

  • Hull, B., Bychkovsky, V., Zhang, Y., Chen, K., Goraczko, M., Shih, E., et al. (2006). CarTel: A distributed mobile sensor computing system. In Proceedings of the 4th ACM SenSys.

  • Ichihara, E., & Ohta, Y., (2000). NaviView: Visual assistance using roadside cameras—evaluation of virtual views. In Proceedings of intelligent transportation system conference (pp. 322–327).

  • JISC Digital Media. (2011). Digital cameras. 18 May 2009. http://www.jiscdigitalmedia.ac.uk/stillimages/advice/digital-cameras/. Accessed 22 June 2011.

  • Johnson, R. C. (2006). CMOS imager chips enable true machine vision. Dec. 19, 2006. http://www.eetasia.com/ART_8800446742_480500_NP_89a21a9f.HTM. Accessed 22 June 2011.

  • Jonefjäll, M. (2009). Visual assistance HMI for use of video camera applications in the car. ISSN 1402-1617/ISRN LTU-EX-09/003-SE / NR 2009:003, Master’s thesis, Luleå University of Technology, Sweden.

  • Kanevsky D. (2008) Telematics: Artificial passenger and beyond. Human factors and voice interactive systems. Springer, New York, pp 291–325

    Book  Google Scholar 

  • Kim S., Kang J., Oh S., Ryu Y., Kim K., Park S. et al (2008) An intelligent and integrated driver assistance system for increased safety and convenience based on all-around sensing. Journal of Intelligent and Robotics System 51(3): 261–287

    Article  Google Scholar 

  • Kircher, A., Uddman, M., & Sandin, J. (2002). Vehicle control and drowsiness. Swedish National Road and Transport Research Institute, Linkoping, Sweden, Tech. Rep. VTI-922A.

  • Knipling R. (2000) Changing driver behavior with on-board safety monitoring. ITS Quarterly VIII(2): 27–35

    Google Scholar 

  • Kumar, M., & Kim, T. (2005). Dynamic speedometer: Dashboard redesign to discourage drivers from speeding. In CHI ’05 extended abstracts on human factors in computing systems (Portland, OR, USA, April 02–07, 2005). CHI ’05 (pp. 1573–1576). ACM, New York, NY.

  • Li, Y., Yin, L., Jia, Y., & Wang, M. (2008). Vehicle speed measurement based on video images. In 3rd international conference on innovative computing information and control, ICICIC (p. 439).

  • Lim, K. H., Ang, L. M., Seng, K. P., & Chin, S. W. (2009). Lane-vehicle detection and tracking. In Proceedings of the international multiconference of engineers and computer scientists 2009 Vol II IMECS 2009, Hong Kong.

  • Lin H., Li K., Chang C. (2008) Vehicle speed detection from a single motion blurred image. Image and Vision Computing 26(10): 1327–1337

    Article  Google Scholar 

  • Liu, J., Su, Y., Ko, M., & Yu, P. (2008a). Development of a vision-based driver assistance system with lane departure warning and forward collision warning functions. In Proceedings of the 2008 digital image computing: Techniques and applications (December 01–03, 2008). DICTA (pp. 480–485). Washington, DC: IEEE Computer Society.

  • Liu, Y. C., Lin, K. Y., & Chen, Y. S. (2008b). Bird’s-eye view vision system for vehicle surrounding monitoring. In Proceedings of the conferene on robot vision (pp. 207–218). Berlin, Germany, Feb. 20, 2008.

  • Lotan, T., & Toledo, T. (2005). Evaluating the safety implications and benefits of an in-vehicle data recorder to young drivers. In Proceedings of the third international driving symposium on human factors in driver assessment, training and vehicle design. Maine, USA, June 27–30, 2005.

  • Martinez, E., Diaz, M., Melenchon, J., Montero, J. A., Iriondo, I., & Socoro, J. C. (2008). Driving assistance system based on the detection of head-on collisions. In Intelligent vehicles symposium, 2008 IEEE (pp. 913–918). 4–6 June, 2008.

  • McCall J. C., Trivedi M. M. (2006) Video-based lane estimation and tracking for driver assistance: Survey, system, and evaluation. IEEE Transactions on ITS 7(1): 20–37

    Google Scholar 

  • Mohan, P., Padmanabhan, V. N., & Ramjee, R. (2008). Trafficsense: Rich monitoring of road and traffic conditions using mobile smartphones. In Proceedings of the conference on embedded networked Sensor Systems (SenSys) (pp. 323–336).

  • Nakamura, Y. (2008). JAMA Guideline for In-Vehicle Display Systems. HMI Experts Group, JAMA, (Honda R&D Co., Ltd.). Document No: 2008-21-0003, October 2008. http://www.sae.org/technical/papers/2008-21-0003. Accessed 22 June 2011.

  • Nashashibi F., Khammari A., Laurgeau C. (2008) Vehicle recognition and tracking using a generic multisensor and multialgorithm fusion approach. International Journal of Vehicle Autonomous Systems 6(1–2): 134–154

    Article  Google Scholar 

  • Nedevschi, S., Danescu, R., Marita, T., Oniga, F., Pocol, C., Sobol, S., et al. (2007). A sensor for urban driving assistance systems based on dense stereovision. In Intelligent vehicles symposium, 2007 IEEE (pp. 276–283). 13–15 June, 2007.

  • Nookala, M., & Estochen, B. (2002). Minnesota, USA intelligent vehicle initiative. In Intelligent vehicle symposium, 2002. IEEE (vol. 2, pp. 533–536). 17–21 June 2002.

  • Oki, Y., Yamada, F., Seki, Y., Mizutani, H., & Makino,H. (2003). Actual road verification of AHS support system for prevention of vehicle overshooting on curves. In Proceedings of the 2nd ITS Symposium 2003 (pp. 247–252).

  • Peden, M., Scurfield, R., et al., eds. (2004). World report on road traffic injury prevention. World Health Organization, Geneva. http://www.who.int/violence_injury_prevention/publications/road_traffic/world_report/en/index.html. Accessed 22 June 2011.

  • Perrillo, K. V. (1997). Effectiveness of speed trailer on low-speed urban roadway. Master Thesis, Texas A&M University, College Station, TX.

  • Piccioli G., De Micheli E., Parodi P., Campani M. (1996) Robust method for road sign detection and recognition. Image and Vision Computing 14(3): 209–223

    Article  Google Scholar 

  • Porcino, D. (2001). Location of third generation mobile devices: A comparison between terrestrial and satellite positioning systems. In Vehicular technology conference, 200, VTC 2001 Spring. IEEE VTS 53rd (Vol. 4, pp. 2970–2974).

  • Pugh, S. (2008). The Train Operator’s view. The Institution of Engineering and Technology Seminar on User Worked Crossings, Birmingham (pp. 103–112). 30–30 Jan. 2008.

  • Qadri N. N., Altaf M., Fleury M., Ghanbari M. (2010) Robust video communication over an urban VANET. Mobile Information Systems 6(3): 259–280

    Google Scholar 

  • Raghavan, S., Yang, C. Z., Jovanis, P. P., Kitamura, R., Owens, G., & Anwar, M. (1992). California Advanced Driver Information System (CADIS). Final Report. Institute of Transportation Studies, University of California, Davis, Research Report UCD-ITS-RR-92-20, December, 1992, pp. 214.

  • Rakotonirainy, A., Feller, F., & Narelle, L. (2008). Using in-vehicle avatars to prevent road violence. Centre for Accident Research and Road Safety (CARRS-Q). http://pervasive2008.org/Papers/LBR/lbr17.pdf. Accessed 22 June 2011.

  • Reding, V., (2006). The intelligent car initiative: Raising awareness of ICT for smarter, safer and cleaner vehicle. Speech delivered at the Intelligent Car Launching Event, vol. 23, Brussels, Belgium. http://eur-lex.europa.eu/smartapi/cgi/sga_doc?smartapi!celexplus!prod!DocNumber&lg=en&type_doc=COMfinal&an_doc=2006&nu_doc=59. Accessed 22 June 2011.

  • Ross, M., Francis, M., Jill, H., Mark, F., Michael, S., Tracey, R., et al. (2008). Co-Driver Alert project. Road Transport Information and Control—RTIC 2008 and ITS United Kingdom Members’ Conference, IET , vol., no., pp. 1–6, 20–22 May 2008.

  • Santos, J., Merat, N., Mouta, S., Brookhuis, K., & De Waard, D. (2005) The interaction between driving and in-vehicle information systems: Comparison of results from laboratory, simulator and real-world studies. Transportation Research Part F: Traffic Psychology and Behaviour, 8(2), 135–146

    Google Scholar 

  • Sato, Y., & Makanae, K., (2006). Development and evaluation of in-vehicle signing system utilizing RFID tags as Digital Traffic signs. International Journal of ITS Research, Vol. 4, No.1, December 2006.

  • Shibata, M., Makino, T., & Ito, M. (2008). Target distance measurement based on camera moving direction estimated with optical flow. In 10th IEEE international workshop on advanced motion control, AMC ’08, Trento (pp. 62–67). 26–28 March 2008.

  • Smith, P., Shah, M., & Lobo, N. V. (2003). Determining driver visual attention with one camera. IEEE Transactions On Intelligent Transportation Systems, Vol. 4, No. 4, December 2003.

  • Spaho E., Barolli L., Mino G., Xhafa F., Kolici V., Miho R. (2010) Implementation of CAVENET and its usage for performance evaluation of AODV, OLSR and DYMO protocols in vehicular networks. Mobile Information Systems 6(3): 213–227

    Google Scholar 

  • Spelt, P. F., Tufano, D. R., & Knee, H. E. (1997). Development and evaluation of an in-vehicle information system. In Proceedings of the seventh annual meeting and exposition of the intelligent transportation society of America Washington, D.C. June 2–5, 1997.

  • Stein, G. P., Mano, O., & Shashua, A. (2003). Vision-based ACC with a single camera: Bounds on range and range rate accuracy. In Intelligent vehicles symposium, 2003. Proceedings. IEEE, (pp. 120–125). 9–11 June 2003.

  • Su, C. Y., & Fan, G. H. (2008). An effective and fast lane detection algorithm. Lecture Notes in Computer Science (Vol. 5359/2008, pp. 942–948). Berlin/Heidelberg: Springer.

  • Torralba A. (2003) Contextual priming for object detection. International Journal of Computer Vision 53(2): 169–191

    Article  Google Scholar 

  • Toyota, K., Fuji, T., Kimoto, T., & Tanimoto, M. (2000). A proposal of HIR (Human-Oriented Image Restructuring) Systems for ITS. In Proceedings of the intelligent vehicle symposium (pp. 540– 544).

  • Treat, J. R., Tumbas, N. S., McDonald, S. T., Shinar, D., Hume, R. D., Mayer, R. E., et al. (1977). Tri-level study of the causes of traffic accidents. Report No. DOT-HS-034-3-535-77, Indiana University.

  • Trivedi M. M., Gandhi T., McCall J. (2007) Looking-in and looking-out of a vehicle: Computer-vision-based enhanced vehicle safety. Intelligent Transportation Systems. IEEE Transactions on 8(1): 108–120

    Article  Google Scholar 

  • US Department of Transportation (2004). Volume I: Guidelines,. In-Vehicle Display Icons and Other Information Elements. Publication NO. FHWA-RD-03-065 September 2004. http://www.tfhrc.gov/safety/pubs/03065/03065.pdf. Accessed 22 June 2011.

  • US Department of Transportation. (1996). Development of Human Factors Guidelines for Advanced Traveler Information Systems and Commercial Vehicle Operations: Comparable Systems Analysis. Publication Number: FHWA-RD-95-197, December 1996, http://www.fhwa.dot.gov/tfhrc/safety/pubs/95197/. Accessed 22 June 2011.

  • Vashitz G., Shinar D., Blum Y. (2008) In-vehicle information systems to improve traffic safety in road tunnels. Transportation Research Part F: Traffic Psychology and Behaviour 11(1): 61–74

    Article  Google Scholar 

  • Walker, J. G., Barnes, N., & Anstey, K. (2006). Sign detection and driving competency for older drivers with impaired vision. In B. MacDonald (Ed.), Proceedings of the 2006 Australasian conference on robotics & automation Auckland, New Zealand, December 6–8, 2006.

  • Wang C.-C., Chen C.-J., Chan Y.-M., Fu L.-C., Hsiao P.-Y. (2006) Lane detection and vehicle recognition for driver assistance system at daytime and nighttime. Image and Recognition Magazine 12(2): 4–17

    Google Scholar 

  • Wilcox, D. (1999). Mercedes moves into mechatronic overdrive. Transport Engineer, pp. 21–30. Aug. 1999.

  • Woll, J. D. (1995). VORAD collision warning radar. In Radar conference, 1995., Record of the IEEE 1995 International (pp. 369–372). 8–11 May 1995.

  • Xuan K., Zhao G., Taniar D., Safar M., Srinivasan B. (2011a) Voronoi-based multi-level range search in mobile navigation. Multimedia Tools Applications 53(2): 459–479

    Article  Google Scholar 

  • Xuan K., Zhao G., Taniar D., Safar M., Srinivasan B. (2011b) Constrained range search query processing on road networks. Concurrency and Computation: Practice and Experience 23(5): 491–504

    Article  Google Scholar 

  • Xuan, K., Zhao, G., Taniar, D., & Srinivasan, B. (2008a). Continuous range search query processing in mobile navigation. In Proceedings of the 14th international conference on parallel and distributed systems (ICPADS 2008), IEEE (pp. 361–368).

  • Xuan K., Zhao G., Taniar D., Srinivasan B. (2008b) Incremental k-nearest-neighbor search on road networks. Journal of Interconnection Networks 9(4): 455–470

    Article  Google Scholar 

  • Yoo, S., Chong, P. K., Park, T., Kim, Y., Kim, D., Shin, C., et al. (2008). DGS: Driving guidance system based on wireless sensor network. In Advanced information networking and applications—Workshops, 2008. AINAW 2008. 22nd International conference on (pp. 628–633), 25–28 March 2008.

  • Zhao G., Xuan K., Rahayu W., Taniar D., Safar M., Gavrilova M., et al. (2010). Voronoi-based continuous k nearest neighbor search in mobile navigation. IEEE Transactions on Industrial Electronics 58(6), 2247–2257

    Google Scholar 

  • Zhu, Y., Comanicciu, D., Ramesh, V., et al. (2005). An integrated framework of vision-based vehicle detection with knowledge fusion. In Proceedings of the intelligent vehicle Symposium (pp. 199–204).

  • Zin T. T., Koh S. S., Hama H. (2007) Robust signboard recognition for vision-based navigation. The Journal of The Institute of Image Information and Television Engineers 61(8): 1192–1200

    Article  Google Scholar 

  • Zoratti, P. (2011). FPGAS provide platform for driver assistance. Electronic Product Design Magazine. 28 April 2011. http://www.epdonthenet.net/article/41913/FPGAS-provide-platform-for-driver-assistance.aspx. Accessed 22 June 2011.

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Engineering, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia

    M. Akhlaq

  2. Department of Computer Engineering, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia

    Tarek R. Sheltami

  3. School of Computing, Blekinge Institute of Technology, Box 520, 372 25, Ronneby, Sweden

    Bo Helgeson

  4. Jodrey School of Computer Science, Acadia University, Wolfville, NS, B4P 2R6, Canada

    Elhadi M. Shakshuki

Authors
  1. M. Akhlaq
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tarek R. Sheltami
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bo Helgeson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elhadi M. Shakshuki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elhadi M. Shakshuki.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Akhlaq, M., Sheltami, T.R., Helgeson, B. et al. Designing an integrated driver assistance system using image sensors. J Intell Manuf 23, 2109–2132 (2012). https://doi.org/10.1007/s10845-011-0618-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-011-0618-1

Keywords

Search

Navigation