Neural Processing Letters

, Volume 42, Issue 1, pp 5–26 | Cite as

Multi-sensor Fusion Based on Asymmetric Decision Weighting for Robust Activity Recognition

  • Oresti Banos
  • Miguel Damas
  • Alberto Guillen
  • Luis-Javier Herrera
  • Hector Pomares
  • Ignacio Rojas
  • Claudia Villalonga


The recognition of human activity has been deeply explored during the recent years. However, most proposed solutions are mainly devised to operate in ideal conditions, thus not addressing crucial real-world issues. One of the most prominent challenges refers to common sensor technological anomalies. Sensor faults and failures introduce variations in the measured sensor data with respect to the equivalent observations in ideal conditions. As a consequence, predefined recognition systems may potentially fail to identify actions in the anomalous sensor data. This paper presents a novel model devised to cope with the effects introduced by sensor technological anomalies. The model builds on the knowledge gained from multi-sensor configurations, through asymmetrically weighting the decisions provided at both activity and sensor levels. Insertion and rejection weighting metrics are particularly used to eventually yield a unique recognized activity. For the sake of comparison, the tolerance to sensor faults and failures of standard activity recognition systems and the new proposed model are evaluated. The results prove classic activity-aware systems to be incapable of recognition under the effects of sensor technological anomalies, while the proposed model demonstrates to be robust against both sensor faults and failures.


Wearable sensors Sensor anomalies Sensor failures Sensor faults Decision fusion Weighted decision Activity recognition 



This work was partially supported by the HPC-Europa2 project (no. 228398), the Spanish CICYT Project SAF2010-20558, Junta de Andalucia Project P09-TIC-175476 and the FPU Spanish Grant AP2009-2244.


  1. 1.
    Adidas\(^{\textregistered }\) (2013) Adidas micoach.
  2. 2.
    Albert MV, Toledo S, Shapiro M, Kording K (2012) Using mobile phones for activity recognition in Parkinsons patients. Front Neurol 3:158Google Scholar
  3. 3.
    Altun K, Barshan B (2010) Human activity recognition using inertial/magnetic sensor units. In: Salah AA, Ruiz-del-Solar J, Çetin M, Oudeyer P-Y (eds) Human behavior understanding. Proceedings of the third international workshop, HBU 2012, Vilamoura, Portugal, October 7, 2012. Lecture notes in computer science, pp. 38–51. Springer, BerlinGoogle Scholar
  4. 4.
    Arlot S, Celisse A (2010) A survey of cross-validation procedures for model selection. Statist Surv 4:40–79CrossRefMathSciNetzbMATHGoogle Scholar
  5. 5.
    Atallah L, Lo B, King R, Yang GZ (2011) Sensor positioning for activity recognition using wearable accelerometers. IEEE Trans Biomed Circ Syst 5(4):320–329CrossRefGoogle Scholar
  6. 6.
    Avci A, Bosch S, Marin-Perianu M, Marin-Perianu R, Havinga P (2010) Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: a survey. In: 23rd international conference on architecture of computing systems, pp 1–10Google Scholar
  7. 7.
    Banos O, Damas M, Pomares H, Prieto A, Rojas I (2012) Daily living activity recognition based on statistical feature quality group selection. Expert Syst Appl 39(9):8013–8021CrossRefGoogle Scholar
  8. 8.
    Banos O, Damas M, Pomares H, Rojas F, Delgado-Marquez B, Valenzuela O (2013) Human activity recognition based on a sensor weighting hierarchical classifier. Soft Comput 17:333–343CrossRefGoogle Scholar
  9. 9.
    Banos O, Galvez JM, Damas M, Pomares H, Rojas I (2014) Window size impact in human activity recognition. Sensors 14(4):6474–6499CrossRefGoogle Scholar
  10. 10.
    Banos O, Toth MA, Damas M, Pomares H, Rojas I (2014) Dealing with the effects of sensor displacement in wearable activity recognition. Sensors 14(6):9995–10023Google Scholar
  11. 11.
    Bao L, Intille SS (2004) Activity recognition from user-annotated acceleration data. Perv Comput 23:1–17CrossRefGoogle Scholar
  12. 12.
    Bouten C, Koekkoek K, Verduin M, Kodde R, Janssen J (1997) A triaxial accelerometer and portable data processing unit for the assessment of daily physical activity. IEEE Trans Biomed Eng 44(3):136–147CrossRefzbMATHGoogle Scholar
  13. 13.
    Breiman L, Spector P (1992) Submodel selection and evaluation in regression. the x-random case. Int Statist Rev 60(3):291–319CrossRefGoogle Scholar
  14. 14.
    Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theor 13(1):21–27CrossRefzbMATHGoogle Scholar
  15. 15.
    Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines: and other kernel-based learning methods. Cambridge University Press, New YorkCrossRefGoogle Scholar
  16. 16.
    Dietterich TG (2000) Ensemble methods in machine learning. In: Proceedings of the first international workshop on multiple classifier systems, pp 1–15. Springer, LondonGoogle Scholar
  17. 17.
    Duda RO, Hart PE, Stork DG (2000) Pattern classification, 2nd edn. Wiley-Interscience, New YorkGoogle Scholar
  18. 18.
    Fitbit\(^{\textregistered }\) (2013) Fitbit products.
  19. 19.
    He Z, Jin L (2009) Activity recognition from acceleration data based on discrete consine transform and svm. In: IEEE international conference on systems, man and cybernetics, pp 5041–5044Google Scholar
  20. 20.
    Hidalgo\(^{\textregistered }\) (2012) Equivital eq02.
  21. 21.
    Huynh T, Blanke U, Schiele B (2007) Scalable recognition of daily activities with wearable sensors. In: Location-and context-awareness. Springer, Berlin, pp 50–67Google Scholar
  22. 22.
    Jawbone\(^{\textregistered }\) (2013) Jawbone up.
  23. 23.
    Jiang M, Shang H, Wang Z, Li H, Wang Y (2011) A method to deal with installation errors of wearable accelerometers for human activity recognition. Physiol Measure 32(3):347CrossRefGoogle Scholar
  24. 24.
    Kailas A (2012) Capturing basic movements for mobile platforms embedded with motion sensors. In: International conference of the IEEE engineering in medicine and biology society, pp 2480–2483Google Scholar
  25. 25.
    Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the 14th international joint conference on artificial intelligence, pp 1137–1143. San Francisco, CA, USAGoogle Scholar
  26. 26.
    Kusserow M, Amft O, Gubelmann H, Troester G (2010) Arousal pattern analysis of an olympic champion in ski jumping. Sports Technol 3(3):192–203Google Scholar
  27. 27.
    Kwapisz JR, Weiss GM, Moore SA (2011) Activity recognition using cell phone accelerometers. ACM SigKDD Explor 12(2):74–82Google Scholar
  28. 28.
    Lara O, Labrador M (2012) A survey on human activity recognition using wearable sensors. IEEE Commun Surv Tutor 99:1–18Google Scholar
  29. 29.
    Lester J, Choudhury T, Kern N, Borriello G, Hannaford B (2005) A hybrid discriminative/generative approach for modeling human activities. In: Proceedings of the 19th international joint conference on artificial intelligence, pp 766–772. San Francisco, CA, USAGoogle Scholar
  30. 30.
    Mannini A, Intille SS, Rosenberger M, Sabatini AM, Haskell W (2013) Activity recognition using a single accelerometer placed at the wrist or ankle. Med Sci Sports Exer 45(11):2193–2203CrossRefzbMATHGoogle Scholar
  31. 31.
    Mäntyjärvi J, Himberg J, Seppanen T (2001) Recognizing human motion with multiple acceleration sensors. In: Proceedings of the international IEEE conference on systems, man and cybernetics, pp 747–752Google Scholar
  32. 32.
    Mathie MJ, Coster ACF, Lovell NH, Celler BG (2004) Accelerometry: providing an integrated, practical method for long-term, ambulatory monitoring of human movement. Physiol Measure 25(2):1–20CrossRefGoogle Scholar
  33. 33.
    Maurer U, Smailagic A, Siewiorek D, Deisher M (2006) Activity recognition and monitoring using multiple sensors on different body positions. In: International workshop on wearable and implantable body sensor networks, pp 113–116Google Scholar
  34. 34.
    Maurtua I, Kirisci PT, Stiefmeier T, Sbodio ML, Witt H (2007) A wearable computing prototype for supporting training activities in automative production. In: 4th international forum on applied wearable computingGoogle Scholar
  35. 35.
    Najafi B, Aminian K, Paraschiv-Ionescu A, Loew F, Bula CJ, Robert P (2003) Ambulatory system for human motion analysis using a kinematic sensor: monitoring of daily physical activity in the elderly. IEEE Trans Biomed Eng 50(6):711–723CrossRefGoogle Scholar
  36. 36.
    Nike\(^{\textregistered }\) (2013) Nike \(+\) running.
  37. 37.
    Nike\(^{\textregistered }\) (2013) Nike \(+\) sportwatch.
  38. 38.
    Parkka J, Ermes M, Korpipaa P, Mantyjarvi J, Peltola J, Korhonen I (2006) Activity classification using realistic data from wearable sensors. IEEE Trans Inf Technol Biomed 10(1):119–128CrossRefGoogle Scholar
  39. 39.
    Pirttikangas S, Fujinami K, Seppanen T (2006) Feature selection and activity recognition from wearable sensors. In: Third international symposium ubiquitous computing systems. LNCS vol 4239, pp 516–527. Springer, BerlinGoogle Scholar
  40. 40.
    Preece SJ, Goulermas JY, Kenney LPJ, Howard D, Meijer K, Crompton R (2009) Activity identification using body-mounted sensors—a review of classification techniques. Physiol Measure 30(4):1–33CrossRefGoogle Scholar
  41. 41.
    Ravi N, Mysore P, Littman ML (2005) Activity recognition from accelerometer data. In: Proceedings of the seventeenth conference on innovative applications of artificial intelligence, pp 1541–1546Google Scholar
  42. 42.
    Roggen D, Calatroni A, Rossi M, Holleczek T, Förster K, Tröster G, Lukowicz P, Bannach D, Pirkl G, Ferscha A, Doppler J, Holzmann C, Kurz M, Holl G, Chavarriaga R, Creatura M, del Millán JR (2010) Collecting complex activity data sets in highly rich networked sensor environments. In: 7th international conference on networked sensing systems, pp 233–240Google Scholar
  43. 43.
    Roggen D, Magnenat S, Waibel M, Tröster G (2011) Wearable computing: designing and sharing activity-recognition systems across platforms. IEEE Robot Automat Mag 18(2):83–95Google Scholar
  44. 44.
    Sagha H, Bayati H, del Millan JR, Chavarriaga R (2013) On-line anomaly detection and resilience in classifier ensembles. Pattern Recognit Lett 34(15):1916–1927Google Scholar
  45. 45.
    Sazonov E, Fulk G, Sazonova N, Schuckers S (2009) Automatic recognition of postures and activities in stroke patients. In: International conference of the IEEE engineering in medicine and biology society, pp 2200–2203Google Scholar
  46. 46.
    Selles R, Formanoy M, Bussmann J, Janssens P, Stam H (2005) Automated estimation of initial and terminal contact timing using accelerometers; development and validation in transtibial amputees and controls. IEEE Trans Neural Syst Rehab Eng 13(1):81–88CrossRefGoogle Scholar
  47. 47.
    Stiefmeier T, Roggen D, Ogris G, Lukowicz P, Tröster G (2008) Wearable activity tracking in car manufacturing. IEEE Perv Comput Mag 7(2):42–50CrossRefGoogle Scholar
  48. 48.
    Stone M (1977) Asymptotics for and against cross-validation. Biometrika 64(1):29–35Google Scholar
  49. 49.
    Sun L, Zhang D, Li B, Guo B, Li S (2010) Activity recognition on an accelerometer embedded mobile phone with varying positions and orientations. In: Yu Z, Liscano R, Chen G, Zhang D, Zhou X (eds) Ubiquitous intelligence and computing. Proceedings of the 7th international conference, UIC 2010, Xi’an, China, October 2010. LNCS pp. 548–562. Springer, BerlinGoogle Scholar
  50. 50.
    Theodoridis S, Koutroumbas K (2008) Pattern recognition, 4th edn. Academic, San DiegoGoogle Scholar
  51. 51.
    UnderArmour\(^{\textregistered }\) (2014) Armour39.
  52. 52.
    Ward JA, Lukowicz P, Gellersen HW (2011) Performance metrics for activity recognition. ACM Trans Intell Syst Technol 2(1):6:1–6:23Google Scholar
  53. 53.
    Zwartjes D, Heida T, van Vugt J, Geelen J, Veltink P (2010) Ambulatory monitoring of activities and motor symptoms in Parkinson’s disease. IEEE Trans Biomed Eng 57(11):2778–2786CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Oresti Banos
    • 1
  • Miguel Damas
    • 1
  • Alberto Guillen
    • 1
  • Luis-Javier Herrera
    • 1
  • Hector Pomares
    • 1
  • Ignacio Rojas
    • 1
  • Claudia Villalonga
    • 1
  1. 1.Department of Computer Architecture and Computer TechnologyResearch Center for Information and Communication Technologies of the University of Granada (CITIC-UGR)GranadaSpain

Personalised recommendations