Skip to main content

Using off-the-shelf data-human interface platforms: traps and tricks

Multimedia Tools and Applications volume 80, pages 12907–12929 (2021)Cite this article

Abstract

With the development of learning algorithms, the constantly increasing computing power and the available amount of multimedia data, the adoption rate of data science techniques is steadily growing. Machine and deep learning algorithms are already used in a wide variety of ways to solve domain-specific problems. However, the potential of such methodologies will be fulfilled when also non specialized data scientists will be empowered with their use. Focusing on such perspective, this work does not deal with a classical data science problem, but instead exploits existing and available easy to use data-human interfaces. To this aim, we picked an exemplar scenario, amounting to an existing qualitative activity recognition data set that was in the past analyzed utilizing feature selection techniques and custom machine learning paradigms. We here verify how it is today possible, without changing the default settings and/or performing any type of feature selection, to employ the machine and deep learning algorithms provided by different publicly accessible tools (namely, Weka, Orange, Ludwig and KNIME) to address the same problem. Nevertheless, not all of the utilized platforms and algorithms provided satisfactorily results: we here finally discuss the possible issues and opportunities posed by such approach.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M et al (2016) Tensorflow: A system for large-scale machine learning. In: 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16), pp 265–283

  2. Alsheikh M A, Selim A, Niyato D, Doyle L, Lin S, Tan HP (2016) Deep activity recognition models with triaxial accelerometers. In: Workshops at the Thirtieth AAAI Conference on Artificial Intelligence

  3. Angeli A, Riedel N, Marfia G (2019) Data science models. http://shorturl.at/asxF0

  4. Athey S (2018) The impact of machine learning on economics In: The economics of artificial intelligence: An agenda, University of Chicago Press

  5. Bao L, Intille SS (2004) Activity recognition from user-annotated acceleration data. In: International conference on pervasive computing. Springer, pp 1–17

  6. Bayat A, Pomplun M, Tran D A (2014) A study on human activity recognition using accelerometer data from smartphones. Procedia Comput Sci 34:450–457

    Article  Google Scholar 

  7. Bohanec M, MK Borštnar, Robnik-Šikonja M (2017) Explaining machine learning models in sales predictions. Expert Syst Appl 71:416–428

    Article  Google Scholar 

  8. Bujari A, Licar B, Palazzi C E (2011) Road crossing recognition through smartphone’s accelerometer. In: 2011 IFIP Wireless Days (WD). IEEE, pp 1–3

  9. Bujari A, Licar B, Palazzi C E (2012) Movement pattern recognition through smartphone’s accelerometer. In: 2012 IEEE Consumer Communications and Networking Conference (CCNC) IEEE, pp 502–506

  10. Buscher G, Dumais ST, Cutrell E (2010) The good, the bad, and the random: an eye-tracking study of ad quality in web search. In: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 42–49

  11. Chen Y, Xue Y (2015) A deep learning approach to human activity recognition based on single accelerometer. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics. IEEE, pp 1488–1492

  12. Chen D, Bellamy RK, Malkin PK, Erickson T (2016) Diagnostic visualization for non-expert machine learning practitioners: A design study. In: 2016 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, pp 87–95

  13. Chen M, Hao Y, Hwang K, Wang L, Wang L (2017) Disease prediction by machine learning over big data from healthcare communities. IEEE Access 5:8869–8879

    Article  Google Scholar 

  14. Cook D, Feuz K D, Krishnan N C (2013) Transfer learning for activity recognition: a survey. Knowl Inform Syst 36(3):537–556

    Article  Google Scholar 

  15. Crisci C, Ghattas B, Perera G (2012) A review of supervised machine learning algorithms and their applications to ecological data. Ecol Model 240:113–122

    Article  Google Scholar 

  16. Dems~ar J, Zupan B, Leban G, Curk T (2004) Orange: From experimental machine learning to interactive data mining. In: European Conference on Principles of Data Mining and Knowledge Discovery. Springer, pp 537–539

  17. Demšar J, Curk T, Erjavec A, Gorup Č, Hočevar T, Milutinovič M, Možina M, Polajnar M, Toplak M, Starič A et al (2013) Orange: data mining toolbox in python. J Mach Learn Res 14(1):2349–2353

    MATH  Google Scholar 

  18. Fatima M, Pasha M (2017) Survey of machine learning algorithms for disease diagnostic. J Intell Learn Syst Appl 9(01):1

    Google Scholar 

  19. Faust O, Hagiwara Y, Hong T J, Lih O S, Acharya U R (2018) Deep learning for healthcare applications based on physiological signals: a review. Comput Methods Programs Biomed 161:1–13

    Article  Google Scholar 

  20. Fillbrunn A, Dietz C, Pfeuffer J, Rahn R, Landrum G A, Berthold M R (2017) Knime for reproducible cross-domain analysis of life science data. J Biotechnol 261:149–156

    Article  Google Scholar 

  21. García M, Domínguez C, Heras J, Mata E, Pascual V (2018) An on-going framework for easily experimenting with deep learning models for bioimaging analysis. In: International Symposium on Distributed Computing and Artificial Intelligence. Springer, pp 330–333

  22. Guyon I, Chaabane I, Escalante HJ, Escalera S, Jajetic D, Lloyd JR, Macià N, Ray B, Romaszko L, Sebag M et al (2016) A brief review of the chalearn automl challenge: any-time any-dataset learning without human intervention. In: Workshop on Automatic Machine Learning, pp 21–30

  23. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten I H (2009) The weka data mining software: an update. ACM SIGKDD Explo Newslett 11(1):10–18

    Article  Google Scholar 

  24. Hammerla N Y, Fisher J, Andras P, Rochester L, Walker R, Plötz T (2015) Pd disease state assessment in naturalistic environments using deep learning. In: Twenty-Ninth AAAI Conference on Artificial Intelligence

  25. Heaton J, Polson N, Witte J H (2017) Deep learning for finance: deep portfolios. Appl Stoch Model Bus Ind 33(1):3–12

    MathSciNet  Article  Google Scholar 

  26. Holmes G, Donkin A, Witten IH (1994) Weka: A machine learning workbench. In: Proceedings of ANZIIS’94-Australian New Zealnd Intelligent Information Systems Conference. IEEE, pp 357–361

  27. Jordan M I, Mitchell T M (2015) Machine learning: trends, perspectives, and prospects. Science 349(6245):255–260

    MathSciNet  Article  Google Scholar 

  28. KNIME Open for Innovation (2006) https://www.knime.com

  29. Ketkar N (2017a) Introduction to keras. In: Deep Learning with Python. Springer, pp 97–111

  30. Ketkar N (2017b) Introduction to pytorch. In: Deep learning with python. Springer, pp 195–208

  31. Khan A, Mellor S, Berlin E, Thompson R, McNaney R, Olivier P, Plötz T (2015) Beyond activity recognition: skill assessment from accelerometer data In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, pp 1155–1166

  32. Keras (2015) https://www.keras.io

  33. Kranz M, MöLler A, Hammerla N, Diewald S, PlöTz T, Olivier P, Roalter L (2013) The mobile fitness coach: Towards individualized skill assessment using personalized mobile devices. Perv Mob Comput 9(2):203–215

    Article  Google Scholar 

  34. Kroes M, Kessels A G, Kalff A C, Feron F J, Vissers Y L, Jolles J, Vles J S (2002) Quality of movement as predictor of adhd: results from a prospective population study in 5-and 6-year-old children. Dev Med Child Neurol 44(11):753–760

    Article  Google Scholar 

  35. Kwapisz J R, Weiss G M, Moore S A (2011) Activity recognition using cell phone accelerometers. ACM SigKDD Explor Newslett 12(2):74–82

    Article  Google Scholar 

  36. Ladha C, Hammerla N Y, Olivier P, Plötz T (2013) Climbax: skill assessment for climbing enthusiasts. In: Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing ACM, pp 235–244

  37. Lane N D, Georgiev P, Qendro L (2015) Deepear: robust smartphone audio sensing in unconstrained acoustic environments using deep learning In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, pp 283–294

  38. Lara O D, Labrador M A (2013) A survey on human activity recognition using wearable sensors. IEEE Commun Surv Tutorials 15(3):1192–1209

    Article  Google Scholar 

  39. Lau SL, König I, David K, Parandian B, Carius-Düssel C, Schultz M (2010) Supporting patient monitoring using activity recognition with a smartphone. In: 2010 7th International Symposium on Wireless Communication Systems. IEEE, pp 810–814

  40. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  41. Ludwig Deep Learning (2019) https://uber.github.io/ludwig/

  42. Logan B, Healey J, Philipose M, Tapia EM, Intille S (2007) A long-term evaluation of sensing modalities for activity recognition. In: International conference on Ubiquitous computing. Springer, pp 483–500

  43. Malik F (2019) Neural networks: a solid practical guide. https://medium.com/fintechexplained/neural-networks-a-solid-practical-guide-9f343594b02a

  44. Marfia G, Roccetti M (2017) A practical computer based vision system for posture and movement sensing in occupational medicine. Multimed Tools Appl 76(6):8109–8129

    Article  Google Scholar 

  45. Maurtua I, Kirisci PT, Stiefmeier T, Sbodio ML, Witt H (2007) A wearable computing prototype for supporting training activities in automotive production. In: 4th International Forum on Applied Wearable Computing 2007. VDE, pp 1–12

  46. Molino P, Dudin Y, Miryala SS (2019) Ludwig Deep Learning. https://eng.uber.com/introducing-ludwig/https://eng.uber.com/introducing-ludwig/

  47. Naik A, Samant L (2016) Correlation review of classification algorithm using data mining tool: weka, rapidminer, tanagra, orange and knime. Procedia Comput Sci 85:662–668

    Article  Google Scholar 

  48. Orange (1996) https://orange.biolab.si/

  49. Ortiz Laguna J, Olaya A G, Borrajo D (2011) A dynamic sliding window approach for activity recognition. In: Konstan JA, Conejo R, Marzo J L, Oliver N (eds) User Modeling, Adaption and Personalization. Springer, Berlin, pp 219–230

  50. Parkka J, Ermes M, Korpipaa P, Mantyjarvi J, Peltola J, Korhonen I (2006) Activity classification using realistic data from wearable sensors. IEEE Trans Inf Technol Biomed 10(1):119–128

    Article  Google Scholar 

  51. Patel K (2010) Lowering the barrier to applying machine learning. In: Adjunct proceedings of the 23nd annual ACM symposium on User interface software and technology. ACM, pp 355–358

  52. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, et al. (2011) Scikit-learn: Machine learning in python. J Mach Learn Res 12(Oct):2825–2830

    MathSciNet  MATH  Google Scholar 

  53. Plötz T, Hammerla N Y, Olivier P L (2011) Feature learning for activity recognition in ubiquitous computing. In: Twenty-Second International Joint Conference on Artificial Intelligence

  54. Pourbabaee B, Roshtkhari M J, Khorasani K (2017) Deep convolutional neural networks and learning ecg features for screening paroxysmal atrial fibrillation patients. IEEE Trans Syst Man Cybern: Syst 48(12):2095–2104

    Article  Google Scholar 

  55. Python programming language (2006) https://www.python.org

  56. Ravi N, Dandekar N, Mysore P, Littman M L (2005) Activity recognition from accelerometer data. In: Aaai, vol 5, pp 1541–1546

  57. Riedel N, Angeli A, Marfia G (2019) Qualitative activity recognition using machine and deep learning: Experimenting with data-human interfaces for non data-scientists. In: Proceedings of the 5th EAI International Conference on Smart Objects and Technologies for Social Good. ACM, pp 7–12

  58. Ronao C A, Cho S B (2016) Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst Appl 59:235–244

    Article  Google Scholar 

  59. Rossum G (1995) Python reference manual

  60. Scikit-learn (2007) https://www.scikit-learn.org

  61. Stiefmeier T, Roggen D, Ogris G, Lukowicz P, Tröster G (2008) Wearable activity tracking in car manufacturing. IEEE Pervasive Computing 7(2):42–50

    Article  Google Scholar 

  62. Sung M, Marci C, Pentland A (2005) Wearable feedback systems for rehabilitation. J Neuroeng Rehab 2(1):17

    Article  Google Scholar 

  63. Tarca A L, Carey V J, Xw Chen, Romero R, Drȧghici S (2007) Machine learning and its applications to biology. PLoS Comput Biol 3(6):e116

    Article  Google Scholar 

  64. Tessendorf B, Gravenhorst F, Arnrich B, Tröster G (2011) An imu-based sensor network to continuously monitor rowing technique on the water. In: 2011 Seventh international conference on intelligent sensors, Sensor Networks and Information Processing. IEEE, pp 253–258

  65. Ugulino W, Velloso E, Fuks H (2019) Human activity recognition. http://groupware.les.inf.puc-rio.br/har#ixzz34dpS6oks

  66. Van der Aalst WM (2014) Data scientist: The engineer of the future. In: Enterprise interoperability VI Springer, pp 13–26

  67. Velloso E, Bulling A, Gellersen H, Ugulino W, Fuks H (2013) Qualitative activity recognition of weight lifting exercises. In: Proceedings of the 4th Augmented Human International Conference, pp 116–123

  68. Waller M A, Fawcett S E (2013) Data science, predictive analytics, and big data: a revolution that will transform supply chain design and management. J Bus Logist 34 (2):77–84

    Article  Google Scholar 

  69. Wang J, Chen Y, Hao S, Peng X, Hu L (2019) Deep learning for sensor-based activity recognition: a survey. Pattern Recogn Lett 119:3–11

    Article  Google Scholar 

  70. Weka (1993) https://www.cs.waikato.ac.nz/ml/weka/

  71. Yang Q, Suh J, Chen NC, Ramos G (2018) Grounding interactive machine learning tool design in how non-experts actually build models. In: Proceedings of the 2018 on Designing Interactive Systems Conference 2018. ACM, pp 573–584

  72. Zorrilla M, García-Saiz D (2013) A service oriented architecture to provide data mining services for non-expert data miners. Decis Support Syst 55(1):399–411

Download references

Acknowledgments

The authors gratefully thank the University of Bologna for the Alma Attrezzature 2017 grant and the Golinelli Foundation for the Data Science scholarship.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Bologna, Bologna, Italy

    Alessia Angeli

  2. Department for Life Quality Studies, University of Bologna, Bologna, Italy

    Gustavo Marfia

  3. University of Bielefeld, Bielefeld, Germany

    Norman Riedel

Authors
  1. Alessia Angeli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gustavo Marfia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Norman Riedel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gustavo Marfia.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Angeli, A., Marfia, G. & Riedel, N. Using off-the-shelf data-human interface platforms: traps and tricks. Multimed Tools Appl 80, 12907–12929 (2021). https://doi.org/10.1007/s11042-020-08929-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08929-z

Keywords

  • Data-human-interface
  • Machine learning
  • Deep learning