Advertisement

Studying the characteristics of logging practices in mobile apps: a case study on F-Droid

  • Yi ZengEmail author
  • Jinfu Chen
  • Weiyi Shang
  • Tse-Hsun (Peter) Chen
Article
  • 29 Downloads

Abstract

Logging is a common practice in software engineering. Prior research has investigated the characteristics of logging practices in system software (e.g., web servers or databases) as well as desktop applications. However, despite the popularity of mobile apps, little is known about their logging practices. In this paper, we sought to study logging practices in mobile apps. In particular, we conduct a case study on 1,444 open source Android apps in the F-Droid repository. Through a quantitative study, we find that although mobile app logging is less pervasive than server and desktop applications, logging is leveraged in almost all studied apps. However, we find that there exist considerable differences between the logging practices of mobile apps and the logging practices in server and desktop applications observed by prior studies. In order to further understand such differences, we conduct a firehouse email interview and a qualitative annotation on the rationale of using logs in mobile app development. By comparing the logging level of each logging statement with developers’ rationale of using the logs, we find that all too often (35.4%), the chosen logging level and the rationale are inconsistent. Such inconsistency may prevent the useful runtime information to be recorded or may generate unnecessary logs that may cause performance overhead. Finally, to understand the magnitude of such performance overhead, we conduct a performance evaluation between generating all the logs and not generating any logs in eight mobile apps. In general, we observe a statistically significant performance overhead based on various performance metrics (response time, CPU and battery consumption). In addition, we find that if the performance overhead of logging is significantly observed in an app, disabling the unnecessary logs indeed provides a statistically significant performance improvement. Our results show the need for a systematic guidance and automated tool support to assist in mobile logging practices.

Keywords

Software logs Logging practices Logging performance Mining software repositories 

Notes

Acknowledgements

The authors gratefully thank the developers who participated and shared their thoughts in our email interview.

References

  1. Boulon J, Konwinski A, Qi R, Rabkin A, Yang E, Yang M (2008) Chukwa, a large-scale monitoring system. In: Proceedings of CCA, vol 8, pp 1–5Google Scholar
  2. Chen B, Jiang ZMJ (2017) Characterizing logging practices in java-based open source software projects–a replication study in apache software foundation. Empir Softw Eng 22(1):330–374Google Scholar
  3. Chen J, Shang W (2017) An exploratory study of performance regression introducing code changes. In: 2017 IEEE international conference on software maintenance and evolution (ICSME). IEEE, pp 341–352Google Scholar
  4. Chen TH, Shang W, Jiang ZM, Hassan AE, Nasser M, Flora P (2014) Detecting performance anti-patterns for applications developed using object-relational mapping. In: Proceedings of the 36th international conference on software engineering. ACM, pp 1001–1012Google Scholar
  5. Chen TH, Shang W, Jiang ZM, Hassan AE, Nasser M, Flora P (2016) Finding and evaluating the performance impact of redundant data access for applications that are developed using object-relational mapping frameworks. IEEE Trans Softw Eng 42(12):1148–1161Google Scholar
  6. Chowdhury S, Di Nardo S, Hindle A, Jiang ZMJ (2017) An exploratory study on assessing the energy impact of logging on android apps. Empir Softw Eng, 1–35Google Scholar
  7. Cliff N (1993) Dominance statistics: ordinal analyses to answer ordinal questions. Psychol Bull 114(3):494Google Scholar
  8. Developer A (2017) Write and view logs with logcat. https://developer.android.com/studio/debug/am-logcat#WriteLogs
  9. Ding R, Zhou H, Lou JG, Zhang H, Lin Q, Fu Q, Zhang D, Xie T (2015) Log2: a cost-aware logging mechanism for performance diagnosis. In: USENIX annual technical conference, pp 139–150Google Scholar
  10. Elye (2018) Debug messages your responsibility to strip it before release!. https://medium.com/@elye.project/debug-messages-your-responsible-to-clear-it-before-release-1a0f872d66f
  11. F-Droid (2017) Free and open source android app repository. https://f-droid.org/
  12. Fleiss JL, Cohen J (1973) The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Measur 33(3):613–619Google Scholar
  13. Fu Q, Zhu J, Hu W, Lou JG, Ding R, Lin Q, Zhang D, Xie T (2014) Where do developers log? An empirical study on logging practices in industry. In: Companion proceedings of the 36th international conference on software engineering. ACM, pp 24–33Google Scholar
  14. Harpstead E, Zimmermann T, Nagapan N, Guajardo JJ, Cooper R, Solberg T, Greenawalt D (2015) What drives people: creating engagement profiles of players from game log data. In: Proceedings of the 2015 annual symposium on computer-human interaction in play. ACM, pp 369–379Google Scholar
  15. Hassani M, Shang W, Shihab E, Tsantalis N (2018) Studying and detecting log-related issues. Empir Softw Eng, 1–33Google Scholar
  16. Kabinna S, Bezemer CP, Shang W, Hassan AE (2016a) Logging library migrations: a case study for the apache software foundation projects. In: 2016 IEEE/ACM 13th working conference on mining software repositories (MSR), pp 154–164Google Scholar
  17. Kabinna S, Shang W, Bezemer CP, Hassan AE (2016b) Examining the stability of logging statements. In: 2016 IEEE 23rd international conference on software analysis, evolution, and reengineering (SANER), vol 1. IEEE, pp 326–337Google Scholar
  18. Kabinna S, Bezemer CP, Shang W, Syer MD, Hassan AE (2018) Examining the stability of logging statements. Empir Softw Eng 23(1):290–333.  https://doi.org/10.1007/s10664-017-9518-0 Google Scholar
  19. Kernighan BW, Pike R (1999) The practice of programming. Addison-Wesley Longman Publishing Co., Inc., BostonGoogle Scholar
  20. Li H, Shang W, Hassan AE (2017a) Which log level should developers choose for a new logging statement? Empir Softw Eng 22(4):1684–1716Google Scholar
  21. Li H, Shang W, Zou Y, Hassan AE (2017b) Towards just-in-time suggestions for log changes. Empir Softw Eng 22(4):1831–1865Google Scholar
  22. Li H, Chen THP, Shang W, Hassan AE (2018) Studying software logging using topic models. Empir Softw Eng 23(5):2655–2694.  https://doi.org/10.1007/s10664-018-9595-8 Google Scholar
  23. Lin D, Bezemer CP, Zou Y, Hassan AE (2018) An empirical study of game reviews on the steam platform. Empir Softw Eng, 1–38Google Scholar
  24. Malik H, Hemmati H, Hassan AE (2013) Automatic detection of performance deviations in the load testing of large scale systems. In: Proceedings of the 2013 international conference on software engineering, ICSE ’13. IEEE Press, Piscataway, pp 1012–1021. http://dl.acm.org/citation.cfm?id=2486788.2486927
  25. Moore DS, Craig BA, McCabe GP (2012) Introduction to the practice of statistics. WH FreemanGoogle Scholar
  26. Murphy-Hill E, Zimmermann T, Bird C, Nagappan N (2015) The design space of bug fixes and how developers navigate it. IEEE Trans Softw Eng 41(1):65–81Google Scholar
  27. Mytkowicz T, Diwan A, Hauswirth M, Sweeney PF (2009a) Producing wrong data without doing anything obviously wrong!. ACM Sigplan Not 44(3):265–276Google Scholar
  28. Mytkowicz T, Diwan A, Hauswirth M, Sweeney PF (2009b) Producing wrong data without doing anything obviously wrong! SIGPLAN Not 44(3):265–276Google Scholar
  29. Nagappan N, Ball T (2005) Use of relative code churn measures to predict system defect density. In: 27th international conference on software engineering, 2005. ICSE 2005. Proceedings. IEEE, pp 284–292Google Scholar
  30. Pinjia H, Zhuangbin C, Shilin H, Lyu MR (2018) Characterizing the natural language descriptions in software logging statements. In: Proceedings of the 33rd IEEE/ACM international conference on automated software engineering. IEEE PressGoogle Scholar
  31. Romano J, Kromrey JD, Coraggio J, Skowronek J (2006) Appropriate statistics for ordinal level data: should we really be using t-test and cohen’sd for evaluating group differences on the nsse and other surveys. In: Annual meeting of the Florida association of institutional research, pp 1–33Google Scholar
  32. Shang W, Jiang ZM, Adams B, Hassan AE, Godfrey MW, Nasser M, Flora P (2011) An exploratory study of the evolution of communicated information about the execution of large software systems. In: 2011 18th working conference on reverse engineering, pp 335–344Google Scholar
  33. Shang W, Jiang ZM, Adams B, Hassan AE, Godfrey MW, Nasser M, Flora P (2014a) An exploratory study of the evolution of communicated information about the execution of large software systems. J Softw Evol Process 26(1):3–26Google Scholar
  34. Shang W, Nagappan M, Hassan AE, Jiang ZM (2014b) Understanding log lines using development knowledge. In: 2014 IEEE international conference on software maintenance and evolution (ICSME). IEEE, pp 21–30Google Scholar
  35. Shang W, Nagappan M, Hassan AE (2015) Studying the relationship between logging characteristics and the code quality of platform software. Empir Softw Eng 20(1):1–27.  https://doi.org/10.1007/s10664-013-9274-8 Google Scholar
  36. Shull F, Singer J, Sjøberg DI (2007) Guide to advanced empirical software engineering. SpringerGoogle Scholar
  37. StackOverflow (2017) Why doesn’t “system.out.println” work in android? https://stackoverflow.com/a/2220559
  38. Syer MD, Jiang ZM, Nagappan M, Hassan AE, Nasser M, Flora P (2013) Leveraging performance counters and execution logs to diagnose memory-related performance issues. In: 2013 IEEE international conference on software maintenance, pp 110–119Google Scholar
  39. Tan J, Pan X, Kavulya S, Gandhi R, Narasimhan P (2008) Salsa: analyzing logs as state machines. In: Proceedings of the first USENIX conference on analysis of system logs, WASL’08. USENIX Association, Berkeley, pp 6–6. http://dl.acm.org/citation.cfm?id=1855886.1855892
  40. Weiyi S, Ming JZ, Bram A, HA E, GM W, Mohamed N, Parminder F (2013) An exploratory study of the evolution of communicated information about the execution of large software systems. J Softw: Evol Process 26(1):3–26.  https://doi.org/10.1002/smr.1579 Google Scholar
  41. Xu W, Huang L, Fox A, Patterson D, Jordan MI (2009) Detecting large-scale system problems by mining console logs. In: Proceedings of the ACM SIGOPS 22nd symposium on operating systems principles, SOSP ’09. ACM, New York, pp 117–132.  https://doi.org/10.1145/1629575.1629587
  42. Yamane T (1973) Statistics: an introductory analysisGoogle Scholar
  43. Yuan D, Park S, Zhou Y (2012a) Characterizing logging practices in open-source software. In: Proceedings of the 34th international conference on software engineering. IEEE Press, pp 102–112Google Scholar
  44. Yuan D, Park S, Huang P, Liu Y, Lee MMJ, Tang X, Zhou Y, Savage S (2012b) Be conservative: enhancing failure diagnosis with proactive logging. In: OSDI, vol 12, pp 293–306Google Scholar
  45. Yuan D, Zheng J, Park S, Zhou Y, Savage S (2012c) Improving software diagnosability via log enhancement. ACM Trans Comput Syst (TOCS) 30(1):4Google Scholar
  46. Zhao X, Rodrigues K, Luo Y, Stumm M, Yuan D, Zhou Y (2017) Log20: fully automated optimal placement of log printing statements under specified overhead threshold. In: Proceedings of the 26th symposium on operating systems principles. ACM, pp 565–581Google Scholar
  47. Zhu J, He P, Fu Q, Zhang H, Lyu MR, Zhang D (2015) Learning to log: helping developers make informed logging decisions. In: Proceedings of the 37th international conference on software engineering, vol 1. IEEE Press, pp 415–425Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science and Software EngineeringConcordia UniversityMontrealCanada

Personalised recommendations