Abstract
Mobile applications are used for accomplishing everyday life activities, such as shopping, banking, and social communications. To leverage the features of mobile apps, users often need to share sensitive information. However, recent research demonstrated that most of such apps present critical security and privacy defects. In this context, we define as vulnerability-proneness the risk level(s) that users meet in downloading specific apps, to better understand whether (1) users select apps with lower risk levels and if (2) vulnerability-proneness of an app might affect its success. We use as proxy to measure such risk level the “number of different types of potential security issues exhibited by the app”. We conjecture that the vulnerability-proneness levels may vary based on (i) the types of data handled by the app, and (ii) the operations for which the app is supposed to be used. Hence, we investigate how the vulnerability-proneness of apps varies when observing (i) different app categories, and (ii) apps with different success levels. Finally, to increase the awareness of both users and developers on the vulnerability-proneness of apps, we evaluate the extent to which contextual information provided by the app market can be exploited to estimate the vulnerability-proneness levels of mobile apps. Results of our study show that apps in the Medical category exhibit the lowest levels of vulnerability-proneness. Besides, while no strong relations between vulnerability-proneness and average rating are observed, apps with a higher number of downloads tend to have higher vulnerability-proneness levels, but lower vulnerability-proneness density. Finally, we found that apps’ contextual information can be used to predict, in the early stages, the vulnerability-proneness levels of mobile apps.
Similar content being viewed by others
Notes
https://buildfire.com/app-statistics - accessed on February 2021.
References
Acar Y, Backes M, Bugiel S, Fahl S, McDaniel P D, Smith M (2016) Sok: Lessons learned from android security research for appified software platforms. In: IEEE symposium on security and privacy, SP 2016. IEEE Computer Society, San Jose, pp 433–451
Afroz S, Islam A C, Santell J, Chapin A, Greenstadt R (2013) How privacy flaws affect consumer perception. In: Workshop on Socio-Technical Aspects in Security and Trust, pp 10–17
Alenezi M, Almomani I (2018) Empirical analysis of static code metrics for predicting risk scores in android applications. In: 5th International Symposium on Data Mining Applications. Springer, pp 84–94
Ali M, Joorabchi M E, Mesbah A (2017) Same app, different app stores: A comparative study. In: 4th IEEE/ACM International Conference on Mobile Software Engineering and Systems, MOBILESoft@ICSE 2017, Buenos Aires, Argentina, May 22-23, 2017, pp 79–90
Aliasgari M, Black M, Yadav N (2018) Security vulnerabilities in mobile health applications. In: Conference on Application, Information and Network Security, pp 21–26
Allix K, Bissyandé T F, Klein J, Traon Y L (2016) Androzoo: collecting millions of android apps for the research community. In: Proceedings of the 13th International Conference on Mining Software Repositories, MSR 2016, Austin, pp 468–471
Amin A, Eldessouki A, Magdy M T, Abdeen N, Hindy H, Hegazy I (2019) Androshield: Automated android applications vulnerability detection, a hybrid static and dynamic analysis approach. Inf 10(10):326. https://doi.org/10.3390/info10100326
Antoniol G, Ayari K, Penta M D, Khomh F, Guéhéneuc Y-G (2008) Is it a bug or an enhancement?: a text-based approach to classify change requests. In: Proceedings of Centre for Advanced Studies on Collaborative Research, p 23
Baeza-Yates R, Ribeiro-Neto B, et al. (1999) Modern information retrieval, vol 463. ACM press New York
Bavota G, Vásquez M L, Bernal-Cárdenas C E, Penta M D, Oliveto R, Poshyvanyk D (2015) The impact of API change- and fault-proneness on the user ratings of android apps. IEEE Trans Softw Eng 41(4):384–407. https://doi.org/10.1109/TSE.2014.2367027
Bhattacharya P, Ulanova L, Neamtiu I, Koduru S C (2013) An empirical analysis of bug reports and bug fixing in open source android apps. In: 17th European Conference on Software Maintenance and Reengineering, CSMR 2013, Genova, pp 133–143
Businge J, Openja M, Kavaler D, Bainomugisha E, Khomh F, Filkov V (2019) Studying android app popularity by cross-linking github and google play store. In: 26th IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2019, Hangzhou, pp 287–297
Cai Y, Tang Y, Li H, Yu L, Zhou H, Luo X, He L, Su P (2020) Resource race attacks on android. In: 27th IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2020, London, pp 47–58
Canfora G, Di Sorbo A, Mercaldo F, Visaggio C A (2016) Exploring mobile user experience through code quality metrics. In: Product-Focused Software Process Improvement - 17th International Conference, Proceedings, pp 705–712
Canfora G, Di Sorbo A, Forootani S, Pirozzi A, Visaggio C A (2020) Investigating the vulnerability fixing process in oss projects: Peculiarities and challenges. Comput Secur 99:102067
Cao C, Gao N, Liu P, Xiang J (2015) Towards analyzing the input validation vulnerabilities associated with android system services. In: Annual Computer Security Applications Conference, pp 361–370
Chia P H, Yamamoto Y, Asokan N (2012) Is this app safe?: a large scale study on application permissions and risk signals. In: Proceedings of the World Wide Web Conference, pp 311–320
Chin E, Felt A P, Greenwood K, Wagner D A (2011) Analyzing inter-application communication in android. In: International Conference on Mobile Systems, pp 239–252
Chin E, Wagner D A (2013) Bifocals: Analyzing webview vulnerabilities in android applications. In: Information Security Applications - International Workshop, WISA, pp 138–159
Clark J, van Oorschot P C (2013) Sok: SSL and HTTPS: revisiting past challenges and evaluating certificate trust model enhancements. In: Symposium on Security and Privacy, pp 511–525
Conover WJ (1998) Practical nonparametric statistics. Wiley series in probability and statistics: Applied probability and statistics, Wiley
Corral L, Fronza I (2015) Better code for better apps: A study on source code quality and market success of android applications. In: International Conference on Mobile Software Engineering and Systems, MOBILESoft, pp 22–32
Darvish H, Husain M I (2018) Security analysis of mobile money applications on android. In: IEEE international conference on big data, big data 2018, seattle, wa, usa, december 10-13, 2018, pp 3072–3078
Deka B, Huang Z, Franzen C, Hibschman J, Afergan D, Li Y, Nichols J, Kumar R (2017) Rico: A mobile app dataset for building data-driven design applications. In: Annual ACM Symposium on User Interface Software and Technology, pp 845–854
Di Sorbo A, Panichella S, Visaggio C A, Di Penta M, Canfora G, Gall H C (2019) Exploiting natural language structures in software informal documentation. IEEE Trans Softw Eng:1–1. https://doi.org/10.1109/TSE.2019.2930519
Di Sorbo A, Grano G, Visaggio C A, Panichella S (2021) Investigating the criticality of user-reported issues through their relations with app rating. J Softw Evol Process 33(3):e2316. https://doi.org/10.1002/smr.2316
Di Sorbo A, Panichella S, Alexandru C V, Shimagaki J, Visaggio C A, Canfora G, Gall H C (2016) What would users change in my app? summarizing app reviews for recommending software changes. In: Zimmermann T, Cleland-Huang J, Su Z (eds) Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016. ACM, Seattle, pp 499–510
Fahl S, Harbach M, Muders T, Smith M, Baumgärtner L, Freisleben B (2012) Why eve and mallory love android: an analysis of android SSL (in)security. In: Conference on Computer and Communications Security, pp 50–61
Felt A P, Chin E, Hanna S, Song D, Wagner D A (2011a) Android permissions demystified. In: ACM Conference on Computer and Communications Security, CCS 2011, Chicago, pp 627–638
Felt A P, Wang H J, Moshchuk A, Hanna S, Chin E (2011b) Permission re-delegation: Attacks and defenses. In: USENIX security symposium
Gajrani J, Tripathi M, Laxmi V, Somani G, Zemmari A, Gaur M S (2020) Vulvet: Vetting of vulnerabilities in android apps to thwart exploitation. Digit Threats Res Practice 1(2):1–25
Gao J, Li L, Kong P, Bissyandé T F, Klein J (2019) Understanding the evolution of android app vulnerabilities. IEEE Trans Reliab:1–19. https://doi.org/10.1109/TR.2019.2956690
Gartner (2015) Gartner Says More than 75 Percent of Mobile Applications will Fail Basic Security Tests Through 2015. https://tinyurl.com/uavh5nq. Online; accessed 20 January 2020
Giger E, D’Ambros M, Pinzger M, Gall H C (2012) Method-level bug prediction. In: International Symposium on Empirical Software Engineering and Measurement, pp 171–180
Gorla A, Tavecchia I, Gross F, Zeller A (2014) Checking app behavior against app descriptions. In: International Conference on Software Engineering, pp 1025–1035
Grano G, Di Sorbo A, Mercaldo F, Visaggio C A, Canfora G, Panichella S (2017) Android apps and user feedback: a dataset for software evolution and quality improvement. In: Proceedings of the 2nd ACM SIGSOFT International Workshop on App Market Analytics, WAMA@ESEC/SIGSOFT FSE 2017, Paderborn, pp 8–11
Grissom R J, Kim J J (2005) Effect sizes for research: A broad practical approach, 2nd edn. Lawrence Earlbaum Associates
Guerrouj L, Azad S, Rigby P C (2015) The influence of app churn on app success and stackoverflow discussions. In: International Conference on Software Analysis, Evolution, and Reengineering, pp 321–330
Harman M, Jia Y, Zhang Y (2012) App store mining and analysis: MSR for app stores. In: Working Conference of Mining Software Repositories, pp 108–111
Hay R, Tripp O, Pistoia M (2015) Dynamic detection of inter-application communication vulnerabilities in android. In: International Symposium on Software Testing and Analysis, pp 118–128
Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6(2):65–70
Islam M R (2014) Numeric rating of apps on google play store by sentiment analysis on user reviews. In: International Conference on Electrical Engineering and Information & Communication Technology. IEEE, pp 1–4
Jimenez M, Papadakis M, Bissyandé T F, Klein J (2016) Profiling android vulnerabilities. In: International Conference on Software Quality, Reliability and Security, pp 222–229
Johann T, Stanik C, B. A M A, Maalej W (2017) SAFE: A simple approach for feature extraction from app descriptions and app reviews. In: International Requirements Engineering Conference, pp 21–30
Kallis R, Di Sorbo A, Canfora G, Panichella S (2019) Ticket tagger: Machine learning driven issue classification. In: 2019 IEEE International Conference on Software Maintenance and Evolution, pp 406–409
Kantola D, Chin E, He W, Wagner D A (2012) Reducing attack surfaces for intra-application communication in android. In: Workshop on Security and Privacy in Smartphones and Mobile Devices, Co-located with CCS 2012, pp 69–80
Kaur A, Kaur I (2014) Empirical evaluation of machine learning algorithms for fault prediction. Lect Notes Softw Eng 2(2):176
Khalid H, Nagappan M, Hassan A E (2016) Examining the relationship between findbugs warnings and app ratings. IEEE Softw 33(4):34–39. https://doi.org/10.1109/MS.2015.29
Kochhar P S, Thung F, Nagappan N, Zimmermann T, Lo D (2015) Understanding the test automation culture of app developers. In: 8th IEEE International Conference on Software Testing, Verification and Validation, ICST 2015, Graz, Austria, April 13-17, 2015, pp 1–10
Kruskal W H, Wallis W A (1952) Use of ranks in one-criterion variance analysis. J Amer Stat Assocss 47(260):583–621
Krutz D E, Munaiah N, Meneely A, Malachowsky S A (2016) Examining the relationship between security metrics and user ratings of mobile apps: a case study. In: Proceedings of the International Workshop on App Market Analytics, pp 8–14
Li L, Bartel A, Bissyandé T F, Klein J, Le Traon Y, Arzt S, Rasthofer S, Bodden E, Octeau D, McDaniel P (2015) Iccta: Detecting inter-component privacy leaks in android apps. In: IEEE International Conference on Software Engineering, vol 1, pp 280–291
Lu L, Li Z, Wu Z, Lee W, Jiang G (2012) CHEX: statically vetting android apps for component hijacking vulnerabilities. In: the ACM Conference on Computer and Communications Security, pp 229–240
Lyu Y, Gui J, Wan M, Halfond W G J (2017) An empirical study of local database usage in android applications. In: 2017 IEEE International Conference on Software Maintenance and Evolution, ICSME 2017, Shanghai, China, September 17-22, 2017, pp 444–455
Ma Z, Wang H, Guo Y, Chen X (2016) Libradar: fast and accurate detection of third-party libraries in android apps. In: International Conference on Software Engineering, Companion Volume, pp 653–656
Manadhata P K, Wing J M (2011) An attack surface metric. IEEE Trans Softw Eng 37(3):371–386. https://doi.org/10.1109/TSE.2010.60
Minelli R, Lanza M (2013a) Software analytics for mobile applications–insights lessons learned. In: 2013 17th European Conference on Software Maintenance and Reengineering, pp 144–153
Minelli R, Lanza M (2013b) Software analytics for mobile applications-insights & lessons learned. In: 17th European Conference on Software Maintenance and Reengineering, CSMR 2013, Genova, Italy, March 5-8, 2013, pp 144–153
Montealegre C, Njuguna C R, Malik M I, Hannay P, McAteer I N (2018) Security vulnerabilities in android applications. In: Australian Information Security Management Conference. Security Research Institute, Edith Cowan University, pp 14–28
Mutchler P, Safaei Y, Doupé A, Mitchell J C (2016) Target fragmentation in android apps. In: 2016 IEEE Security and Privacy Workshops, SP Workshops 2016, San Jose, CA, USA, May 22-26, 2016, pp 204–213
Nguyen D-C, Derr E, Backes M, Bugiel S (2019) Short text, large effect: Measuring the impact of user reviews on android app security & privacy. In: 2019 IEEE Symposium on Security and Privacy, SP 2019, San Francisco, pp 555–569
Oltrogge M, Huaman N, Amft S, Acar Y, Backes M, Fahl S (2021) Why eve and mallory still love android: Revisiting tls (in) security in android applications. In: 30th USENIX Security Symposium (USENIX Security 21)
Panichella S, Di Sorbo A, Guzman E, Visaggio C A, Canfora G, Gall H C (2015) How can i improve my app? classifying user reviews for software maintenance and evolution. In: Koschke R, Krinke J, Robillard M P (eds) 2015 IEEE International Conference on Software Maintenance and Evolution, ICSME 2015, Bremen, Germany, September 29 - October 1, 2015. IEEE Computer Society, pp 281–290
Panichella S (2018) Summarization techniques for code, change, testing, and user feedback (invited paper). In: Artho C, Ramler R (eds) 2018 IEEE Workshop on Validation, Analysis and Evolution of Software Tests, VST@SANER 2018, Campobasso, Italy, March 20, 2018. IEEE, pp 1–5
Papageorgiou A, Strigkos M, Politou E A, Alepis E, Solanas A, Patsakis C (2018) Security and privacy analysis of mobile health applications: The alarming state of practice. IEEE Access 6:9390–9403. https://doi.org/10.1109/ACCESS.2018.2799522
Pecorelli F, Catolino G, Ferrucci F, Lucia A D, Palomba F (2020) Testing of mobile applications in the wild: A large-scale empirical study on android apps. In: ICPC ’20: 28th international conference on program comprehension, seoul, republic of korea, july 13-15, 2020, pp 296–307
Qian C, Luo X, Le Y, Gu G (2015) Vulhunter: Toward discovering vulnerabilities in android applications. IEEE Micro 35(1):44–53. https://doi.org/10.1109/MM.2015.25
Quinlan J R (1986) Induction of decision trees. Mach Learn 1 (1):81–106
Ruiz I J M, Nagappan M, Adams B, Berger T, Dienst S, Hassan A E (2014) Impact of ad libraries on ratings of android mobile apps. IEEE Softw 31(6):86–92. https://doi.org/10.1109/MS.2014.79
Ruiz I J M, Nagappan M, Adams B, Berger T, Dienst S, Hassan A E (2016) Examining the rating system used in mobile-app stores. IEEE Softw 33(6):86–92. https://doi.org/10.1109/MS.2015.56
Russo E R, Di Sorbo A, Visaggio C A, Canfora G (2019) Summarizing vulnerabilities’ descriptions to support experts during vulnerability assessment activities. J Syst Softw 156:84–99. https://doi.org/10.1016/j.jss.2019.06.001
Scandariato R, Walden J (2012) Predicting vulnerable classes in an android application. In: International Workshop on Security Measurements and Metrics, MetriSec ’12. Association for Computing Machinery, pp 11–16
Shapiro S S, Wilk M B (1965) An analysis of variance test for normality (complete samples). Biometrika 52(3/4):591–611
Silva D B, Eler M M, Durelli V H S, Endo A T (2018) Characterizing mobile apps from a source and test code viewpoint. Inf Softw Technol 101:32–50. https://doi.org/10.1016/j.infsof.2018.05.006
Slavin R, Wang X, Hosseini M B, Hester J, Krishnan R, Bhatia J, Breaux T D, Niu J (2016) Toward a framework for detecting privacy policy violations in android application code. In: Dillon L K, Visser W, Williams L (eds) International Conference on Software Engineering. ACM, pp 25–36
Song W, Huang Q, Huang J (2018) Understanding javascript vulnerabilities in large real-world android applications. IEEE Trans Depend Sec Comput:1–1
Sounthiraraj D, Sahs J, Greenwood G, Lin Z, Khan L (2014) Smv-hunter: Large scale, automated detection of SSL/TLS man-in-the-middle vulnerabilities in android apps. In: 21st Annual Network and Distributed System Security Symposium
Taba S E S, Keivanloo I, Zou Y, Ng J W, Ng T (2014) An exploratory study on the relation between user interface complexity and the perceived quality. In: Web Engineering, International Conference, pp 370–379
Tao C, Guo H, Huang Z (2020) Identifying security issues for mobile applications based on user review summarization. Inf Softw Technol 122:106290. https://doi.org/10.1016/j.infsof.2020.106290
Taylor V F, Martinovic I (2017a) Short paper: A longitudinal study of financial apps in the google play store. In: Financial Cryptography and Data Security - International Conference, pp 302–309
Taylor V F, Martinovic I (2017b) To update or not to update: Insights from a two-year study of android app evolution. In: ACM on asia conference on computer and communications security, pp 45–57
Thomas D R, Beresford A R, Coudray T, Sutcliffe T, Taylor A (2015a) The lifetime of android API vulnerabilities: Case study on the javascript-to-java interface. In: Security Protocols XXIII - 23rd International Workshop, pp 126–138
Thomas D R, Beresford A R, Rice A C (2015b) Security metrics for the android ecosystem. In: Annual ACM CCS Workshop on Security and Privacy in Smartphones and Mobile Devices, pp 87–98
Tian Y, Nagappan M, Lo D, Hassan A E (2015) What are the characteristics of high-rated apps? A case study on free android applications. In: International Conference on Software Maintenance and Evolution, pp 301–310
Tien C-W, Huang T-Y, Huang T-C, Chung W-H, Kuo S-Y (2017) MAS: mobile-apps assessment and analysis system. In: International Conference on Dependable Systems and Networks Workshops, pp 145–148
Vásquez M L, Bavota G, Bernal-Cárdenas C, Penta M D, Oliveto R, Poshyvanyk D (2013) API change and fault proneness: a threat to the success of android apps. In: Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pp 477–487
Vásquez M L, McMillan C, Poshyvanyk D, Grechanik M (2014) On using machine learning to automatically classify software applications into domain categories. Empir Softw Eng 19(3):582–618. https://doi.org/10.1007/s10664-012-9230-z
Vásquez M L, Holtzhauer A, Poshyvanyk D (2016) On automatically detecting similar android apps. In: 24th IEEE International Conference on Program Comprehension, ICPC 2016, Austin, TX, USA, May 16-17, 2016, pp 1–10
Vásquez M L, Bavota G, Escobar-Velasquez C (2017) An empirical study on android-related vulnerabilities. In: Proceedings of the 14th International Conference on Mining Software Repositories, MSR 2017, Buenos Aires, Argentina, May 20-28, 2017, pp 2–13
Votipka D, Stevens R, Redmiles E M, Hu J, Mazurek M L (2018) Hackers vs. testers: A comparison of software vulnerability discovery processes. In: 2018 IEEE symposium on security and privacy, SP 2018, proceedings, 21-23 may 2018, san francisco, california, USA, pp 374–391
Wang H, Li H, Li L, Guo Y, Xu G (2018) Why are android apps removed from google play?: a large-scale empirical study. In: Zaidman A, Kamei Y, Hill E (eds) Proceedings of the 15th International Conference on Mining Software Repositories, MSR 2018, Gothenburg, Sweden, May 28-29, 2018. ACM, pp 231–242
Watanabe T, Akiyama M, Kanei F, Shioji E, Takata Y, Sun B, Ishii Y, Shibahara T, Yagi T, Mori T (2017) Understanding the origins of mobile app vulnerabilities: a large-scale measurement study of free and paid apps. In: International Conference on Mining Software Repositories, pp 14–24
Wu D, Chang R K C (2014) Analyzing android browser apps for file: // vulnerabilities. In: Information Security - International Conference, pp 345–363
Xu M, Song C, Ji Y, Shih M-W, Lu K, Zheng C, Duan R, Jang Y, Lee B, Qian C, Lee S, Kim T (2016) Toward engineering a secure android ecosystem: A survey of existing techniques. ACM Comput Surv 49(2):38:1–38:47. https://doi.org/10.1145/2963145
Yang Z, Yang M, Zhang Y, Gu G, Ning P, Wang X S (2013) Appintent: analyzing sensitive data transmission in android for privacy leakage detection. In: 2013 ACM SIGSAC Conference on Computer and Communications Security, CCS’13, Berlin, Germany, November 4-8, 2013, pp 1043–1054
Yang W, Zhang Y, Li J, Liu H, Wang Q, Zhang Y, Gu D (2017) Show me the money! finding flawed implementations of third-party in-app payment in android apps. In: Annual Network and Distributed System Security Symposium
Yeom C, Won Y (2019) Vulnerability evaluation method through correlation analysis of android applications. Sustainability 11(23). https://doi.org/10.3390/su11236637
Zampetti F, Di Sorbo A, Visaggio C A, Canfora G, Di Penta M (2020) Demystifying the adoption of behavior-driven development in open source projects. Inf Softw Technol 123:106311. https://doi.org/10.1016/j.infsof.2020.106311
Zhou Y, Jiang X (2013) Detecting passive content leaks and pollution in android applications. In: Annual Network and Distributed System Security Symposium
Acknowledgments
We gratefully thank Prof. Dr. Harald Gall, Dean of the Faculty of Business, Economics, and Informatics of the University of Zurich and director of the Software Evolution and Architecture Lab, for supporting this research, making the lab facilities available to the development of this research project. We also thank Prof. Gall for the qualitative feedback on the direction of this work and the ongoing collaboration in close-related research projects. Finally, we thank the anonymous reviewers and the editors for the constructive and relevant feedback on our study. Their openness to dialogue has been fundamental to improve the manuscript.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Denys Poshyvanyk
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Di Sorbo, A., Panichella, S. Exposed! A case study on the vulnerability-proneness of Google Play Apps. Empir Software Eng 26, 78 (2021). https://doi.org/10.1007/s10664-021-09978-0
Accepted:
Published:
DOI: https://doi.org/10.1007/s10664-021-09978-0