Abstract
“App stores” are online software stores where end users may browse, purchase, download, and install software applications. By far, the best known app stores are associated with mobile platforms, such as Google Play for Android and Apple’s App Store for iOS. The ubiquity of smartphones has led to mobile app stores becoming a touchstone experience of modern living. App stores have been the subject of many empirical studies. However, most of this research has concentrated on properties of the apps rather than the stores themselves. Today, there is a rich diversity of app stores and these stores have largely been overlooked by researchers: app stores exist on many distinctive platforms, are aimed at different classes of users, and have different end-goals beyond simply selling a standalone app to a smartphone user.
The goal of this paper is to survey and characterize the broader dimensionality of app stores, and to explore how and why they influence software development practices, such as system design and release management. We begin by collecting a set of app store examples from web search queries. By analyzing and curating the results, we derive a set of features common to app stores. We then build a dimensional model of app stores based on these features, and we fit each app store from our web search result set into this model. Next, we performed unsupervised clustering to the app stores to find their natural groupings. Our results suggest that app stores have become an essential stakeholder in modern software development. They control the distribution channel to end users and ensure that the applications are of suitable quality; in turn, this leads to developers adhering to various store guidelines when creating their applications. However, we found the app stores operational model could vary widely between stores, and this variability could in turn affect the generalizability of existing understanding of app stores.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
A dataset consists of the Google query results and the app store labeling results are available on Zenodo.
We would like to thank the organizers, administrators, and attendees of the Shonan meeting No. 152 (McIntosh et al. 2019) on “Release Engineering for Mobile Applications”, where the paper’s idea was conceived.
One of the authors has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 825328 (FASTEN).
Notes
The Windows Phone Store was absorbed into the broader Windows Store in 2015.
Android Marketplace has since been re-branded as Google Play.
See Update to 10.8.7 https://docs.microsoft.com/en-us/windows/uwp/publish/store-policies-change-history
References
Adolph S, Hall W, Kruchten P (2011) “Using grounded theory to study the experience of software development,” in Empirical Software Engineering, Springer
Aljedaani W, Nagappan M, Adams B, Godfrey M (2019) “A comparison of bugs across the ios and android platforms of two open source cross platform browser apps,” in Int. Conf. on Mobile Software Engineering and Systems, IEEE
Almanee S, Ünal A, Payer M, Garcia J (2021) “Too Quiet in the Library: An Empirical Study of Security Updates in Android Apps Native Code,” in Int Conf on Software Engineering, IEEE
Alshayban A, Ahmed I, Malek S (2020) “Accessibility issues in android apps: state of affairs, sentiments, and ways forward,” in Int Conf on Software Engineering, IEEE
Al-Subaihin A, Sarro F, Black S, Capra L (2019) “Empirical comparison of text-based mobile apps similarity measurement techniques,” Empir Softw Eng
Al-Subaihin AA, Sarro F, Black S, Capra L, Harman M (2021) “App store effects on software engineering practices,” in Transactions on software engineering, IEEE
Amazon (2022) “AWS Marketplace: Homepage.” https://aws.amazon.com/marketplace/. Accessed: Jun. 22, 2022
Apple (2008) “Apple Introduces the New iPhone 3G.” https://www.apple.com/ca/newsroom/2008/06/09Apple-Introduces-the-New-iPhone-3G/. Accessed: Jul. 17, 2022
Arthur D, Vassilvitskii S (2006) “K-means++: the advantages of careful seeding,” tech rep, Stanford
Arzt S (2021) “Sustainable Solving: Reducing The Memory Footprint of IFDS-Based Data Flow Analyses Using Intelligent Garbage Collection,” in Int Conf on Software Engineering, IEEE
Autodesk (2022) “Autodesk App Store : Plugins, Add-ons for Autodesk software, AutoCAD, Revit, Inventor, 3ds Max, Maya ....” https://apps.autodesk.com/. Accessed: Jun. 22, 2022
Canonical (2009) “Ubuntu Software Center in Launchpad.” https://launchpad.net/software-center. Accessed: Jun. 22, 2022
Chen J, Chen C, Xing Z, Xu X, Zhut L, Li G, Wang J (2020b) “Unblind your apps: Predicting natural-language labels for mobile gui components by deep learning,” in Int Conf on Software Engineering, IEEE
Chen J, Xia X, Lo D, Grundy J, Yang X (2021) “Maintenance-related concerns for post-deployed Ethereum smart contract development: issues, techniques, and future challenges,” in Empirical Software Engineering, Springer
Chen S, Fan L, Meng G, Su T, Xue M, Xue Y, Liu Y, Xu L (2020a) “An empirical assessment of security risks of global android banking apps,” in Int Conf on Software Engineering, IEEE
Claes M, Mens T, Di Cosmo R, Vouillon J (2015) “A historical analysis of Debian package incompatibilities,” in Int Conf on Mining Software Repositories, IEEE
Cohen J (1960) “A coefficient of agreement for nominal scales,” in Educational and psychological measurement, Sage
Coxon APM et al. (1999) Sorting data: collection and analysis. Sage
Dabrowski J, Letier E, Perini A, Susi A (2022) “Analysing app reviews for software engineering: a systematic literature review,” in Empirical Software Engineering, Springer
Dixon C, Mahajan R, Agarwal S, Brush A, Lee B, Saroiu S, Bahl V (2010) “The home needs an operating system (and an app store),” in SIGCOMM Workshop on Hot Topics in Networks, ACM
Docker (2022) “Explore Docker’s Container Image Repository | Docker Hub.” https://hub.docker.com/search?q=. Accessed: Jun. 22, 2022
Dong Z, Böhme M, Cojocaru L, Roychoudhury A (2020) “Time-travel testing of android apps,” in Int Conf on Software Engineering, IEEE
E. Commission (2022) “Digital Markets Act: Commission welcomes political agreement on rules to ensure fair and open digital markets.” https://ec.europa.eu/commission/presscorner/detail/en/IP_22_1978. Accessed: Jul. 13, 2022
E. International (2022) “TC39 - Specifying JavaScript..” https://tc39.es/. Accessed: Oct. 02, 2022
F-Droid (2022) “F-Droid - Free and Open Source Android App Repository.” https://f-droid.org/. Accessed: Oct. 02, 2022
Ferreira G, Jia L, Sunshine J, Kästner C (2021) “Containing malicious package updates in npm with a lightweight permission system,” in Int Conf on Software Engineering (ICSE), IEEE
Finkelstein A, Harman M, Jia Y, Martin W, Sarro F, Zhang Y (2017) “Investigating the relationship between price, rating, and popularity in the Blackberry world app store,” Inf Softw Technol
Fischer RA-L, Walczuch R, Guzman E (2021) “Does culture matter? impact of individualism and uncertainty avoidance on app reviews,” in Int Conf on Software Engineering: Software Engineering in Society, IEEE
Gaber T, Ahmed A, Mostafa A (2020) “Privdrm: A privacy-preserving secure digital right management system,” in Evaluation and Assessment in Software Engineering, ACM
Gholami S, Khazaei H, Bezemer C-P (2021) “Should you upgrade official docker hub images in production environments?,” in Int Conf on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), IEEE
GitHub (2022) “GitHub Marketplace · to improve your workflow · GitHub.” https://github.com/marketplace?type=. Accessed: Jun. 06 2022
Google (2022a) “Chrome Web Store - Extensions.” https://chrome.google.com/webstore/category/extensions. Accessed: Jun. 22, 2022
Google (2022b) “Chrome Web Store payments deprecation.” https://developer.chrome.com/docs/webstore/cws-payments-deprecation/. Accessed: Mar. 16, 2022
Guo H, Singh MP (2020) “Caspar: extracting and synthesizing user stories of problems from app reviews,” in Int Conf on Software Engineering, IEEE
Guzman E, Oliveira L, Steiner Y, Wagner LC, Glinz M (2018) “User feedback in the app store: a cross-cultural study,” in Int Conf on Software Engineering: Software Engineering in Society
Haering M, Stanik C, Maalej W (2021) “Automatically matching bug reports with related app reviews,” in Int Conf on Software Engineering, IEEE
Haggag O, Haggag S, Grundy J, Abdelrazek M (2021) “COVID-19 vs social media apps: does privacy really matter?,” in Int Conf on Software Engineering: Software Engineering in Society, IEEE
Harman M, Jia Y, Zhang Y (2012) “App store mining and analysis: MSR for App Stores,” in Int. Conf. on Mining Software Repositories, IEEE
He X, Dai W, Cao G, Tang R, Yuan M, Yang Q (2015) “Mining target users for online marketing based on app store data,” in Int Conf on Big Data (Big Data), IEEE
Hoda R, Noble J, Marshall S (2012) “Developing a grounded theory to explain the practices of self-organizing Agile teams,” in Empirical Software Engineering, Springer
Hu Y, Wang H, Ji T, Xiao X, Luo X, Gao P, Guo Y (2021) “Champ: Characterizing undesired app behaviors from user comments based on market policies,” in Int Conf on Software Engineering, IEEE
Ibrahim MH, Sayagh M, Hassan A. E (2020) “Too many images on dockerhub! How different are images for the same system?,” Empir Softw Eng
Jansen S, Bloemendal E (2013) “Defining app stores: The role of curated marketplaces in software ecosystems,” in ICSOB, Springer
Khalid H, Nagappan M, Shihab E, Hassan AE (2014) “Prioritizing the devices to test your app on: A case study of android game apps,” in Int Symposium on Foundations of Software Engineering
Khatibi Bardsiri V, Jawawi DNA, Hashim SZM, Khatibi E (2014) “A flexible method to estimate the software development effort based on the classification of projects and localization of comparisons,” Empir Softw Eng
Kodi T (2022) “The Movie Database Python | Matrix | Addons | Kodi.” https://kodi.tv/addons/matrix/metadata.themoviedb.org.python. Accessed: Jul. 13, 2022
Kuchta T, Lutellier T, Wong E, Tan L, Cadar C (2018) “On the correctness of electronic documents: studying, finding, and localizing inconsistency bugs in PDF readers and files,” Empir Softw Eng
Kuznetsov K, Fu C, Gao S, Jansen DN, Zhang L, Zeller A (2021) “Frontmatter: mining Android user interfaces at scale,” in Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ACM
Lantz CA, Nebenzahl E (1996) “Behavior and interpretation of the κ statistic: Resolution of the two paradoxes,” in J Clin Epidemiol, Elsevier
Lee D, Lin D, Bezemer C-P, Hassan AE (2020b) “Building the perfect game–an empirical study of game modifications,” Empir Softw Eng
Lee D, Rajbahadur GK, Lin D, Sayagh M, Bezemer C-P, Hassan AE (2020a) “An empirical study of the characteristics of popular Minecraft mods,” Empir Softw Eng
Lemon M (2018) “Two Point Hospital no longer uses Denuvo DRM.” https://www.vg247.com/two-point-hospital-no-longer-uses-denuvo-drm. Accessed: Mar. 31, 2023
Lin D, Bezemer C-P, Zou Y, Hassan AE (2019) “An empirical study of game reviews on the steam platform,” in Empirical Software Engineering, Springer
Liu P, Li L, Yan Y, Fazzini M, Grundy J (2021) “Identifying and characterizing silently-evolved methods in the android API,” in Int Conf on Software Engineering: Software Engineering in Practice, IEEE
Lu Z, Shi Y, Tao R, Zhang Z (2019) “Blockchain for digital rights management of design works,” in Int. Conf on Software Engineering and Service Science (ICSESS), IEEE
Ma S, Li J, Kim H, Bertino E, Nepal S, Ostry D, Sun C (2021) “Fine with 1234? An Analysis of SMS One-Time Password Randomness in Android Apps,” in Int Conf on Software Engineering, IEEE
Maalej W, Nayebi M, Ruhe G (2019) “Data-driven requirements engineering-an update,” in Int Conf on Software Engineering: Software Engineering in Practice (ICSE-SEIP), IEEE
MacQueen J et al. (1967) “Some methods for classification and analysis of multivariate observations,” in Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, Oakland, CA, USA
Manotas I, Bird C, Zhang R, Shepherd D, Jaspan C, Sadowski C, Pollock L, Clause J (2016) “An empirical study of practitioners’ perspectives on green software engineering,” in Int Conf on Software Engineering
Martin W, Sarro F, Harman M (2016b) “Causal impact analysis for app releases in google play,” in Int. Symposium on Foundations of software engineering
Martin W, Sarro F, Jia Y, Zhang Y, Harman M (2016a) “A survey of app store analysis for software engineering,” in Transactions on software engineering, IEEE
Masood Z, Hoda R, Blincoe K (2020) “How agile teams make self-assignment work: a grounded theory study,” in Empirical Software Engineering, Springer
McIntosh S, Kamei Y, Nagappan M (2019) Release engineering for Mobile applications — communications of NII Shonan meetings. Springer
McMillan C, Grechanik M, Poshyvanyk D (2012) “Detecting similar software applications,” in Int Conf on Software Engineering (ICSE), IEEE
Murali V, Yao E, Mathur U, Chandra S (2021) “Scalable statistical root cause analysis on app telemetry,” in Int Conf on Software Engineering: Software Engineering in Practice, IEEE
Murphy-Hill E, Zimmermann T, Nagappan N (2014) “Cowboys, ankle sprains, and keepers of quality: How is video game development different from software development?,” in Int Conf on Software Engineering
Nayebi M, Adams B, Ruhe G (2016) “Release Practices for Mobile Apps–What do Users and Developers Think?,” in Int Conf On software analysis, evolution, and reengineering (saner), IEEE
Nayebi M, Cho H, Farrahi H, Ruhe G (2017b) “App store mining is not enough,” in Int Conf on Software Engineering Companion (ICSE-C), IEEE
Nayebi M, Farahi H, Ruhe G (2017a) “Which version should be released to app store?,” in Int Symposium on Empirical Software Engineering and Measurement (ESEM), IEEE
Ng YY, Zhou H, Ji Z, Luo H, Dong Y (2014) “Which Android app store can be trusted in China?,” in Computer Software and Applications Conference, IEEE
Nguyen T, Vu P, Nguyen T (2020) “Code recommendation for exception handling,” in Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ACM
npm, “npm About.” (2022) https://www.npmjs.com/about. Accessed: Oct. 02, 2022
Obie HO, Hussain W, Xia X, Grundy J, Li L, Turhan B, Whittle J, Shahin M (2021) “A first look at human values-violation in app reviews,” in Int Conf on Software Engineering: Software Engineering in Society, IEEE
Pan L, Cui B, Liu H, Yan J, Wang S, Yan J, Zhang J (2020) “Static asynchronous component misuse detection for Android applications,” in Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ACM
Pascarella L, Palomba F, Di Penta M, Bacchelli A (2018) “How is video game development different from software development in open source?,” in Int Conf on Mining Software Repositories, IEEE
Pérez J, Daz J, Garcia-Martin J, Tabuenca B (2020) “Systematic literature reviews in software engineering–Enhancement of the study selection process using Cohens kappa statistic,” in J Syst Softw, Elsevier
Pham VVH, Liu X, Zheng X, Fu M, Deshpande SV, Xia W, Zhou R, Abdelrazek M (2017) “PaaS-black or white: an investigation into software development model for building retail industry SaaS,” in Int. Conf. On software engineering companion (ICSE-C), IEEE
Pickerill P, Jungen HJ, Ochodek M, Maćkowiak M, Staron M (2020) “Phantom: curating github for engineered software projects using time-series clustering,” Empir Softw Eng
Prévost R, McQuaid M, Lalonde D (2022) “The Missing Package Manager for macOS (or Linux) — Homebrew.” https://brew.sh/. Accessed: Jun. 22, 2022
Rahaman S, Neamtiu I, Yin X (2021) “Algebraic-datatype taint tracking, with applications to understanding Android identifier leaks,” in Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ACM
Rosen C, Shihab E (2016) “What are mobile developers asking about? a large scale study using stack overflow,” in Empirical Software Engineering, Springer
Rousseeuw PJ (1987) “Silhouettes: a graphical aid to the interpretation and validation of cluster analysis,” in J Comput Appl Math, Elsevier
Ruiz IJM, Nagappan M, Adams B, Hassan AE (2012) “Understanding reuse in the android market,” in Int. Conf. on Program Comprehension, IEEE
Sarro F, Al-Subaihin AA, Harman M, Jia Y, Martin W, Zhang Y (2015) “Feature lifecycles as they spread, migrate, remain, and die in app stores,” in Int requirements engineering conference (RE), IEEE
Scoccia GL, Autili M, Stilo G, Inverardi P (2022) “An empirical study of privacy labels on the apple iOS mobile app store,” in Int Conf on Mobile Software Engineering and Systems
Shams RA, Hussain W, Oliver G, Nurwidyantoro A, Perera H, Whittle J (2020) “Society-oriented applications development: Investigating users values from bangladeshi agriculture mobile applications,” in Int. Conf. on Software Engineering: Software Engineering in Society, IEEE
Shen S, Lu X, Hu Z, Liu X (2017) “Towards release strategy optimization for apps in Google play,” in Proceedings of the 9th Asia-Pacific symposium on Internetware
Song W, Han M, Huang J (2021) “IMGDroid: Detecting Image Loading Defects in Android Applications,” in Int Conf on Software Engineering, IEEE
Subramanian GH, Pendharkar PC, Wallace M (2006) “An empirical study of the effect of complexity, platform, and program type on software development effort of business applications,” Empir Softw Eng
Sun R, Wang W, Xue M, Tyson G, Camtepe S, Ranasinghe DC (2021) “An empirical assessment of global COVID-19 contact tracing applications,” in Int Conf on Software Engineering, IEEE
Sung A, Kim S, Kim Y, Jang Y, Kim J (2019) “Test automation and its limitations: a case study,” in Int Conf On automated software engineering (ASE), IEEE
Tang C, Chen S, Fan L, Xu L, Liu Y, Tang Z, Dou L (2019) “A large-scale empirical study on industrial fake apps,” in Int Conf on Software Engineering: Software Engineering in Practice (ICSE-SEIP), IEEE
Truelove A, de Almeida ES, Ahmed I (2021) “Well Fix It in Post: What Do Bug Fixes in Video Game Update Notes Tell Us?,” in Int Conf on Software Engineering, IEEE
Ullmann GC, Politowski C, Guéhéneuc Y-G, Petrillo F (2022) “What makes a game high-rated? towards factors of video game success,” in Int ICSE Workshop on Games and Software Engineering: Engineering Fun, Inspiration, and Motivation
Valve (2022) “Welcome to Steam.” https://store.steampowered.com/, Accessed: Jun. 22 2022
Van Der Linden D, Anthonysamy P, Nuseibeh B, Tun TT, Petre M, Levine M, Towse J, Rashid A (2020) “Schrödinger’s security: Opening the box on app developers’ security rationale,” in Int Conf on Software Engineering, IEEE
Vassallo C, Panichella S, Palomba F, Proksch S, Gall HC, Zaidman A (2020) “How developers engage with static analysis tools in different contexts,” in Empirical Software Engineering, Springer
Vouillon J, Cosmo RD (2013) “On software component co-installability,” Trans Softw Eng Methodol (TOSEM)
Walker D, Myrick F (2006) “Grounded theory: An exploration of process and procedure,” in Qualitative health research, Sage
Wang H, Liu Z, Liang J, Vallina-Rodriguez N, Guo Y, Li L, Tapiador J, Cao J, Xu G (2018) “Beyond google play: a large-scale comparative study of chinese android app markets,” in Internet measurement conference 2018
Wang H, Wang X, Guo Y (2019) “Characterizing the global mobile app developers: a large-scale empirical study,” in Int Conf on Mobile Software Engineering and Systems, IEEE
Wang P, Brown C, Jennings JA, Stolee KT (2022) “Demystifying regular expression bugs,” in Empirical Software Engineering, Springer
Wikipedia (2022) “Electronic AppWrapper - Wikipedia.” https://en.wikipedia.org/wiki/Electronic_AppWrapper. Accessed: Jun. 22, 2022
Wold S, Esbensen K, Geladi P (1987) “Principal component analysis,” in Chemometrics and intelligent laboratory systems, Elsevier
WordPress (2022) “WordPress Plugins | WordPress.org.” https://wordpress.org/plugins/. Accessed: Jun. 22, 2022
Wu H, Deng W, Niu X, Nie C (2021) “Identifying key features from app user reviews,” in Int Conf on Software Engineering, IEEE
Yang B, Xing Z, Xia X, Chen C, Ye D, Li S (2021) “Don’t do that! hunting down visual design smells in complex uis against design guidelines,” in Int Conf on Software Engineering, IEEE
Yang S, Wang Y, Yao Y, Wang H, Ye YF, Xiao X (2022) DescribeCtx: context-aware description synthesis for sensitive behaviors in mobile apps. In: Int Conf on Software Engineering, IEEE
Ye J, Chen K, Xie X, Ma L, Huang R, Chen Y, Xue Y, Zhao J (2021) “An empirical study of GUI widget detection for industrial mobile games,” in Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ACM
Yu S, Fang C, Cao Z, Wang X, Li T, Chen Z (2021b) “Prioritize crowdsourced test reports via deep screenshot understanding,” in Int Conf on Software Engineering, IEEE
Yu S, Fang C, Yun Y, Feng Y (2021a) “Layout and image recognition driving cross-platform automated mobile testing,” in Int Conf on Software Engineering, IEEE
Zhan X, Fan L, Chen S, Wu F, Liu T, Luo X, Liu Y (2021) “Atvhunter: Reliable version detection of third-party libraries for vulnerability identification in android applications,” in Int. Conf. on Software Engineering, IEEE
Zhang X, Wang X, Slavin R, Breaux T, Niu J (2020) “How does misconfiguration of analytic services compromise mobile privacy?,” in Int Conf on Software Engineering, IEEE
Zhang Z, Feng Y, Ernst MD, Porst S, Dillig I (2021) “Checking conformance of applications against GUI policies,” in Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ACM
Zhao T, Chen C, Liu Y, Zhu X (2021) “GUIGAN: Learning to Generate GUI Designs Using Generative Adversarial Networks,” in Int Conf on Software Engineering, IEEE
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
The authors declared that they have no conflict of interest.
Additional information
Responsible Editor: Tayana Conte
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhu, W., Proksch, S., German, D.M. et al. What is an app store? The software engineering perspective. Empir Software Eng 29, 35 (2024). https://doi.org/10.1007/s10664-023-10362-3
Accepted:
Published:
DOI: https://doi.org/10.1007/s10664-023-10362-3