Skip to main content
SpringerLink
Log in

Evolutionary Fuzzing of Android OS Vendor System Services

  • SI: Mobile SE
  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Android devices are shipped in several flavors by more than 100 manufacturer partners, which extend the Android “vanilla” OS with new system services, and modify the existing ones. These proprietary extensions expose Android devices to reliability and security issues. In this paper, we propose a coverage-guided fuzzing platform (Chizpurfle) based on evolutionary algorithms to test proprietary Android system services. A key feature of this platform is the ability to profile coverage on the actual, unmodified Android device, by taking advantage of dynamic binary re-writing techniques. We applied this solution on three high-end commercial Android smartphones. The results confirmed that evolutionary fuzzing is able to test Android OS system services more efficiently than blind fuzzing. Furthermore, we evaluate the impact of different choices for the fitness function and selection algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. The source code and documentation of the Chizpurfle platform is available at https://github.com/dessertlab/fantastic_beasts

  2. Kruskal-Wallis test extends the Mann–Whitney test for more than two groups.

References

  • Android (2018) Android – Certified - Partners. https://www.android.com/certified/partners/

  • Android Open-Source Project (2016) Android Open Source Project. https://source.android.com/

  • Android Studio (2017) Android Debug Bridge. https://developer.android.com/studio/command-line/adb.html

  • AndroidXRef (2017a) Cross Reference: InputManager.java - injectInputEvent. http://androidxref.com/7.0.0_r1/xref/frameworks/base/core/java/android/hardware/input/InputManager.java#833

  • AndroidXRef (2017b) Cross Reference: Intent.java. http://androidxref.com/7.0.0_r1/xref/frameworks/base/core/java/android/content/Intent.java

  • AndroidXRef (2019) Cross Reference: AIDL service definitions. androidxref.com/7.0.0_r1/s?path=aidl&project=frameworks

  • Arcuri A (2018) Evomaster: Evolutionary multi-context automated system test generation. In: IEEE Conference on Software Testing, Validation and Verification

  • Arcuri A, Briand L (2014) A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability 24(3):219–250

    Article  Google Scholar 

  • Au KWY, Zhou YF, Huang Z, Lie D (2012) Pscout: Analyzing the android permission specification. In: Proc ACM Conf on Computer and Communications Security

  • Back T (1996) Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms. Oxford University Press, Oxford

    MATH  Google Scholar 

  • Bäck T, Hoffmeister F (1991) Extended selection mechanisms in genetic algorithms. Morgan Kaufmann, pp 92–99

  • Bhansali S, Chen WK, De Jong S, Edwards A, Murray R, Drinić M, Mihočka D, Chau J (2006) Framework for instruction-level tracing and analysis of program executions. In: Proc 2nd Intl Conference on Virtual Execution Environments, ACM

  • Böhme M, Pham VT, Roychoudhury A (2016) Coverage-based greybox fuzzing as markov chain. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, ACM, pp 1032–1043

  • Böhme M, Pham VT, Nguyen MD, Roychoudhury A (2017) Directed greybox fuzzing. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS’17), fix this citation

  • Bounimova E, Godefroid P, Molnar D (2013) Billions and billions of constraints: Whitebox fuzz testing in production. In: Proceedings of the 2013 International Conference on Software Engineering, IEEE Press, pp 122–131

  • Cadar C, Dunbar D, Engler DR, et al. (2008) KLEE: Unassisted And automatic generation of High-Coverage tests for complex systems programs. In: OSDI, vol 8. pp 209–224

  • Cao C, Gao N, Liu P, Xiang J (2015) Towards Analyzing the Input Validation Vulnerabilities associated with Android System Services. In: Proc 31st Annual Computer Security Applications Conf, ACM

  • Cotroneo D, Di Leo D, Fucci F, Natella R (2013a) Sabrine: State-based robustness testing of operating systems. In: Proc IEEE/ACM 28th Intl Conf Automated Software Engineering (ASE)

  • Cotroneo D, Grottke M, Natella R, Pietrantuono R, Trivedi KS (2013b) Fault Triggers in Open-Source Software: An Experience Report. In: Proc 24th Intl Symp Software Reliability Engineering (ISSRE)

  • Fabre JC, Salles F, Moreno MR, Arlat J (1999) Assessment of COTS microkernels by fault injection. In: Proc. Dependable Computing for Critical Applications 7

  • Feng H, Shin KG (2016) Understanding and defending the Binder attack surface in Android. In: Proc 32nd Annual Conf on Computer Security Applications, ACM

  • Fisher RA (1922) On the interpretation of χ 2 from contingency tables, and the calculation of p. J R Stat Soc 85(1):87–94

    Article  Google Scholar 

  • Fraser G (2017) EvoSuite — Automatic Test Suite Generation for Java. http://http://www.evosuite.org/

  • Fraser G, Arcuri A (2011a) Evosuite: automatic test suite generation for object-oriented software. In: Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, ACM, pp 416–419

  • Fraser G, Arcuri A (2011b) It is not the length that matters, it is how you control it. In: 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation (ICST), IEEE, pp 150–159

  • Fraser G, Arcuri A (2015) 1600 faults in 100 projects: Automatically finding faults while achieving high coverage with evosuite. Empir Softw Eng 20(3):611–639

    Article  Google Scholar 

  • Godefroid P, Levin MY, Molnar DA et al (2008) Automated whitebox fuzz testing. In: NDSS, vol 8

  • Goldberg DE, Deb K (1991) A comparative analysis of selection schemes used in genetic algorithms. In: Foundations of Genetic Algorithms, vol 1. Elsevier, pp 69–93

  • Google Inc (2017) OSS-Fuzz - Continuous Fuzzing for Open Source Software. https://github.com/google/oss-fuzz

  • Grefenstette JJ (1986) Optimization of control parameters for genetic algorithms. IEEE Trans Syst Man Cybern 16(1):122–128

    Article  Google Scholar 

  • Harman M, McMinn P, De Souza JT, Yoo S (2012) Search based software engineering: Techniques, taxonomy, tutorial. In: Empirical Software Engineering and Verification, Springer, pp 1–59

  • Hu Y, Neamtiu I (2016) Fuzzy and cross-app replay for smartphone apps. In: Proc 11th Intl Wksp, Automation of Software Test, ACM

  • Iannillo AK, Natella R, Cotroneo D, Nita-Rotaru C (2017) Chizpurfle: A gray-box android fuzzer for vendor service customizations. In: Proc 28th Intl Symp on Software Reliability Engineering (ISSRE), fix this citation

  • Kanoun K, Crouzet Y, Kalakech A, Rugina AE, Rumeau P (2005) Benchmarking the dependability of Windows and Linux using PostMark workloads. In: Proc 16th IEEE Intl Symp on Software Reliability Engineering

  • Klees G, Ruef A, Cooper B, Wei S, Hicks M (2018) Evaluating fuzz testing. In: Proc ACM SIGSAC Conf on Computer and Communications Security, pp 2123–2138

  • Kochhar PS, Thung F, Lo D (2015) Code coverage and test suite effectiveness: Empirical study with real bugs in large systems. In: 2015 IEEE 22nd International Conference on Software Analysis, Evolution and Reengineering (SANER), IEEE, pp 560–564

  • Koopman P, DeVale J (2000) The exception handling effectiveness of POSIX operating systems. IEEE Trans Softw Eng 26(9):837–848

    Article  Google Scholar 

  • Kruskal WH, Wallis WA (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47(260):583–621

    Article  Google Scholar 

  • Levin J (2015) Android internals:: A confectioner’s cookbook

  • Luk CK, Cohn R, Muth R, Patil H, Klauser A, Lowney G, Wallace S, Reddi VJ, Hazelwood K (2005) Pin: Building customized program analysis tools with dynamic instrumentation. In: Acm Sigplan Notices, ACM, vol 40. pp 190–200

    Article  Google Scholar 

  • Mahmood R, Esfahani N, Kacem T, Mirzaei N, Malek S, Stavrou A (2012) A whitebox approach for automated security testing of android applications on the cloud. In: Proc 7th Intl Wksp Automation of Software Test (AST), IEEE

  • Maji AK, Arshad FA, Bagchi S, Rellermeyer JS (2012) An empirical study of the robustness of Inter-Component communication in android. In: Proc IEEE/IFIP Intl Conf on Dependable Systems and Networks (DSN)

  • Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics pp 50–60

  • Michal Zalewski (2016) American Fuzzy Lop (AFL). http://lcamtuf.coredump.cx/afl/

  • Miller BP, Fredriksen L, So B (1990) An empirical study of the reliability of UNIX utilities. Commun ACM 33(12):32–44

    Article  Google Scholar 

  • Mitchell M, Forrest S, Holland JH (1992) The royal road for genetic algorithms: Fitness landscapes and ga performance. In: Proceedings of the first European Conference on Artificial Life, pp 245–254

  • Mulliner C, Miller C (2009) Fuzzing the Phone in your Phone. Black Hat, USA

    Google Scholar 

  • Naudts B, Kallel L (2000) A comparison of predictive measures of problem difficulty in evolutionary algorithms. IEEE Trans Evol Comput 4(1):1–15

    Article  Google Scholar 

  • Nethercote N, Seward J (2007) Valgrind: A framework for heavyweight dynamic binary instrumentation. In: ACM Sigplan notices, ACM, vol 42

    Article  Google Scholar 

  • Ravnås OAV (2017) F R IDA. https://www.frida.re

  • Petsios T, Zhao J, Keromytis AD, Jana S (2017) Slowfuzz: Automated domain-independent detection of algorithmic complexity vulnerabilities. arXiv:170808437 Fix this citation

  • Rawat S, Jain V, Kumar A, Cojocar L, Giuffrida C, Bos H (2017) Vuzzer: Application-aware evolutionary fuzzing. In: Proceedings of the Network and Distributed System Security Symposium (NDSS)

  • Samsung (2017a) WE VoIP Application for Business. http://www.samsung.com/us/business/business-communication-systems/unified-communication-solutions/IPX-LSMP/STD

  • Samsung (2017b) What are the advantages of S Pen. http://www.samsung.com/global/galaxy/what-is/s-pen/

  • Sasnauskas R, Regehr J (2014) Intent fuzzer: Crafting intents of death. in: Proc. Joint intl. Wksp on dynamic analysis (WODA) and Software and System Performance Testing, Debugging, and Analytics (PERTEA)

  • Stack Overflow (2016) What is SystemServer for Android?. https://stackoverflow.com/questions/34651015/what-is-systemserver-for-android

  • Statista (2018) Global market share held by the leading smartphone operating systems in sales to end users from 1st quarter 2009 to 2nd quarter 2017. https://www.statista.com/statistics/266136/global-market-share-held-by-smartphone-operating-systems/

  • Vargha A, Delaney HD (2000) A critique and improvement of the cl common language effect size statistics of mcgraw and wong. J Educ Behav Stat 25(2):101–132

    Google Scholar 

  • Veggalam S, Rawat S, Haller I, Bos H (2016) Ifuzzer: An evolutionary interpreter fuzzer using genetic programming. In: European Symposium on Research in Computer Security, Springer, pp 581–601

  • Whitley D (2001) An overview of evolutionary algorithms: practical issues and common pitfalls. Inf Softw Technol 43(14):817–831

    Article  Google Scholar 

  • Xu M, Song C, Ji Y, Shih MW, Lu K, Zheng C, Duan R, Jang Y, Lee B, Qian C et al (2016) Toward engineering a secure android ecosystem: a survey of existing techniques. ACM Comput Surv (CSUR) 49(2):38

    Article  Google Scholar 

  • Yaghmour K (2013) Embedded android: Porting, extending and customizing. O’Reilly Media, Inc

  • Yang K, Zhuge J, Wang Y, Zhou L, Duan H (2014) IntentFuzzer: detecting capability leaks of android applications. In: Proc 9th ACM Symp on Information, Computer and Communications Security

  • Ye H, Cheng S, Zhang L, Jiang F (2013) Droidfuzzer: Fuzzing the android apps with intent-filter tag. In: Proc Intl Conference on Advances in Mobile Computing & Multimedia

Download references

Acknowledgments

This research was carried out in the frame of Programme STAR, financially supported by UniNA and Compagnia di San Paolo.

Author information

Authors and Affiliations

  1. Università degli Studi di Napoli Federico II, via Claudio 21, 80125, Napoli, Italy

    Domenico Cotroneo, Antonio Ken Iannillo & Roberto Natella

  2. SnT, University of Luxembourg, 29 avenue J.F. Kennedy, 1855, Esch-sur-Alzette, Luxembourg

    Antonio Ken Iannillo

Authors
  1. Domenico Cotroneo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio Ken Iannillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roberto Natella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roberto Natella.

Additional information

Communicated by: David Lo, Meiyappan Nagappan, Fabio Palomba, and Sebastiano Panichella

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cotroneo, D., Iannillo, A.K. & Natella, R. Evolutionary Fuzzing of Android OS Vendor System Services. Empir Software Eng 24, 3630–3658 (2019). https://doi.org/10.1007/s10664-019-09725-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-019-09725-6

Keywords

Search

Navigation