Skip to main content
SpringerLink
Log in

Generalized Dynamic Opaque Predicates: A New Control Flow Obfuscation Method

  • Conference paper
  • First Online:
Information Security (ISC 2016)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 9866))

Included in the following conference series:

Abstract

Opaque predicate obfuscation, a low-cost and stealthy control flow obfuscation method to introduce superfluous branches, has been demonstrated to be effective to impede reverse engineering efforts and broadly used in various areas of software security. Conventional opaque predicates typically rely on the invariant property of well-known number theoretic theorems, making them easy to be detected by the dynamic testing and formal semantics techniques. To address this limitation, previous work has introduced the idea of dynamic opaque predicates, whose values may vary in different runs. However, the systematical design and evaluation of dynamic opaque predicates are far from mature. In this paper, we generalize the concept and systematically develop a new control flow obfuscation scheme called generalized dynamic opaque predicates. Compared to the previous work, our approach has two distinct advantages: (1) We extend the application scope by automatically transforming more common program structures (e.g., straight-line code, branch, and loop) into dynamic opaque predicates; (2) Our system design does not require that dynamic opaque predicates to be strictly adjacent, which is more resilient to the deobfuscation techniques. We have developed a prototype tool based on LLVM IR and evaluated it by obfuscating the GNU core utilities. Our experimental results show the efficacy and generality of our method. In addition, the comparative evaluation demonstrates that our method is resilient to the latest formal program semantics-based opaque predicate detection method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We do not consider dynamic link library functions because our approach takes the target program source code as input.

  2. 2.

    http://www.zynamics.com/bindiff.html.

References

  1. Arboit, G.: A method for watermarking Java programs via opaque predicates. In: Proceedings of 5th International Conference on Electronic Commerce Research (ICECR-5) (2002)

    Google Scholar 

  2. Bruschi, D., Martignoni, L., Monga, M.: Detecting self-mutating malware usingcontrol-flow graph matching. In: Proceedings of Detection of Intrusions and Malware and Vulnerability Assessment (DIMVA 2006) (2006)

    Google Scholar 

  3. Bruschi, D., Martignoni, L., Monga, M.: Code normalization for self-mutating malware. IEEE Secur. Priv. 5(2), 46–54 (2007)

    Article  Google Scholar 

  4. Cappaert, J., Preneel, B.: A general model for hiding control flow. In: Proceedings of the 10th Annual ACM Workshop on Digital Rights Management (DRM 2010) (2010)

    Google Scholar 

  5. Chen, H., Yuan, L., Wu, X., Zang, B., Huang, B., Yew, P.C.: Control flow obfuscation with information flow tracking. In: Proceedings of the 42nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO 42) (2009)

    Google Scholar 

  6. Collberg, C., Thomborson, C., Low, D.: A taxonomy of obfuscating transformations. The University of Auckland, Technical report (1997)

    Google Scholar 

  7. Collberg, C., Thomborson, C., Low, D.: Manufacturing cheap, resilient, and stealthy opaque constructs. In: Proceedings of the 25th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL 1998) (1998)

    Google Scholar 

  8. Collberg, C., Myles, G., Huntwork, A.: Sandmark-a tool for software protection research. IEEE Secur. Priv. 1(4), 40–49 (2003)

    Article  Google Scholar 

  9. Conte, S.D., Dunsmore, H.E., Shen, V.Y.: Software Engineering Metrics and Models. Benjamin-Cummings Publishing Co. Inc., REdwood City (1986)

    Google Scholar 

  10. Coppens, B., De Sutter, B., Maebe, J.: Feedback-driven binary code diversification. ACM Trans. Architect. Code Optim. (TACO) 9(4), 24:1–24:26 (2013)

    Google Scholar 

  11. Drape, S.: Intellectual property protection using obfuscation. Technical report, RR-10-02, Oxford University Computing Laboratory (2010)

    Google Scholar 

  12. Hind, M., Pioli, A.: Which pointer analysis should i use?. In: Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2000), pp. 113–123. ACM (2000)

    Google Scholar 

  13. Junod, P., Rinaldini, J., Wehrli, J., Michielin, J.: Obfuscator-LLVM - software protection for the masses. In: Proceedings of the 1st International Workshop on Software Protection (SPRO 2015) (2015)

    Google Scholar 

  14. Kovacheva, A.: Efficient code obfuscation for Android. Master’s thesis, University of Luxembourg (2013)

    Google Scholar 

  15. Larsen, P., Homescu, A., Brunthaler, S., Franz, M.: SoK: automated software diversity. In: Proceedings of the 2014 IEEE Symposium on Security and Privacy (SP 2014) (2014)

    Google Scholar 

  16. Lattner, C., Adve, V.: LLVM: a compilation framework for lifelong program analysis and transformation. In: Proceedings of the International Symposium on Code Generation and Optimization (CGO 2004) (2004)

    Google Scholar 

  17. Madou, M.: Application security through program obfuscation. Ph.D. thesis, Ghent University (2007)

    Google Scholar 

  18. Madou, M., Van Put, L., De Bosschere, K.: LOCO: an interactive code (de)obfuscation tool. In: Proceedings of the 2006 ACM SIGPLAN Symposium on Partial Evaluation and Semantics-Based Program Manipulation (PEPM 2006) (2006)

    Google Scholar 

  19. Ming, J., Xu, D., Wang, L., Wu, D.: LOOP: logic-oriented opaque predicate detection in obfuscated binary code. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS 2015) (2015)

    Google Scholar 

  20. Moser, A., Kruegel, C., Kirda, E.: Limits of static analysis for malware detection. In: Proceedings of the 23th Annual Computer Security Applications Conference (ACSAC 2007), December 2007

    Google Scholar 

  21. Myles, G., Collberg, C.: Software watermarking via opaque predicates: implementation, analysis, and attacks. Electron. Commer. Res. 6(2), 155–171 (2006)

    Article  Google Scholar 

  22. Palsberg, J., Krishnaswamy, S., Kwon, M., Ma, D., Shao, Q., Zhang, Y.: Experience with software watermarking. In: Proceedings of the 16th Annual Computer Security Applications Conference (ACSAC 2000) (2000)

    Google Scholar 

  23. Preda, M.D., Madou, M., Bosschere, K.D., Giacobazzi, R.: Opaque predicate detection by abstract interpretation. In: Proceedings of 11th International Conference on Algebriac Methodology and Software Technology (AMAST 2006) (2006)

    Google Scholar 

  24. Roundy, K.A., Miller, B.P.: Binary-code obfuscations in prevalent packer tools. ACM J. Name 1, 21 (2012)

    Google Scholar 

  25. Udupa, S.K., Debray, S.K., Madou, M.: Deobfuscation: Reverse engineering obfuscated code. In: Proceedings of the 12th Working Conference on Reverse Engineering (WCRE 2005) (2005)

    Google Scholar 

  26. Wang, C., Hill, J., Knight, J.C., Davidson, J.W.: Protection of software-based survivability mechanisms. In: Proceedings of the 2001 International Conference on Dependable Systems and Networks (DSN 2001) (2001)

    Google Scholar 

Download references

Acknowledgements

We thank the anonymous reviewers for their valuable feedback. This research was supported in part by the National Science Foundation (NSF) grants CNS-1223710 and CCF-1320605, and the Office of Naval Research (ONR) grants N00014-13-1-0175 and N00014-16-1-2265.

Author information

Authors and Affiliations

  1. College of Information Sciences and Technology, The Pennsylvania State University, University Park, PA, 16802, USA

    Dongpeng Xu, Jiang Ming & Dinghao Wu

Authors
  1. Dongpeng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiang Ming
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dinghao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dinghao Wu .

Editor information

Editors and Affiliations

  1. University of California , Davies, USA

    Matt Bishop

  2. University of Washington , Tacoma, Washington, USA

    Anderson C A Nascimento

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Xu, D., Ming, J., Wu, D. (2016). Generalized Dynamic Opaque Predicates: A New Control Flow Obfuscation Method. In: Bishop, M., Nascimento, A. (eds) Information Security. ISC 2016. Lecture Notes in Computer Science(), vol 9866. Springer, Cham. https://doi.org/10.1007/978-3-319-45871-7_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-45871-7_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-45870-0

  • Online ISBN: 978-3-319-45871-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation