Skip to main content
SpringerLink
Log in

Empirical assessment of generating adversarial configurations for software product lines

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Software product line (SPL) engineering allows the derivation of products tailored to stakeholders’ needs through the setting of a large number of configuration options. Unfortunately, options and their interactions create a huge configuration space which is either intractable or too costly to explore exhaustively. Instead of covering all products, machine learning (ML) approximates the set of acceptable products (e.g., successful builds, passing tests) out of a training set (a sample of configurations). However, ML techniques can make prediction errors yielding non-acceptable products wasting time, energy and other resources. We apply adversarial machine learning techniques to the world of SPLs and craft new configurations faking to be acceptable configurations but that are not and vice-versa. It allows to diagnose prediction errors and take appropriate actions. We develop two adversarial configuration generators on top of state-of-the-art attack algorithms and capable of synthesizing configurations that are both adversarial and conform to logical constraints. We empirically assess our generators within two case studies: an industrial video synthesizer (MOTIV) and an industry-strength, open-source Web-app configurator (JHipster). For the two cases, our attacks yield (up to) a 100% misclassification rate without sacrificing the logical validity of adversarial configurations. This work lays the foundations of a quality assurance framework for ML-based SPLs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Notes

  1. Most common functions are linear, radial-based functions and polynomial

  2. https://secml.gitlab.io/index.html

  3. Therefore it was not available in our previous SPLC’19 contribution (Temple et al. 2019)

  4. https://secml.gitlab.io/tutorials.adv.html

  5. https://secml.gitlab.io/secml.adv.attacks.evasion.html

  6. https://secml.gitlab.io/secml.adv.attacks.evasion.html#module-secml.adv.attacks.evasion.cleverhans.c_attack_evasion_cleverhans

  7. https://secml.gitlab.io/tutorials/09-Cleverhans.html

  8. http://yeoman.io/

  9. https://ejs.co/

  10. https://github.com/templep/EMSE_2020

  11. https://github.com/templep/EMSE_2019

  12. Such tables can be found easily on the Internet: http://ocw.umb.edu/psychology/psych-270/other-materials/RelativeResourceManager.pdf

  13. Except when d_max is set to 0.1 and for which we do not have any explanation.

  14. Note that the baselines are reported for two different models; secML provides a complete library which comes with its own framework and pipeline, necessitating to learn a classifier with this library. The implementation can differ from the ones provided by scikit-learn which is the other library we have used before using secML.

References

  • Acher M, Cleve A, Perrouin G, Heymans P, Vanbeneden C, Collet P, Lahire P (2012) On extracting feature models from product descriptions. In: Proceedings of the sixth international workshop on variability modeling of software-intensive systems, VaMoS ’12. ACM, New York, pp 45–54. https://doi.org/10.1145/2110147.2110153. http://doi.acm.org/10.1145/2110147.2110153

  • Acher M, Temple P, Jezequel JM, Galindo JA, Martinez J, Ziadi T (2018) Varylatex: learning paper variants that meet constraints. In: Proceedings of the 12th international workshop on variability modelling of software-intensive systems. ACM, pp 83–88

  • Al-Hajjaji M, Benduhn F, Thüm T, Leich T, Saake G (2016) Mutation operators for preprocessor-based variability. In: Proceedings of the tenth international workshop on variability modelling of software-intensive systems, Salvador, Brazil, January 27–29, 2016, pp 81–88. https://doi.org/10.1145/2866614.2866626

  • Alférez M, Acher M, Galindo JA, Baudry B, Benavides D (2019) Modeling variability in the video domain: language and experience report. Softw Qual J 27(1):307–347. https://doi.org/10.1007/s11219-017-9400-8

    Article  Google Scholar 

  • Alves Pereira J, Acher M, Martin H, Jézéquel JM (2020) Sampling effect on performance prediction of configurable systems: a case study. In: 11th International conference on performance engineering (ICPE’20). https://hal.inria.fr/hal-02356290

  • Amand B, Cordy M, Heymans P, Acher M, Temple P, Jézéquel J M (2019) Towards learning-aided configuration in 3d printing: feasibility study and application to defect prediction. In: Proceedings of the 13th international workshop on variability modelling of software-intensive systems. ACM, p 7

  • Barreno M, Nelson B, Sears R, Joseph AD, Tygar JD (2006) Can machine learning be secure?. In: Proceedings of the 2006 ACM symposium on information, computer and communications security. ACM, New York, pp 16–25

  • Batory DS (2005) Feature models, grammars, and propositional formulas. In: SPLC’05, LNCS, vol 3714. Springer, Berlin, pp 7–20

  • Bécan G, Behjati R, Gotlieb A, Acher M (2015) Synthesis of attributed feature models from product descriptions. In: SPLC’15

  • Bellman R (1957) Dynamic programming, 1st edn. Princeton University Press, Princeton

    Google Scholar 

  • Benavides D, Segura S, Ruiz-Cortes A (2010) Automated analysis of feature models 20 years later: a literature review. Inf Syst 35(6):615–636

    Article  Google Scholar 

  • Berger T, Rublack R, Nair D, Atlee JM, Becker M, Czarnecki K, Węsowski A (2013) A survey of variability modeling in industrial practice. In: Proceedings of the seventh international workshop on variability modelling of software-intensive systems, VaMoS ’13. ACM, New York, pp 7:1–7:8. https://doi.org/10.1145/2430502.2430513. http://doi.acm.org/10.1145/2430502.2430513

  • Biggio B, Roli F (2018) Wild patterns: ten years after the rise of adversarial machine learning. Pattern Recognit 84:317–331

    Article  Google Scholar 

  • Biggio B, Nelson B, Laskov P (2012) Poisoning attacks against support vector machines. In: Proceedings of the 29th international conference on international conference on machine learning, ICML’12. Omnipress, pp 1467–1474. http://dl.acm.org/citation.cfm?id=3042573.3042761

  • Biggio B, Corona I, Maiorca D, Nelson B, Šrndić N, Laskov P, Giacinto G, Roli F (2013a) Evasion attacks against machine learning at test time. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, Berlin, pp 387–402

  • Biggio B, Didaci L, Fumera G, Roli F (2013b) Poisoning attacks to compromise face templates. In: 2013 International conference on biometrics (ICB). IEEE, New York, pp 1–7. https://doi.org/10.1109/ICB.2013.6613006

  • Biggio B, Fumera G, Roli F (2014a) Pattern recognition systems under attack: design issues and research challenges. Int J Pattern Recognit Artif Intell 28(7):1460002

    Article  Google Scholar 

  • Biggio B, Fumera G, Roli F (2014b) Security evaluation of pattern classifiers under attack. IEEE Trans Knowl Data Eng 26(4):984–996

    Article  Google Scholar 

  • Bodden E, Tolêdo T, Ribeiro M, Brabrand C, Borba P, Mezini M (2013) Spllift: statically analyzing software product lines in minutes instead of years. In: ACM SIGPLAN conference on programming language design and implementation, PLDI ’13, Seattle, WA, USA, June 16–19, 2013. ACM, New York, pp 355–364. https://doi.org/10.1145/2491956.2491976. http://doi.acm.org/10.1145/2491956.2491976

  • Boucher Q, Classen A, Faber P, Heymans P (2010) Introducing tvl, a text-based feature modelling. In: Benavides D, Batory DS, Grünbacher P (eds) Fourth international workshop on variability modelling of software-intensive systems, Linz, Austria, January 27–29, 2010. Proceedings, ICB-Research Report, vol 37. Universität Duisburg-Essen, Essen, pp 159–162. http://www.vamos-workshop.net/proceedings/VaMoS_2010_Proceedings.pdf

  • Brown T, Mane D, Roy A, Abadi M, Gilmer J (2017) Adversarial patch. https://arxiv.org/pdf/1712.09665.pdf

  • Carvalho L, Guimarães MA, Ribeiro M, Fernandes L, Al-Hajjaji M, Gheyi R, Thüm T (2018) Equivalent mutants in configurable systems:an empirical study. In: Proceedings of the 12th international workshop on variability modelling of software-intensive systems, VAMOS 2018, Madrid, Spain, February 7–9, 2018, pp 11–18. https://doi.org/10.1145/3168365.3168379

  • Chakraborty S, Fremont DJ, Meel KS, Seshia SA, Vardi MY (2015) On parallel scalable uniform SAT witness generation. In: Tools and algorithms for the construction and analysis of systems TACAS’15 2015, London, UK, April 11–18, 2015. Proceedings, pp 304–319

  • Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

    Article  Google Scholar 

  • Classen A, Boucher Q, Heymans P (2011) A text-based approach to feature modelling: syntax and semantics of TVL. Sci Comput Program Spec Iss Softw Evol Adapt Var 76(12):1130–1143

    Google Scholar 

  • Clements P, Northrop LM (2001) Software product lines: practices and patterns. Addison-Wesley Professional, Boston

    Google Scholar 

  • Cohen MB, Dwyer MB, Society IC (2008) Constructing interaction test suites for highly-configurable systems in the presence of constraints : a greedy approach. 34, . IEEE Trans Softw Eng 34:633–650

    Article  Google Scholar 

  • Davril JM, Heymans P, Bécan G, Acher M (2015) On breaking the curse of dimensionality in reverse engineering feature models. In: 17th international configuration workshop, Vienna. https://hal.inria.fr/hal-01243571

  • Demontis A, Melis M, Pintor M, Jagielski M, Biggio B, Oprea A, Nita-Rotaru C, Roli F (2018) On the intriguing connections of regularization, input gradients and transferability of evasion and poisoning attacks. CoRR arXiv:1809.02861

  • Demontis A, Melis M, Pintor M, Jagielski M, Biggio B, Oprea A, Nita-Rotaru C, Roli F (2019) Why do adversarial attacks transfer? explaining transferability of evasion and poisoning attacks. In: 28th USENIX Security Symposium (USENIX Security 19). USENIX Association, Santa Clara. https://www.usenix.org/conference/usenixsecurity19/presentation/demontis

  • Dhillon GS, Azizzadenesheli K, Lipton ZC, Bernstein J, Kossaifi J, Khanna A, Anandkumar A (2018) Stochastic activation pruning for robust adversarial defense. arXiv:1803.01442

  • Dosselman RW, Yang XD (2012) No-reference noise and blur detection via the fourier transform. Tech. rep., University of Regina, Canada

  • Elsayed GF, Shankar S, Cheung B, Papernot N, Kurakin A, Goodfellow I, Sohl-Dickstein J (2018) Adversarial examples that fool both human and computer vision. arXiv:1802.08195

  • Evtimov I, Eykholt K, Fernandes E, Kohno T, Li B, Prakash A, Rahmati A, Song D (2017) Robust physical-world attacks on deep learning models, 1. arXiv:1707.08945

  • Galindo JA, Alférez M, Acher M, Baudry B, Benavides D (2014) A variability-based testing approach for synthesizing video sequences. In: International symposium on software testing and analysis, ISSTA 2014. ACM, pp 293–303. https://doi.org/10.1145/2610384.2610411. http://doi.acm.org/10.1145/2610384.2610411

  • Galindo Duarte JA, Alférez M, Acher M, Baudry B, Benavides D (2014) A variability-based testing approach for synthesizing video sequences. In: ISSTA ’14: international symposium on software testing and analysis, San José. https://hal.inria.fr/hal-01003148

  • Gargantini A, Petke J, Radavelli M (2017) Combinatorial interaction testing for automated constraint repair. In: 2017 IEEE international conference on software testing, verification and validation workshops (ICSTW). IEEE, pp 239–248

  • Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  • Guo J, Czarnecki K, Apel S, Siegmund N, Wasowski A (2013) Variability-aware performance prediction: a statistical learning approach. In: ASE, vol 55, pp 491–507

  • Guo C, Rana M, Cisse M, van der Maaten L (2017) Countering adversarial images using input transformations. arXiv:1711.00117

  • Halin A, Nuttinck A, Acher M, Devroey X, Perrouin G, Baudry B (2018) Test them all, is it worth it? Assessing configuration sampling on the jhipster web development stack. Empir Softw Eng. Empirical Software Engineering Journal. https://doi.org/10.07980. https://hal.inria.fr/hal-01829928

  • Halin A, Nuttinck A, Acher M, Devroey X, Perrouin G, Baudry B (2019) Test them all, is it worth it? Assessing configuration sampling on the jhipster web development stack. Empir Softw Eng 24(2):674–717. https://doi.org/10.1007/s10664-018-9635-4

    Article  Google Scholar 

  • Ierusalimschy R (2006) Programming in Lua, 2nd edn Lua.Org

  • JHipsterTeam: Jhipster Website (2020) https://jhipster.github.io. Accessed Jan 2020

  • Johansen MF, Haugen OY, Fleurey F (2012) An algorithm for generating t-wise covering arrays from large feature models SPLC’12

  • Kaltenecker C, Grebhahn A, Siegmund N, Guo J, Apel S (2019) Distance-based sampling of software configuration spaces. In: Proceedings of the IEEE/ACM international conference on software engineering (ICSE). ACM

  • Kaner C, Bach J, Pettichord B (2001) Lessons learned in software testing. Wiley, New York

    Google Scholar 

  • Kang KC, Cohen SG, Hess JA, Novak WE, Peterson AS (1990) Feature-oriented domain analysis (FODA) feasibility study. Tech. rep., DTIC Document

  • Knüppel A, Thüm T, Mennicke S, Meinicke J, Schaefer I (2018) Is there a mismatch between real-world feature models and product-line research? In: Tichy M, Bodden E, Kuhrmann M, Wagner S, Steghöfer J (eds) Software Engineering und Software Management 2018, Fachtagung des GI-Fachbereichs Softwaretechnik, SE 2018, 5.-9. März 2018, Ulm, Germany. LNI, vol P-279. Gesellschaft für Informatik, pp 53–54. https://dl.gi.de/20.500.12116/16312

  • Krismayer T, Rabiser R, Grünbacher P (2017) Mining constraints for event-based monitoring in systems of systems. In: ASE. IEEE Press, pp 826–831

  • Kurakin A, Goodfellow I, Bengio S (2016) Adversarial examples in the physical world. arXiv:1607.02533

  • Legay A, Perrouin G (2017) On quantitative requirements for product lines. In: Proceedings of the eleventh international workshop on variability modelling of software-intensive systems, VAMOS ’17. ACM, New York, pp 2–4. https://doi.org/10.1145/3023956.3023970. http://doi.acm.org/10.1145/3023956.3023970

  • Lopez-Herrejon RE, Galindo JA, Benavides D, Segura S, Egyed A (2012) Reverse engineering feature models with evolutionary algorithms: an exploratory study. In: SSBSE’12, LNCS, vol 7515. Springer, pp 168–182

  • Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2017) Towards deep learning models resistant to adversarial attacks. arXiv:1706.06083

  • Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18:50–60

    Article  MathSciNet  Google Scholar 

  • Medeiros F, Kästner C, Ribeiro M, Gheyi R, Apel S (2016) A comparison of 10 sampling algorithms for configurable systems. In: Proceedings of the 38th international conference on software engineering, ICSE ’16. ACM, New York, pp 643–654. https://doi.org/10.1145/2884781.2884793. http://doi.acm.org/10.1145/2884781.2884793

  • Nadi S, Berger T, Kästner C, Czarnecki K (2014) Mining configuration constraints: static analyses and empirical results. In: 36th International conference on software engineering, ICSE ’14, Hyderabad, India—May 31–June 07, 2014, pp 140–151. https://doi.org/10.1145/2568225.2568283. http://doi.acm.org/10.1145/2568225.2568283

  • Nelson B, Barreno M, Chi FJ, Joseph AD, Rubinstein BI, Saini U, Sutton CA, Tygar JD, Xia K (2008) Exploiting machine learning to subvert your spam filter. LEET 8:1–9

    Google Scholar 

  • Oh J, Batory DS, Myers M, Siegmund N (2017a) Finding near-optimal configurations in product lines by random sampling. In: Proceedings of the 2017 11th joint meeting on foundations of software engineering, ESEC/FSE 2017, Paderborn, Germany, September 4–8, 2017, pp 61–71. https://doi.org/10.1145/3106237.3106273. http://doi.acm.org/10.1145/3106237.3106273

  • Oh J, Batory DS, Myers M, Siegmund N (2017b) Finding near-optimal configurations in product lines by random sampling. In: Proceedings of the 2017 11th joint meeting on foundations of software engineering, ESEC/FSE 2017, Paderborn, Germany, September 4–8, 2017, pp 61–71

  • Papernot N, McDaniel P, Jha S, Fredrikson M, Celik ZB, Swami A (2016) The limitations of deep learning in adversarial settings. In: 2016 IEEE European symposium on security and privacy (EuroS P), pp 372–387. https://doi.org/10.1109/EuroSP.2016.36

  • Pascual GG, Lopez-Herrejon RE, Pinto M, Fuentes L, Egyed A (2015) Applying multiobjective evolutionary algorithms to dynamic software product lines for reconfiguring mobile applications. J Syst Softw 103:392–411

    Article  Google Scholar 

  • Pei K, Cao Y, Yang J, Jana S (2017a) Deepxplore: automated whitebox testing of deep learning systems. In: Proceedings of the 26th symposium on operating systems principles, SOSP ’17. ACM, New York, pp 1–18. https://doi.org/10.1145/3132747.3132785. http://doi.acm.org/10.1145/3132747.3132785

  • Pei K, Cao Y, Yang J, Jana S (2017b) Deepxplore: automated whitebox testing of deep learning systems. In: Proceedings of the 26th symposium on operating systems principles, SOSP ’17. ACM, New York, pp 1–18. https://doi.org/10.1145/3132747.3132785. http://doi.acm.org/10.1145/3132747.3132785

  • Pereira JA, Martin H, Acher M, Jézéquel J M, Botterweck G, Ventresque A (2019) Learning software configuration spaces: a systematic literature review

  • Plazar Q, Acher M, Perrouin G, Devroey X, Cordy M (2019a) Uniform sampling of SAT solutions for configurable systems: are we there yet?. In: 12th IEEE conference on software testing, validation and verification, ICST 2019, Xi’an, China, April 22–27, 2019, pp 240–251. https://doi.org/10.1109/ICST.2019.00032

  • Plazar Q, Acher M, Perrouin G, Devroey X, Cordy M (2019b) Uniform sampling of sat solutions for configurable systems: are we there yet?. In: ICST 2019—12th international conference on software testing, verification, and validation, Xian, pp 1–12. https://hal.inria.fr/hal-01991857

  • Pohl K, Böckle G, van der Linden FJ (2005) Software product line engineering: foundations, principles and techniques. Springer, Berlin

    Book  Google Scholar 

  • R Core Team (2020) R: a language and environment for statistical computing, R Foundation for Statistical Computing, Vienna. https://www.R-project.org/

  • Raible M (2015) The JHipster mini-book. C4Media

  • Sarkar A, Guo J, Siegmund N, Apel S, Czarnecki K (2015) Cost-efficient sampling for performance prediction of configurable systems (t). In: ASE’15

  • Schobbens PY, Heymans P, Trigaux JC (2006) Feature diagrams: a survey and a formal semantics. In: RE ’06: proceedings of the 14th IEEE international requirements engineering conference (RE’06). IEEE Computer Society, Washington, DC, pp 136–145. https://doi.org/10.1109/RE.2006.23

  • Schobbens PY, Heymans P, Trigaux JC, Bontemps Y (2007) Generic semantics of feature diagrams. Comput Netw 51(2):456–479

    Article  Google Scholar 

  • Sharif M, Bhagavatula S, Bauer L, Reiter MK (2016) Accessorize to a crime: real and stealthy attacks on state-of-the-art face recognition. In: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. ACM, pp 1528–1540

  • She S, Lotufo R, Berger T, Wasowski A, Czarnecki K (2011) Reverse engineering feature models. In: ICSE, pp 461–470

  • She S, Ryssel U, Andersen N, Wasowski A, Czarnecki K (2014) Efficient synthesis of feature models. Inf Softw Technol 56(9):106–115

    Article  Google Scholar 

  • Siegmund N, RosenmüLler M, KäStner C, Giarrusso PG, Apel S, Kolesnikov SS (2013) Scalable prediction of non-functional properties in software product lines: Footprint and memory consumption. Inf Softw Technol 55: 491–507

  • Siegmund N, Grebhahn A, Kästner C, Apel S (2015) Performance-influence models for highly configurable systems. In: ESEC/FSE’15

  • Siegmund N, Sobernig S, Apel S (2017) Attributed variability models: outside the comfort zone. In: Proceedings of the 2017 11th joint meeting on foundations of software engineering, ESEC/FSE 2017. ACM, New York, pp 268–278. https://doi.org/10.1145/3106237.3106251. http://doi.acm.org/10.1145/3106237.3106251

  • Strüber D, Rubin J, Arendt T, Chechik M, Taentzer G, Plöger J (2018) Variability-based model transformation: formal foundation and application. Formal Asp Comput 30(1):133–162. https://doi.org/10.1007/s00165-017-0441-3

    Article  MathSciNet  Google Scholar 

  • Temple P, Galindo Duarte JA, Acher M, Jézéquel JM (2016) Using machine learning to infer constraints for product lines. In: Software Product Line Conference (SPLC), Beijing. https://doi.org/10.1145/2934466.2934472. https://hal.inria.fr/hal-01323446

  • Temple P, Acher M, Jézéquel J, Barais O (2017) Learning contextual-variability models. IEEE Softw 34(6):64–70. https://doi.org/10.1109/MS.2017.4121211

    Article  Google Scholar 

  • Temple P, Acher M, Perrouin G, Biggio B, Jezequel JM, Roli F (2019) Towards quality assurance of software product lines with adversarial configurations. In: Proceedings of the 23rd international systems and software product line conference—Volume A, SPLC ’19. ACM, New York, pp 277–288. https://doi.org/10.1145/3336294.3336309. http://doi.acm.org/10.1145/3336294.3336309

  • ter Beek MH, Legay A (2019) Quantitative variability modeling and analysis. In: Proceedings of the 13th international workshop on variability modelling of software-intensive systems, VAMOS ’19. ACM, New York, pp 13:1–13:2. https://doi.org/10.1145/3302333.3302349. http://doi.acm.org/10.1145/3302333.3302349

  • ter Beek MH, Fantechi A, Gnesi S, Mazzanti F (2016a) Modelling and analysing variability in product families: model checking of modal transition systems with variability constraints. J Log Algebr Meth Program 85(2):287–315. https://doi.org/10.1016/j.jlamp.2015.11.006

    Article  MathSciNet  Google Scholar 

  • ter Beek MH, Fantechi A, Gnesi S, Semini L (2016b) Variability-based design of services for smart transportation systems. In: Leveraging Applications of formal methods, verification and validation: discussion, dissemination, applications—7th international symposium, ISoLA 2016, Imperial, Corfu, Greece, October 10-14, 2016, Proceedings, Part II, pp 465–481. https://doi.org/10.1007/978-3-319-47169-3_38

  • Thüm T, Apel S, Kästner C, Schaefer I, Saake G (2014) A classification and survey of analysis strategies for software product lines. ACM Comput Surv

  • Tian Y, Pei K, Jana S, Ray B (2018) Deeptest: automated testing of deep-neural-network-driven autonomous cars. In: Proceedings of the 40th international conference on software engineering, ICSE, pp 303–314. https://doi.org/10.1145/3180155.3180220

  • Varshosaz M, Al-Hajjaji M, Thüm T, Runge T, Mousavi MR, Schaefer I (2018) A classification of product sampling for software product lines. In: Proceedings of the 22nd international systems and software product line conference—volume 1, SPLC 2018, Gothenburg, Sweden, September 10–14, 2018, pp 1–13. https://doi.org/10.1145/3233027.3233035

  • Xiong Y, Hubaux A, She S, Czarnecki K (2012) Generating range fixes for software configuration. In: 34th International conference on software engineering

  • Yilmaz C, Cohen MB, Porter AA (2006) Covering arrays for efficient fault characterization in complex configuration spaces. IEEE Trans Softw Eng 32(1):20–34

    Article  Google Scholar 

  • Zhang M, Zhang Y, Zhang L, Liu C, Khurshid S (2018) Deeproad: Gan-based metamorphic testing and input validation framework for autonomous driving systems. In: Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, ASE 2018. ACM, New York, pp 132–142. https://doi.org/10.1145/3238147. http://doi.acm.org/10.1145/3238147

Download references

Acknowledgements

Gilles Perrouin is an FNRS Research Associate. This research was partly supported by EOS Verilearn project grant no. O05518F-RG03. This research was also funded by the ANR-17-CE25-0010-01 VaryVary project.

Author information

Authors and Affiliations

  1. NaDI, PReCISE, Faculty of Computer Science, University of Namur, Namur, Belgium

    Paul Temple & Gilles Perrouin

  2. University of Rennes, IRISA, Inria, CNRS, Rennes Cedex, France

    Mathieu Acher & Jean-Marc Jézéquel

  3. University of Cagliari, Cagliari, Italy

    Battista Biggio & Fabio Roli

Authors
  1. Paul Temple
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gilles Perrouin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mathieu Acher
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Battista Biggio
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jean-Marc Jézéquel
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fabio Roli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul Temple.

Additional information

Communicated by: Laurence Duchien, Thomas Thüm and Paul Grünbacher

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Configurable Systems

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Temple, P., Perrouin, G., Acher, M. et al. Empirical assessment of generating adversarial configurations for software product lines. Empir Software Eng 26, 6 (2021). https://doi.org/10.1007/s10664-020-09915-7

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-020-09915-7

Keywords

Search

Navigation