Skip to main content
SpringerLink
Log in

Learning from Positive and Negative Examples: Dichotomies and Parameterized Algorithms

  • Conference paper
  • First Online:
Combinatorial Algorithms (IWOCA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13270))

Included in the following conference series:

  • 501 Accesses

Abstract

We take a closer look on the complexity landscape of one of the most fundamental and well-studied problems in computational learning theory: the problem of learning a finite automaton A consistent with a set \(P\) of positive examples and with a set \(N\) of negative examples. By consistency, we mean that A accepts all strings in \(P\) and rejects all strings in \(N\). It is well known that this problem is NP-hard when parameterized only by the number of states of the automaton. Therefore, our analysis takes a more refined parameterization: we consider the number k of states in A, the size \(|\varSigma |\) of the alphabet, the maximum size l of a string in \(P\cup N\), and the number \(c=|P\cup N|\) of strings in both sets. First, we prove several Pvs. NP-hard dichotomy results for these parameters when the learned automaton is drawn from different classes of finite automata. One of our dichotomy results closes a gap for the general DFA consistency problem, as here, for fixed alphabet size, the NP-hardness proofs in the literature have some issues. Interestingly, our NP-hardness results hold even for severely restricted classes of automata, such as partially-ordered automata and permutation automata. On the other hand, we provide parameterized algorithms for several combinations of parameters and show that most of them are optimal under the exponential time hypothesis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Also in [2] Mealy automata instead of DFAs are considered.

  2. 2.

    Given a Boolean formula in conjunctive normal form where all clauses have exactly three literals (this form is called 3CNF) and all literals are positive. Is there a variable assignment such that exactly one literal per clause evaluates to true?.

  3. 3.

    In general \(V_0, V_1, V_2\) is not a partition of V since some of the sets might be empty if G is 1- or 2-colorable.

References

  1. Abdulla, P.A., Collomb-Annichini, A., Bouajjani, A., Jonsson, B.: Using forward reachability analysis for verification of lossy channel systems. Formal Methods Syst. Des. 25(1), 39–65 (2004)

    Article  Google Scholar 

  2. Angluin, D.: On the complexity of minimum inference of regular sets. Inf. Control 39(3), 337–350 (1978)

    Article  MathSciNet  Google Scholar 

  3. Angluin, D.: Inference of reversible languages. J. ACM 29(3), 741–765 (1982)

    Article  MathSciNet  Google Scholar 

  4. Bouhmala, N.: A multilevel learning automata for MAX-SAT. Int. J. Mach. Learn. Cybern. 6(6), 911–921 (2015)

    Article  Google Scholar 

  5. Brzozowski, J.A., Fich, F.E.: Languages of R-trivial monoids. J. Comput. Syst. Sci. 20(1), 32–49 (1980)

    Article  MathSciNet  Google Scholar 

  6. Clifford, A.H., Preston, G.B.: The Algebraic Theory of Semigroups, Volume II, vol. 2. American Mathematical Soc., Providence (1967)

    Book  Google Scholar 

  7. Coste, F., Kerbellec, G.: Learning automata on protein sequences. In: JOBIM, pp. 199–210 (2006)

    Google Scholar 

  8. Cygan, M., et al.: Parameterized Algorithms. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21275-3_15

    Book  MATH  Google Scholar 

  9. Downey, R.G., Fellows, M.R.: Fundamentals of Parameterized Complexity, vol. 4. Springer, London (2013). https://doi.org/10.1007/978-1-4471-5559-1

    Book  MATH  Google Scholar 

  10. Fernau, H.: Personal communication (2021)

    Google Scholar 

  11. Fernau, H.: Algorithms for learning regular expressions from positive data. Inf. Comput. 207(4), 521–541 (2009)

    Article  MathSciNet  Google Scholar 

  12. Fernau, H., Heggernes, P., Villanger, Y.: A multi-parameter analysis of hard problems on deterministic finite automata. J. Comput. Syst. Sci. 81(4), 747–765 (2015)

    Article  MathSciNet  Google Scholar 

  13. Fernau, H., Krebs, A.: Problems on finite automata and the exponential time hypothesis. Algorithms 10(1), 24 (2017)

    Article  MathSciNet  Google Scholar 

  14. Gold, E.M.: Language identification in the limit. Inf. Control 10(5), 447–474 (1967)

    Article  MathSciNet  Google Scholar 

  15. Gold, E.M.: Complexity of automaton identification from given data. Inf. Control 37(3), 302–320 (1978)

    Article  MathSciNet  Google Scholar 

  16. Groce, A., Peled, D., Yannakakis, M.: Adaptive model checking. In: Katoen, J.-P., Stevens, P. (eds.) TACAS 2002. LNCS, vol. 2280, pp. 357–370. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46002-0_25

    Chapter  Google Scholar 

  17. Gruber, H., Holzer, M.: Finite automata, digraph connectivity, and regular expression size. In: Aceto, L., Damgård, I., Goldberg, L.A., Halldórsson, M.M., Ingólfsdóttir, A., Walukiewicz, I. (eds.) ICALP 2008. LNCS, vol. 5126, pp. 39–50. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-70583-3_4

    Chapter  MATH  Google Scholar 

  18. Guo, H., Wang, S., Fan, J., Li, S.: Learning automata based incremental learning method for deep neural networks. IEEE Access 7, 41164–41171 (2019)

    Article  Google Scholar 

  19. Hasanzadeh-Mofrad, M., Rezvanian, A.: Learning automata clustering. J. Comput. Sci. 24, 379–388 (2018)

    Article  MathSciNet  Google Scholar 

  20. De la Higuera, C.: Grammatical Inference: Learning Automata and Grammars. Cambridge University Press, Cambridge (2010)

    Book  Google Scholar 

  21. Impagliazzo, R., Paturi, R.: On the complexity of k-SAT. J. Comput. Syst. Sci. 62(2), 367–375 (2001)

    Article  MathSciNet  Google Scholar 

  22. Impagliazzo, R., Paturi, R., Zane, F.: Which problems have strongly exponential complexity? J. Comput. Syst. Sci. 63(4), 512–530 (2001)

    Article  MathSciNet  Google Scholar 

  23. Jirásková, G., Masopust, T.: On the state and computational complexity of the reverse of acyclic minimal DFAs. In: Moreira, N., Reis, R. (eds.) CIAA 2012. LNCS, vol. 7381, pp. 229–239. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31606-7_20

    Chapter  MATH  Google Scholar 

  24. Klíma, O., Polák, L.: Alternative automata characterization of piecewise testable languages. In: Béal, M.-P., Carton, O. (eds.) DLT 2013. LNCS, vol. 7907, pp. 289–300. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38771-5_26

    Chapter  Google Scholar 

  25. Lokshtanov, D., Marx, D., Saurabh, S., et al.: Lower bounds based on the exponential time hypothesis. Bull. EATCS 3(105), 41–71 (2013)

    MathSciNet  MATH  Google Scholar 

  26. Mao, H., Chen, Y., Jaeger, M., Nielsen, T.D., Larsen, K.G., Nielsen, B.: Learning deterministic probabilistic automata from a model checking perspective. Mach. Learn. 105(2), 255–299 (2016). https://doi.org/10.1007/s10994-016-5565-9

    Article  MathSciNet  MATH  Google Scholar 

  27. Martens, W., Neven, F., Schwentick, T.: Complexity of decision problems for XML schemas and chain regular expressions. SIAM J. Comput. 39(4), 1486–1530 (2009)

    Article  MathSciNet  Google Scholar 

  28. Mayr, F., Yovine, S.: Regular inference on artificial neural networks. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-MAKE 2018. LNCS, vol. 11015, pp. 350–369. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99740-7_25

    Chapter  Google Scholar 

  29. Meybodi, M.R., Beigy, H.: New learning automata based algorithms for adaptation of backpropagation algorithm parameters. Int. J. Neural Syst. 12(01), 45–67 (2002)

    Article  Google Scholar 

  30. Mount, D.W., Mount, D.W.: Bioinformatics: Sequence and Genome Analysis, vol. 1. Cold Spring Harbor Laboratory Press, Cold Spring Harbor, NY (2001)

    Google Scholar 

  31. Najim, K., Pibouleau, L., Le Lann, M.: Optimization technique based on learning automata. J. Optim. Theory Appl. 64(2), 331–347 (1990)

    Article  MathSciNet  Google Scholar 

  32. Najim, K., Poznyak, A.S.: Learning Automata: Theory and Applications. Elsevier, Amsterdam (2014)

    MATH  Google Scholar 

  33. Nowé, A., Verbeeck, K., Peeters, M.: Learning automata as a basis for multi agent reinforcement learning. In: Tuyls, K., Hoen, P.J., Verbeeck, K., Sen, S. (eds.) LAMAS 2005. LNCS (LNAI), vol. 3898, pp. 71–85. Springer, Heidelberg (2006). https://doi.org/10.1007/11691839_3

    Chapter  Google Scholar 

  34. Parekh, R., Honavar, V.: Learning DFA from simple examples. Mach. Learn. 44(1), 9–35 (2001)

    Article  Google Scholar 

  35. Pin, J.-E.: On reversible automata. In: Simon, I. (ed.) LATIN 1992. LNCS, vol. 583, pp. 401–416. Springer, Heidelberg (1992). https://doi.org/10.1007/BFb0023844

    Chapter  Google Scholar 

  36. Pitt, L.: Inductive inference, DFAs, and computational complexity. In: Jantke, K.P. (ed.) AII 1989. LNCS, vol. 397, pp. 18–44. Springer, Heidelberg (1989). https://doi.org/10.1007/3-540-51734-0_50

    Chapter  Google Scholar 

  37. Pitt, L., Warmuth, M.K.: The minimum consistent DFA problem cannot be approximated within any polynomial. J. ACM 40(1), 95–142 (1993)

    Article  MathSciNet  Google Scholar 

  38. Ramadge, P.J., Wonham, W.M.: Supervisory control of a class of discrete event processes. SIAM J. Control. Optim. 25(1), 206–230 (1987)

    Article  MathSciNet  Google Scholar 

  39. Rezvanian, A., Saghiri, A.M., Vahidipour, S.M., Esnaashari, M., Meybodi, M.R.: Recent Advances in Learning Automata. SCI, vol. 754. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-72428-7

    Book  MATH  Google Scholar 

  40. Ryzhikov, A.: Synchronization problems in automata without non-trivial cycles. Theoret. Comput. Sci. 787, 77–88 (2019)

    Article  MathSciNet  Google Scholar 

  41. Schaefer, T.J.: The complexity of satisfiability problems. In: Lipton, R.J., Burkhard, W.A., Savitch, W.J., Friedman, E.P., Aho, A.V. (eds.) Proceedings of the 10th Annual ACM Symposium on Theory of Computing, pp. 216–226. ACM (1978)

    Google Scholar 

  42. Schützenberger, M.P.: On finite monoids having only trivial subgroups. Inf. Control 8(2), 190–194 (1965)

    Article  MathSciNet  Google Scholar 

  43. Simon, I.: Piecewise testable events. In: Brakhage, H. (ed.) GI-Fachtagung 1975. LNCS, vol. 33, pp. 214–222. Springer, Heidelberg (1975). https://doi.org/10.1007/3-540-07407-4_23

    Chapter  Google Scholar 

  44. Thierrin, G.: Permutation automata. Mathem. Syst. Theory 2(1), 83–90 (1968)

    Article  MathSciNet  Google Scholar 

  45. Yazidi, A., Bouhmala, N., Goodwin, M.: A team of pursuit learning automata for solving deterministic optimization problems. Appl. Intell. 50(9), 2916–2931 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

Petra Wolf was supported by DFG project FE 560/9-1, and Mateus de Oliveira Oliveira by the RCN projects 288761 and 326537.

Author information

Authors and Affiliations

  1. Eberhard Karls Universität Tübingen, Tübingen, Germany

    Jonas Lingg

  2. University of Bergen, Bergen, Norway

    Mateus de Oliveira Oliveira

  3. Universität Trier, Trier, Germany

    Petra Wolf

Authors
  1. Jonas Lingg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mateus de Oliveira Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Petra Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Petra Wolf .

Editor information

Editors and Affiliations

  1. Université Paris-Dauphine, Paris, France

    Cristina Bazgan

  2. Universität Trier, Trier, Germany

    Henning Fernau

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lingg, J., de Oliveira Oliveira, M., Wolf, P. (2022). Learning from Positive and Negative Examples: Dichotomies and Parameterized Algorithms. In: Bazgan, C., Fernau, H. (eds) Combinatorial Algorithms. IWOCA 2022. Lecture Notes in Computer Science, vol 13270. Springer, Cham. https://doi.org/10.1007/978-3-031-06678-8_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06678-8_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06677-1

  • Online ISBN: 978-3-031-06678-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation