Skip to main content
SpringerLink
Log in

Improving Automatic Classifiers Through Interaction

  • Conference paper
  • First Online:
Artificial Intelligence and Soft Computing (ICAISC 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9693))

Included in the following conference series:

  • 1175 Accesses

Abstract

We consider a scenario where an automatic classifier has been built, but it sometimes decides to ask the correct label of an instance to an oracle, instead of accepting its own prediction. This interactive classifier only knows with certainty the labels provided by the oracle. Our proposal is to use this information to dynamically improve the behavior of the classifier, either increasing its accuracy when it is being used autonomously or reducing the number of queries to the oracle. We have tested our proposal by using twenty data sets and two adaptive classifiers from the Massive Online Analysis (MOA) open source framework for data stream mining.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The evaluation should not be based only on its predictive accuracy but should also take into account the cost of the human intervention.

  2. 2.

    In [1], it was shown that with values of \(\beta \) lesser than or equal to 0.5, the interactive classifiers systematically outperform their non-interactive counterparts.

  3. 3.

    In this paper we assume that the process is stationary and no concept drift occurs. The case of non-stationary problems, where a concept drift can modify the probability distribution, will be considered in future research.

  4. 4.

    By lack of space, we omit the proof of this result.

  5. 5.

    The new instances are supposed to come from the same data distribution.

  6. 6.

    Without taking into account those instances labeled by the oracle, \(n_i\).

References

  1. Acid, S., de Campos, L.M., Fernández, M.: Evaluation methods and strategies for the interactive use of classifiers. Int. J. Hum.-Comput. Stud. 70(5), 321–331 (2012)

    Article  Google Scholar 

  2. Agrawal, R., Imielinski, T., Swami, A.: Database mining: a performance perspective. IEEE Trans. Knowl. Data Eng. 5(6), 914–925 (1993)

    Article  Google Scholar 

  3. Auer, P., Cesa-Bianchi, N., Gentile, C.: Adaptive and self-confident on-line learning algorithms. J. Comput. Syst. Sci. 64, 48–75 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  4. Barlet, P.L., Wegkamp, M.H.: Classification with a reject option using a hinge loss. J. Mach. Learn. Res. 9, 1823–1840 (2008)

    MathSciNet  MATH  Google Scholar 

  5. Bifet, A., Holmes, G., Kirkby, R., Pfahringer, B.: MOA: massive online analysis. J. Mach. Learn. Res. 11, 1601–1604 (2010)

    Google Scholar 

  6. Fu, Y., Zhu, X., Li, B.: A survey on instance selection for active lerning. Knowl. Inf. Syst. 35, 249–283 (2013)

    Article  Google Scholar 

  7. Fumera, G., Roli, F., Giacinto, G.: Reject option with multiple thresholds. Pattern Recogn. 33(12), 2099–2101 (2000)

    Article  MATH  Google Scholar 

  8. Gama, J.: Knowledge Discovery from Data Streams. Chapman and Hall/CRC, Boca Raton (2010)

    Book  MATH  Google Scholar 

  9. Gama, J.: A survey on learning from data streams: current and future trends. Prog. Artif. Intell. 1, 45–55 (2012)

    Article  Google Scholar 

  10. Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: KDD-01, pp. 97–106. ACM Press (2001)

    Google Scholar 

  11. Lewis, D., Gale, W.: A sequential algorithm for training text classifiers. In: Proceedings of the ACM SIGIR Conference, pp. 3–12 (1994)

    Google Scholar 

  12. Settles, B.: Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison (2009)

    Google Scholar 

  13. Shalev-Shwartz, S.: Online learning and online convex optimization. Found. Trends Mach. Learn. 4(2), 107–194 (2012)

    Article  MATH  Google Scholar 

  14. van Rijsbergen, C.J.: Foundation of evaluation. J. Documentation 30(4), 365–373 (1974)

    Article  Google Scholar 

Download references

Acknowledgements

This work has been funded by the Spanish Ministry of Economy and Competitiveness under the project TIN2013-42741-P and the European Regional Development Fund (ERDF-FEDER).

Author information

Authors and Affiliations

  1. Department of Computer Science and Artificial Intelligence, Advanced Technical School for Information Technology and Telecomunication, CITIC-UGR, University of Granada, 18071, Granada, Spain

    Silvia Acid & Luis M. de Campos

Authors
  1. Silvia Acid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luis M. de Campos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Silvia Acid .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Czestochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Czestochowa, Poland

    Marcin Korytkowski

  3. Częstochowa University of Technology, Czestochowa, Poland

    Rafał Scherer

  4. AGH University of Science and Technology, Krakow, Poland

    Ryszard Tadeusiewicz

  5. University of California, Berkeley, California, USA

    Lotfi A. Zadeh

  6. University of Louisville, Louisville, Kentucky, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Acid, S., de Campos, L.M. (2016). Improving Automatic Classifiers Through Interaction. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2016. Lecture Notes in Computer Science(), vol 9693. Springer, Cham. https://doi.org/10.1007/978-3-319-39384-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-39384-1_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-39383-4

  • Online ISBN: 978-3-319-39384-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation