Skip to main content
View expanded cover

Joint European Conference on Machine Learning and Knowledge Discovery in Databases

ECML PKDD 2018: Machine Learning and Knowledge Discovery in Databases pp 447–463Cite as

Frame-Based Optimal Design

  • 1529 Accesses

Part of the Lecture Notes in Computer Science book series (LNAI,volume 11052)

Abstract

Optimal experimental design (OED) addresses the problem of selecting an optimal subset of the training data for learning tasks. In this paper, we propose to efficiently compute OED by leveraging the geometry of data: We restrict computations to the set of instances lying on the border of the convex hull of all data points. This set is called the frame. We (i) provide the theoretical basis for our approach and (ii) show how to compute the frame in kernel-induced feature spaces. The latter allows us to sample optimal designs for non-linear hypothesis functions without knowing the explicit feature mapping. We present empirical results showing that the performance of frame-based OED is often on par or better than traditional OED approaches, but its solution can be computed up to twenty times faster.

Keywords

  • Active learning
  • Fast approximation
  • Frame
  • Optimal experimental design
  • Regression

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-10928-8_27
  • Chapter length: 17 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   99.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-10928-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   129.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.

References

  1. Ageev, A.A., Sviridenko, M.I.: Pipage rounding: a new method of constructing algorithms with proven performance guarantee. J. Comb. Optim. 8(3), 307–328 (2004)

    MathSciNet  CrossRef  Google Scholar 

  2. Allen-Zhu, Z., Li, Y., Singh, A., Wang, Y.: Near-optimal design of experiments via regret minimization. In: International Conference on Machine Learning, pp. 126–135 (2017)

    Google Scholar 

  3. Avron, H., Boutsidis, C.: Faster subset selection for matrices and applications. SIAM J. Matrix Anal. Appl. 34(4), 1464–1499 (2013)

    MathSciNet  CrossRef  Google Scholar 

  4. Barber, C.B., Dobkin, D.P., Huhdanpaa, H.: The quickhull algorithm for convex hulls. ACM Trans. Math. Softw. (TOMS) 22(4), 469–483 (1996)

    MathSciNet  CrossRef  Google Scholar 

  5. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, New York (2004)

    CrossRef  Google Scholar 

  6. Bro, R., De Jong, S.: A fast non-negativity-constrained least squares algorithm. J. Chemom. 11(5), 393–401 (1997)

    CrossRef  Google Scholar 

  7. Brooks, T.F., Pope, D.S., Marcolini, M.A.: Airfoil self-noise and prediction (1989)

    Google Scholar 

  8. Chaudhuri, K., Kakade, S.M., Netrapalli, P., Sanghavi, S.: Convergence rates of active learning for maximum likelihood estimation. In: Advances in Neural Information Processing Systems, pp. 1090–1098 (2015)

    Google Scholar 

  9. Dereziński, M., Warmuth, M.K.: Subsampling for ridge regression via regularized volume sampling. arXiv preprint arXiv:1710.05110 (2017)

  10. Dolia, A.N., De Bie, T., Harris, C.J., Shawe-Taylor, J., Titterington, D.M.: The minimum volume covering ellipsoid estimation in kernel-defined feature spaces. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 630–637. Springer, Heidelberg (2006). https://doi.org/10.1007/11871842_61

    CrossRef  Google Scholar 

  11. Dulá, J.H., Helgason, R.V.: A new procedure for identifying the frame of the convex hull of a finite collection of points in multidimensional space. Eur. J. Oper. Res. 92(2), 352–367 (1996)

    CrossRef  Google Scholar 

  12. Dulá, J.H., López, F.J.: Competing output-sensitive frame algorithms. Comput. Geom. 45(4), 186–197 (2012)

    MathSciNet  CrossRef  Google Scholar 

  13. Fedorov, V.V.: Theory of Optimal Experiments. Elsevier, New York (1972)

    Google Scholar 

  14. Kulesza, A., et al.: Determinantal point processes for machine learning. Found. Trends® Mach. Learn. 5(2–3), 123–286 (2012)

    CrossRef  Google Scholar 

  15. Lawson, C.L., Hanson, R.J.: Solving least squares problems, vol. 15. SIAM (1995)

    Google Scholar 

  16. Li, C., Jegelka, S., Sra, S.: Polynomial time algorithms for dual volume sampling. In: Advances in Neural Information Processing Systems, pp. 5045–5054 (2017)

    Google Scholar 

  17. Lopez, F.J.: Generating random points (or vectors) controlling the percentage of them that are extreme in their convex (or positive) hull. J. Math. Model. Algorithms 4(2), 219–234 (2005)

    MathSciNet  CrossRef  Google Scholar 

  18. Mair, S., Boubekki, A., Brefeld, U.: Frame-based data factorizations. In: International Conference on Machine Learning, pp. 2305–2313 (2017)

    Google Scholar 

  19. Mariet, Z.E., Sra, S.: Elementary symmetric polynomials for optimal experimental design. In: Advances in Neural Information Processing Systems, pp. 2136–2145 (2017)

    Google Scholar 

  20. Nocedal, J., Wright, S.J.: Sequential quadratic programming. In: Nocedal, J., Wright, S.J. (eds.) Numerical Optimization. Springer Series in Operations Research and Financial Engineering. Springer, New York (2006). https://doi.org/10.1007/978-0-387-40065-5_18

    CrossRef  MATH  Google Scholar 

  21. Ottmann, T., Schuierer, S., Soundaralakshmi, S.: Enumerating extreme points in higher dimensions. Nord. J. Comput. 8(2), 179–192 (2001)

    MathSciNet  MATH  Google Scholar 

  22. Pace, R.K., Barry, R.: Sparse spatial autoregressions. Stat. Probab. Lett. 33(3), 291–297 (1997)

    CrossRef  Google Scholar 

  23. Pukelsheim, F.: Optimal Design of Experiments. SIAM (2006)

    Google Scholar 

  24. Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2002)

    Google Scholar 

  25. Smola, A.J., Schölkopf, B., Müller, K.R.: The connection between regularization operators and support vector kernels. Neural Netw. 11(4), 637–649 (1998)

    CrossRef  Google Scholar 

  26. Sugiyama, M., Nakajima, S.: Pool-based active learning in approximate linear regression. Mach. Learn. 75(3), 249–274 (2009)

    CrossRef  Google Scholar 

  27. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  28. Wang, Y., Yu, A.W., Singh, A.: On computationally tractable selection of experiments in measurement-constrained regression models. J. Mach. Learn. Res. 18(143), 1–41 (2017)

    MathSciNet  MATH  Google Scholar 

  29. Yeh, I.C.: Modeling of strength of high-performance concrete using artificial neural networks. Cem. Concr. Res. 28(12), 1797–1808 (1998)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Leuphana University, Lüneburg, Germany

    Sebastian Mair, Yannick Rudolph, Vanessa Closius & Ulf Brefeld

Authors
  1. Sebastian Mair
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yannick Rudolph
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vanessa Closius
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ulf Brefeld
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sebastian Mair .

Editor information

Editors and Affiliations

  1. IBM Research - Ireland, Dublin, Ireland

    Michele Berlingerio

  2. Institute for Scientific Interchange, Turin, Italy

    Francesco Bonchi

  3. University of Nottingham, Nottingham, UK

    Thomas Gärtner

  4. University College Dublin, Dublin, Ireland

    Neil Hurley

  5. University College Dublin, Dublin, Ireland

    Georgiana Ifrim

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Mair, S., Rudolph, Y., Closius, V., Brefeld, U. (2019). Frame-Based Optimal Design. In: Berlingerio, M., Bonchi, F., Gärtner, T., Hurley, N., Ifrim, G. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2018. Lecture Notes in Computer Science(), vol 11052. Springer, Cham. https://doi.org/10.1007/978-3-030-10928-8_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-10928-8_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-10927-1

  • Online ISBN: 978-3-030-10928-8

  • eBook Packages: Computer ScienceComputer Science (R0)