Skip to main content
SpringerLink
Log in

PEBBL: an object-oriented framework for scalable parallel branch and bound

  • Full Length Paper
  • Published:
Mathematical Programming Computation Aims and scope Submit manuscript

Abstract

Parallel Enumeration and Branch-and-Bound Library (PEBBL) is a C++ class library implementing the underlying operations needed to support a wide variety of branch-and-bound algorithms on MPI-based message-passing distributed-memory parallel computing environments. PEBBL can be customized to support application-specific operations, while managing the generic aspects of branch and bound, such as maintaining the active subproblem pool across multiple processors, load balancing, and termination detection. PEBBL is designed to provide highly scalable performance on large numbers of processor cores. We describe the basics of PEBBL’s architecture, with emphasis on the features most critical to is high scalability, including its flexible two-level load balancing architecture and its support for a synchronously parallel ramp-up phase. We also present an example application: the maximum monomial agreement problem arising from certain machine learning applications. For sufficiently difficult problem instances, we show essentially linear speedup on over 6000 processor cores, demonstrating a new state of the art in scalability for branch-and-bound implementations. We also show how processor cache effects can lead to reproducibly superlinear speedups.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Asuncion, A., Newman, D.J.: UCI machine learning repository (2007). http://www.ics.uci.edu/~mlearn/MLRepository.html

  2. Bader, D.A., Hart, W.E., Phillips, C.A.: Parallel algorithm design for branch and bound. In: Greenberg, H.J. (ed.) Tutorials on Emerging Methodologies and Applications in Operations Research, pp. 5-1–5-44. Kluwer Academic, Dordrecht (2004)

    Google Scholar 

  3. Benaïchouche, M., Cung, V.D., Dowaji, S., Cun, B.L., Mautor, T., Roucairol, C.: Building a parallel branch and bound library. In: Solving Combinatorial Optimization Problems in Parallel—Methods and Techniques, Lecture Notes in Computer Science, vol. 1054, pp. 201–231. Springer-Verlag, London (1996)

  4. Bendjoudi, A., Melab, N., Talbi, E.G.: Fault-tolerant mechanism for hierarchical branch and bound algorithm. In: Workshops Proceedings, 25th IEEE International Parallel and Distributed Processing Symposium, Workshop on Large-Scale Parallel Processing (LSPP), pp. 1806–1814. Anchorage, Alaska (20011)

  5. Blelloch, G.E.: Scans as primitive parallel operations. IEEE Trans. Comput. 38(11), 1526–1538 (1989)

    Article  Google Scholar 

  6. Boros, E., Hammer, P.L., Ibaraki, T., Kogan, A.: Logical analysis of numerical data. Math. Program. 79(1–3), 163–190 (1997)

    MATH  MathSciNet  Google Scholar 

  7. Brewer, T.: A highly scalable system utilizing up to 128 PA-RISC processors. In: COMPCON, Technologies for the Information Superhighway, Digest of Papers, pp. 133–140 (1995). http://dblp.uni-trier.de/db/conf/compcon/compcon95.html#Brewer95

  8. Cataldo, A.M., Whaley, R.C.: Scaling LAPACK panel operations using parallel cache assignment. In: ACM Principles and Practice of Parallel Programming. ACM, New York (2010)

  9. Clausen, J.: Branch and bound algorithms—principles and examples. In: Migdalas, A., Pardalos, P., Storøy, S. (eds.) Parallel Computing in Optimization, Applied Optimization, vol. 7, pp. 239–267. Kluwer Academic, Dordrecht (1997)

    Chapter  Google Scholar 

  10. Clausen, J., Perregaard, M.: On the best search strategy in parallel branch-and-bound: Best-first search versus lazy depth-first search. Ann. Oper. Res. 90, 1–17 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  11. Danna, E., Fenelon, M., Gu, Z., Wunderling, R.: Generating multiple solutions for mixed integer programming problems. In: Integer Programming and Combinatorial Optimization, Lecture Notes in Computer Science, vol. 4513, pp. 280–294. Springer, Berlin (2007)

  12. Demiriz, A., Bennett, K.P., Shawe-Taylor, J.: Linear programming boosting via column generation. Mach. Learn. 46, 225–254 (2002)

    Article  MATH  Google Scholar 

  13. Dobkin, D.P., Gunopulos, D., Maass, W.: Computing the maximum bichromatic discrepancy, with applications to computer graphics and machine learning. J. Comp. Syst. Sci. 52(3), 453–470 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  14. Eckstein, J.: Control strategies for parallel mixed integer branch and bound. In: Supercomputing ’94: Proceedings of the 1994 ACM/IEEE conference on Supercomputing, pp. 41–48. ACM, New York (1994)

  15. Eckstein, J.: Parallel branch-and-bound algorithms for general mixed integer programming on the CM-5. SIAM J. Optim. 4(4), 794–814 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  16. Eckstein, J.: Distributed versus centralized storage and control for parallel branch and bound: mixed integer programming on the CM-5. Comput. Optim. Appl. 7(2), 199–220 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  17. Eckstein, J.: How much communication does parallel branch and bound need? INFORMS J. Comput. 9(1), 15–29 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  18. Eckstein, J., Goldberg, N.: An improved branch-and-bound method for maximum monomial agreement. INFORMS J. Comput. 24(2), 328–341 (2012)

    Article  MathSciNet  Google Scholar 

  19. Eckstein, J., Hart, W.E., Phillips, C.: Massively parallel mixed-integer programming: algorithms and applications. In: Heroux, M.A., Raghavan, P., Simon, H. (eds.) Parallel Processing for Scientific Computing, chapter 17, pp. 323–340. Based on the 11th SIAM Conference on Parallel Processing for Scientific Computing. SIAM (2006)

  20. Eckstein, J., Hart, W.E., Phillips, C.A.: Resource management in a parallel mixed integer programming package. In: Proceedings of the Intel Supercomputer Users Group (1997). Online proceedings

  21. Eckstein, J., Phillips, C.A., Hart, W.E.: PICO: An object-oriented framework for parallel branch and bound. In: Inherently parallel algorithms in feasibility and optimization and their applications (Haifa, 2000), Stud. Comput. Math., vol. 8, pp. 219–265. North-Holland, Amsterdam (2001)

  22. Eckstein, J., Phillips, C.A., Hart, W.E.: PEBBL: An object-oriented framework for scalable parallel branch and bound. RUTCOR Research Report RRR 9-2013, Rutgers University (2013)

  23. Eckstein, J., Phillips, C.A., Hart, W.E.: PEBBL 1.4.1 user’s guide. RUTCOR Research Report RRR 2-2014, Rutgers University (2014)

  24. Elf, M., Gutwenger, C., Jünger, M., Rinaldi, G.: Branch-and-cut algorithms for combinatorial optimization and their implementation in ABACUS. In: Computational Combinatorial Optimization, Optimal or Provably Near-Optimal Solutions, pp. 157–222. Springer-Verlag, London (2001)

  25. Goldberg, N., Shan, C.: Boosting optimal logical patterns. In: Proceedings of the Seventh SIAM International Conference on Data Mining, pp. 228–236. SIAM, Minneapolis (2007)

  26. Graham, R.L., Woodall, T.S., Squyres, J.M.: Open MPI: A flexible high performance MPI. In: Proceedings, 6th Annual International Conference on Parallel Processing and Applied Mathematics. Poznan, Poland (2005)

  27. Gropp, W., Lusk, E., Doss, N., Skjellum, A.: High-performance, portable implementation of the MPI Message Passing Interface Standard. Parallel Comput. 22(6), 789–828 (1996)

    Article  MATH  Google Scholar 

  28. Hillis, W.D.: The Connection Machine. MIT Press, Cambridge (1985)

    Google Scholar 

  29. Ichnowski, J., Alterovitz, R.: Parallel sampling-based motion planning with superlinear speedup. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1206–1212 (2012)

  30. Jünger, M., Thienel, S.: Introduction to ABACUS–a branch-and-cut system. Oper. Res. Lett. 22(2–3), 83–95 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  31. Jünger, M., Thienel, S.: The ABACUS system for branch-and-cut-and-price algorithms in integer programming and combinatorial optimization. Software Pract. Expert. 30, 1325–1352 (2000)

    Article  MATH  Google Scholar 

  32. Karypis, G., Kumar, V.: Unstructured tree search on SIMD parallel computers: a summary of results. In: Supercomputing ’92: Proceedings of the 1992 ACM/IEEE conference on Supercomputing, pp. 453–462. IEEE Computer Society Press, Los Alamitos (1992)

  33. Kearns, M.J., Schapire, R.E., Sellie, L.M.: Toward efficient agnostic learning. Mach. Learn. 17(2–3), 115–141 (1994)

    MATH  Google Scholar 

  34. Kim, M., Lee, H., Lee;, J.: A proportional-share scheduler for multimedia applications. In: IEEE International Conference on Multimedia computing and systems ’97, pp. 484–491. IEEE Computer Society Press, Los Alamitos (1997)

  35. Koch, T., Ralphs, T., Shinano, Y.: Could we use a millon cores to solve an integer program? Math. Methods Oper. Res. 76, 67–93 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  36. Ladányi, L., Ralphs, T.K., Trotter Jr., L.E.: Branch, cut, and price: Sequential and parallel. In: Computational combinatorial optimization (Schloß Dagstuhl, 2000), Lecture Notes in Computer Science, vol. 2241, pp. 223–260. Springer, Berlin (2001)

  37. Mahanti, A., Daniel, C.J.: A SIMD approach to parallel heuristic search. Artif. Intel. 60, 243–282 (1993)

    Article  Google Scholar 

  38. Mattern, F.: Algorithms for distributed termination detection. Distrib. Comput. 2, 161–175 (1987)

    Article  Google Scholar 

  39. Menouer, T., LeCun, B., Vander-Swalmen, P.: Partitioning methods to parallelize constraint programming solver using the parallel framework BobPP. In: Proceedings of the First International Conference on Computer Science, Applied Mathematics and Applications (ICCSAMA), pp. 117–127. Springer International Publishing, Switzerland (2013)

  40. Nemhauser, G.L., Savelsbergh, M.W.P., Sigismondi, G.C.: MINTO, a mixed INTeger optimizer. Oper. Res. Lett. 15(1), 47–58 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  41. Ralphs, T.K., Ladányi, L., Saltzman, M.J.: Parallel branch, cut, and price for large-scale discrete optimization. Math. Program. Ser. B 98(1–3), 253–280 (2003)

    Article  MATH  Google Scholar 

  42. Ralphs, T.K., Ládanyi, L., Saltzman, M.J.: A library hierarchy for implementing scalable parallel search algorithms. J. Supercomput. 28(2), 215–234 (2004)

    Article  MATH  Google Scholar 

  43. Rayward-Smith, V.J., Rush, S.A., McKeown, G.P.: Efficiency considerations in the implementation of parallel branch-and-bound. Ann. Oper. Res. 43(1–4), 123–145 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  44. Schulz, M., Galarowicz, J., Maghrak, D., Hachfeld, W., Montoya, D., Cranford, S.: Open\(|\)Speedshop: an open source infrastructure for parallel performance analysis. Sci. Program. 16(2–3), 105–121 (2008)

    Google Scholar 

  45. Shinano, Y.: ParaSCIP and fiberSCIP libraries to parallelize a customized SCIP solver (2014). http://scip.zib.de/workshop/parascip_libraries.pdf. SCIP Workshop 2014

  46. Shinano, Y., Achterberg, T., Berthold, T., Heinz, S., Koch, T.: paraSCIP: a parallel extension of SCIP. In: Competence in High Performance Computing 2010, pp. 135–148. Springer-Verlag, Belin Heidelberg (2012)

  47. Shinano, Y., Harada, K., Hirabayashi, R.: Control schemes in a generalized utility for parallel branch-and-bound algorithms. In: IPPS ’97: Proceedings of the 11th International Symposium on Parallel Processing, p. 621. IEEE Computer Society, Washington, DC (1997)

  48. Shinano, Y., Higaki, M., Hirabayashi, R.: A generalized utility for parallel branch and bound algorithms. In: SPDP ’95: Proceedings of the 7th IEEE Symposium on Parallel and Distributeed Processing, p. 392. IEEE Computer Society, Washington, DC (1995)

  49. Snir, M., Otto, S.W., Walker, D.W., Dongarra, J., Huss-Lederman, S.: MPI: The Complete Reference. MIT Press, Cambridge (1995)

    Google Scholar 

  50. Tschöke, S., Polzer, T.: Portable parallel branch-and-bound library: PPBB-Lib user manual version 2.0. Tech. rep., Department of Computer Science, University of Paderborn (1996)

  51. Waldspurger, C.A.: Lottery and stride scheduling: flexibile proportional-share resource management. Ph.D. thesis, MIT Department of Electrical Engineering and Computer Science, Cambridge (1996)

  52. Wu, M.S., Aluru, S., Kendall, R.A.: Mixed mode matrix multiplication. In: Proceedings of the IEEE International Conference on Cluster Computing, pp. 195–203 (2002)

  53. Wu, X. (ed.): Performance Evaluation, Prediction and Visualization of Parallel Systems, vol. 4 of The Kluwer International Series on Asian Studies in Computer and Information Science. Kluwer Academic, Springer (1999)

  54. Xu, Y., Ralphs, T.K., Ladányi, L., Saltzman, M.J.: Computational experience with a software framework for parallel integer programming. INFORMS J. Comput. 21, 383–397 (2009). http://coral.ie.lehigh.edu/~ted/files/papers/CHiPPS-Rev.pdf

Download references

Acknowledgments

We thank Noam Goldberg for contributing the code to solve the MMA problem in serial using PEBBL. We also thank John Siirola for his stewardship of PEBBL within the ACRO framework. Jonathan Eckstein’s work on PEBBL was supported in part by NSF grant CCR-9902092. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

Author information

Authors and Affiliations

  1. Department of Management Science and Information Systems and RUTCOR, Rutgers University, 100 Rockafeller Road, Piscataway, NJ, 08854, USA

    Jonathan Eckstein

  2. Center for Computing Research, Sandia National Laboratories, Mail Stop 1327, P.O. Box 5800, Albuquerque, NM, 87185-1327, USA

    William E. Hart

  3. Center for Computing Research, Sandia National Laboratories, Mail Stop 1326, P.O. Box 5800, Albuquerque, NM, 87185-1326, USA

    Cynthia A. Phillips

Authors
  1. Jonathan Eckstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William E. Hart
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cynthia A. Phillips
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jonathan Eckstein.

Additional information

This work was performed in part at Sandia National Laboratories. Sandia is a multiprogram laboratory operated by Sandia corporation, a Lockheed Martin Company, for the United States Department of Energy under Contract DE-AC04-94-AL85000. This work was also supported in part by National Science Foundation Grant CCR-9902092.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Eckstein, J., Hart, W.E. & Phillips, C.A. PEBBL: an object-oriented framework for scalable parallel branch and bound. Math. Prog. Comp. 7, 429–469 (2015). https://doi.org/10.1007/s12532-015-0087-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12532-015-0087-1

Keywords

Mathematics Subject Classification

Search

Navigation