Skip to main content
SpringerLink
Log in

Computational bounds for elevator control policies by large scale linear programming

  • Original Article
  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

We computationally assess policies for the elevator control problem by a new column-generation approach for the linear programming method for discounted infinite-horizon Markov decision problems. By analyzing the optimality of given actions in given states, we were able to provably improve the well-known nearest-neighbor policy. Moreover, with the method we could identify an optimal parking policy. This approach can be used to detect and resolve weaknesses in particular policies for Markov decision problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. We chose the notation \(i_2\) instead of \(i_0\) to be consistent with the states considered in Tuchscherer (2010)

References

  • Adelman D, Mersereau A (2008) Relaxations of weakly coupled stochastic dynamic programs. Oper Res 56(3):712–727

    Article  MATH  MathSciNet  Google Scholar 

  • Ascheuer N, Krumke SO, Rambau J (2000) Online dial-a-ride problems: minimizing the completion time. In: Proceedings of the 17th international symposium on theoretical aspects of computer science, vol 1770. Springer, pp 639–650

  • Barto AG, Bradtke SJ, Singh SP (1995) Learning to act using real-time dynamic programming. Artif Intell 72:81–138

    Article  Google Scholar 

  • Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming, vol 1, 1st edn. Athena Scientific, Belmont

    MATH  Google Scholar 

  • Bertsekas DP (2001) Dynamic programming and optimal control, vol 1 and 2, 2nd edn. Athena Scientific, Belmont

    Google Scholar 

  • Crites RH, Barto AG (1998) Elevator group control using multiple reinforcement learning agents. Mach Learn 33(2–3):235–262

    Article  MATH  Google Scholar 

  • Dean TL, Kaelbling LP, Kirman J, Nicholson AE (1993) Planning with deadlines in stochastic domains. In: AAAI, pp 574–579

  • d’Epenoux F (1963) A probabilistic production and inventory problem. Manag Sci 10(1):98–108

    Article  Google Scholar 

  • Desai VV, Farias VF, Moallemi CC (2009) Approximate dynamic programming via a smoothed linear program. Graduate school of business, Columnbia University, working paper

  • Desaulniers G, Desrosiers J, Solomon MM (eds) (2005) Column generation. GERAD 25th anniversary series. Springer

  • de Farias DP, van Roy B (2003) The linear programming approach to approximate dynamic programming. Oper Res 51(6):850–865

    Article  MATH  MathSciNet  Google Scholar 

  • de Farias DP, van Roy B (2004) On constraint sampling in the linear programming approach to approximate dynamic programming. Math Oper Res 29(3):462–478

    Article  MATH  MathSciNet  Google Scholar 

  • de Farias DP, van Roy B (2006) A cost-shaping linear program for average-cost approximate dynamic programming with performance guarantees. Math Oper Res 31(3):597–620

    Article  MATH  MathSciNet  Google Scholar 

  • de Farias DP, Weber T (2008) Choosing the cost vector of the linear programming approach to approximate dynamic programming. In: Decision and control, pp 67–72

  • Feinberg EA, Shwartz A (eds) (2002) Handbook of Markov decision processes: methods and applications. Kluwer Academic Publishers, Dordrecht

    Google Scholar 

  • Friese P, Rambau J (2006) Online-optimization of a multi-elevator transport system with reoptimization algorithms based on set-partitioning models. Disc Appl Math 154(13):1908–1931. Also available as ZIB, Report 05–03

    Google Scholar 

  • Grötschel M, Hauptmeier D, Krumke SO, Rambau J (1999) Simulation studies for the online dial-a-ride problem. Report 99–09, ZIB

  • Hauptmeier D, Krumke SO, Rambau J (2000a) The online dial-a-ride problem under reasonable load. In: CIAC 2000. Lecture notes in computer science, vol 1767. Springer, pp 125–136

  • Hauptmeier D, Krumke SO, Rambau J (2000b) The online dial-a-ride problem under reasonable load. In: Proceedings of the 4th Italian conference on algorithms and complexity. Lecture notes in computer science, vol 1767. Springer, pp 137–149

  • Hiller B, Tuchscherer A (2008) Real-time destination-call elevator group control on embedded microcontrollers. In: Operations research proceedings 2007. Springer

  • Heinz S, Kaibel V, Peinhardt M, Rambau J, Tuchscherer A (2006) LP-based local approximation for Markov decision problems. Report 06–20, ZIB

  • Hiller B, Klug T, Tuchscherer A (2009) Improving the performance of elevator systems using exact reoptimization algorithms. In: Proceedings of MAPSP

  • Hiller B, Klug T, Tuchscherer A (2010) Improved destination call elevator control algorithms for up peak traffic. In: Operations research proceedings 2011. Springer. To appear

  • Kearns MJ, Mansour Y, Ng AJ (1999) A sparse sampling algorithm for near-optimal planning in large Markov decision processes. In: International joint conferences on artificial intelligence, pp 1324– 1331

  • Krumke SO, Rambau J (2012) Stability with uniform bounds for on-line dial-a-ride problems under reasonable load. In: Johansson R, Rantzer A (eds) Distributed decision making and control. Lecture notes in control and information science, vil 417, chap 17, vol 417. Springer, Berlin, pp 387–412

    Google Scholar 

  • Powell WB (2007) Approximate dynamic programming: solving the curses of dimensionality, 1st edn. Wiley, Hoboken

    Book  Google Scholar 

  • Puterman ML (2005) Markov decision processes: discrete stochastic dynamic programming, 2nd edn. Wiley, Hoboken

    Google Scholar 

  • Schröder J. (1990) Advanced dispatching: destination hall calls + instant car-to-call assignments: M10. Elevator World, pp 40–46

  • Schweitzer PJ, Seidmann A (1985) Generalized polynomial approximations in Markov decision processes. J Math Anal Appl 110:568–582

    Article  MATH  MathSciNet  Google Scholar 

  • Sutton RS, Barto AG (1998) Reinforcement learning: an introduction, 1st edn. MIT Press, Cambridge

    Google Scholar 

  • Tuchscherer A (2010) Local evaluation of policies for discounted Markov decision problems. Ph.D. thesis, Technische Universität, Berlin

  • Veatch MH, Walker N (2008) Approximate linear programming for network control: column generation and subproblems. Working paper, Gordon College

Download references

Acknowledgments

We thank the referee for valuable suggestions on the presentation of the paper.

Author information

Authors and Affiliations

  1. Zuse-Institute Berlin, Berlin, Germany

    Stefan Heinz

  2. Paul-Natorp-Gymnasium, Berlin, Germany

    Andreas Tuchscherer

  3. LS Wirtschaftsmathematik, Universität Bayreuth, Bayreuth, Germany

    Jörg Rambau

Authors
  1. Stefan Heinz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jörg Rambau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Tuchscherer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jörg Rambau.

Additional information

A.Tuchscherer—formerly affiliated with  Zuse-Institute Berlin.

Partially supported by the DFG Research Center Matheon “Mathematics for key technologies” in Berlin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Heinz, S., Rambau, J. & Tuchscherer, A. Computational bounds for elevator control policies by large scale linear programming. Math Meth Oper Res 79, 87–117 (2014). https://doi.org/10.1007/s00186-013-0454-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-013-0454-5

Keywords

Mathematics Subject Classification

Search

Navigation