Skip to main content
SpringerLink
Log in

Sparse flexible design: a machine learning approach

  • Published:
Flexible Services and Manufacturing Journal Aims and scope Submit manuscript

Abstract

For a general production network, state-of-the-art methods for constructing sparse flexible designs are heuristic in nature, typically computing a proxy for the quality of unseen networks and using that estimate in a greedy manner to modify a current design. This paper develops two machine learning-based approaches to constructing sparse flexible designs that leverage a neural network to accurately and quickly predict the performance of large numbers of candidate designs. We demonstrate that our heuristics are competitive with existing approaches and produce high-quality solutions for both balanced and unbalanced networks. Finally, we introduce a novel application of process flexibility in healthcare operations to demonstrate the effectiveness of our approach in a large numerical case study. We study the flexibility of linear accelerators that deliver radiation to treat various types of cancer. We demonstrate how clinical constraints can be easily absorbed into the machine learning subroutine and how our sparse flexible treatment networks meet or beat the performance of those designed by state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Akşin OZ, Karaesmen F (2007) Characterizing the performance of process flexibility structures. Oper Res Lett 35(4):477–484

    Article  MathSciNet  MATH  Google Scholar 

  • Applegate DL, Bixby RE, Chvátal V, Cook W, Espinoza DG, Goycoolea M, Helsgaun K (2009) Certification of an optimal tsp tour through 85,900 cities. Oper Res Lett 37(1):11–15

    Article  MathSciNet  MATH  Google Scholar 

  • Atun R, Jaffray DA, Barton MB, Bray F, Baumann M, Vikram B, Hanna TP, Knaul FM, Lievens Y, Lui TYM et al (2015) Expanding global access to radiotherapy. Lancet Oncol 16(10):1153–1186

    Article  Google Scholar 

  • Bassamboo A, Randhawa RS, Mieghem JAV (2012) A little flexibility is all you need: on the asymptotic value of flexible capacity in parallel queuing systems. Oper Res 60(6):1423–1435

    Article  MathSciNet  MATH  Google Scholar 

  • Bello I, Pham H, Le QV, Norouzi M, Bengio S (2016) Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:161109940

  • Bengio Y, Lodi A, Prouvost A (2018) Machine learning for combinatorial optimization: a methodological tour d’horizon. arXiv preprint arXiv:181106128

  • Bikker IA, Kortbeek N, van Os RM, Boucherie RJ (2015) Reducing access times for radiation treatment by aligning the doctor’s schemes. Oper Res Health Care 7:111–121

    Article  Google Scholar 

  • Cavalcante IM, Frazzon EM, Forcellini FA, Ivanov D (2019) A supervised machine learning approach to data-driven simulation of resilient supplier selection in digital manufacturing. Int J Inf Manage 49:86–97

    Article  Google Scholar 

  • Chan CW, Huang M, Sarhangian V (2021a) Dynamic server assignment in multiclass queues with shifts, with applications to nurse staffing in emergency departments. Oper Res

  • Chan CW, Sarhangian V, Talwai P, Gogia K (2021b) Utilizing partial flexibility to improve emergency department flow: theory and implementation. Working Paper http://www.columbia.edu/~cc3179/EDStaffingTrial_2020.pdf

  • Chan TCY, Fearing D (2019) Process flexibility in baseball: the value of positional flexibility. Manage Sci 65(4):1642–1666

    Article  Google Scholar 

  • Chen S, Song JSJ, Wei Y (2020) Data-driven scalable e-commerce transportation network design with unknown flow response. Available at SSRN 3590865

  • Chen X, Ma T, Zhang J, Zhou Y (2019) Optimal design of process flexibility for general production systems. Oper Res 67(2):516–531

    MathSciNet  MATH  Google Scholar 

  • Chou MC, Teo CP, Zheng H (2008) Process flexibility: design, evaluation, and applications. Flex Serv Manuf J 20(1–2):59–94

    Article  MATH  Google Scholar 

  • Chou MC, Chua GA, Teo CP (2010) On range and response: dimensions of process flexibility. Eur J Oper Res 207(2):711–724

    Article  MathSciNet  MATH  Google Scholar 

  • Chou MC, Chua GA, Teo CP, Zheng H (2010) Design for process flexibility: efficiency of the long chain and sparse structure. Oper Res 58(1):43–58

    Article  MathSciNet  MATH  Google Scholar 

  • Chou MC, Chua GA, Teo CP, Zheng H (2011) Process flexibility revisited: the graph expander and its applications. Oper Res 59(5):1090–1105

    Article  MathSciNet  MATH  Google Scholar 

  • Delaney G, Jacob S, Featherstone C, Barton M (2005) The role of radiotherapy in cancer treatment: Estimating optimal utilization from a review of evidence-based clinical guidelines. Cancer: Interdiscip Int J Am Cancer Soc 104(6):1129–1137

  • Deng T, Shen ZJM (2013) Process flexibility design in unbalanced networks. Manuf Serv Oper Manag 15(1):24–32

    Article  Google Scholar 

  • Désir A, Goyal V, Wei Y, Zhang J (2016) Sparse process flexibility designs: Is the long chain really optimal? Oper Res 64(2):416–431

    Article  MathSciNet  MATH  Google Scholar 

  • Dong J, Shi P, Zheng F, Jin X (2019) Off-service placement in inpatient ward network: Resource pooling versus service slowdown. Columbia Business School Research Paper (Forthcoming)

  • Feng W, Wang C, Shen ZJM (2017) Process flexibility design in heterogeneous and unbalanced networks: a stochastic programming approach. IISE Trans 49(8):781–799

    Article  Google Scholar 

  • Fischetti M, Fraccaro M (2017) Using OR + AI to predict the optimal production of offshore wind parks: A preliminary study. In: Sterle C (ed) Sforza A. Optimization and Decision Science, Methodologies and Applications, pp 203–211

    Google Scholar 

  • Graves SC, Tomlin BT (2003) Process flexibility in supply chains. Manage Sci 49(7):907–919

    Article  MATH  Google Scholar 

  • Gupta P, Gasse M, Khalil EB, Kumar MP, Lodi A, Bengio Y (2020) Hybrid models for learning to branch. arXiv preprint arXiv:200615212

  • Gurumurthi S, Benjaafar S (2004) Modeling and analysis of flexible queueing systems. Naval Res Logist (NRL) 51(5):755–782

    Article  MathSciNet  MATH  Google Scholar 

  • Hinton GE, Deng L, Yu D, Dahl GE, Ar Mohamed, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN et al (2012) Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Process Mag 29(6):82–97

    Article  Google Scholar 

  • Hopfield JJ, Tank DW (1985) Neural computation of decisions in optimization problems. Biol Cybern 52(3):141–152

    Article  MATH  Google Scholar 

  • Hopp WJ, Tekin E, Van Oyen MP (2004) Benefits of skill chaining in serial production lines with cross-trained workers. Manage Sci 50(1):83–98

    Article  Google Scholar 

  • Iravani SM, Van Oyen MP, Sims KT (2005) Structural flexibility: a new perspective on the design of manufacturing and service operations. Manage Sci 51(2):151–166

    Article  MATH  Google Scholar 

  • Jordan WC, Graves SC (1995) Principles on the benefits of manufacturing process flexibility. Manage Sci 41(4):577–594

    Article  MATH  Google Scholar 

  • Joustra PE, Van der Sluis E, Van Dijk NM (2010) To pool or not to pool in hospitals: a theoretical and practical comparison for a radiotherapy outpatient department. Ann Oper Res 178(1):77–89

    Article  MathSciNet  MATH  Google Scholar 

  • Joustra PE, Kolfin R, van Dijk NM, Koning CCE, Bakker PJM (2012) Reduce fluctuations in capacity to improve the accessibility of radiotherapy treatment cost-effectively. Flex Serv Manuf J 24(4):448–464

    Article  Google Scholar 

  • Kaempfer Y, Wolf L (2018) Learning the multiple traveling salesmen problem with permutation invariant pooling networks. arXiv preprint arXiv:180309621

  • Khalil E, Dai H, Zhang Y, Dilkina B, Song L (2017) Learning combinatorial optimization algorithms over graphs. In: Advances in neural information processing systems, pp 6348–6358

  • Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  • Larsen E, Lachapelle S, Bengio Y, Frejinger E, Lacoste-Julien S, Lodi A (2018) Predicting solution summaries to integer linear programs under imperfect information with machine learning. arXiv preprint arXiv:180711876

  • LeCun Y, Bengio Y, Hinton GE (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  • Legrain A, Fortin MA, Lahrichi N, Rousseau LM (2015) Online stochastic optimization of radiotherapy patient scheduling. Health Care Manag Sci 18(2):110–123

    Article  Google Scholar 

  • Li S, Geng N, Xie X (2015) Radiation queue: meeting patient waiting time targets. IEEE Robot Autom Mag 22(2):51–63

    Article  Google Scholar 

  • Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, Van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    Article  Google Scholar 

  • Mak HY, Shen ZJM (2009) Stochastic programming approach to process flexibility design. Flex Serv Manuf J 21(3–4):75–91

    Article  MATH  Google Scholar 

  • Price S, Golden B, Wasil E, Zhang HH (2013) Optimizing throughput of a multi-room proton therapy treatment center via simulation. In: Simulation conference (WSC), 2013 Winter, pp 2422–2431

  • Priore P, Gomez A, Pino R, Rosillo R (2014) Dynamic scheduling of manufacturing systems using machine learning: an updated review. Ai Edam 28(1):83–97

    Google Scholar 

  • Priore P, Ponte B, Puente J, Gómez A (2018) Learning-based scheduling of flexible manufacturing systems using ensemble methods. Comput Indu Eng 126:282–291

    Article  Google Scholar 

  • Saure A, Patrick J, Tyldesley S, Puterman ML (2012) Dynamic multi-appointment patient scheduling for radiation therapy. Eur J Oper Res 223(2):573–584

    Article  MathSciNet  MATH  Google Scholar 

  • Seward C (2017) Deep learning for warehouse operations. https://twimlai.com/twiml-talk-038-calvin-seward-deep-learning-warehouse-operations/

  • Sheikhzadeh M, Benjaafar S, Gupta D (1998) Machine sharing in manufacturing systems: total flexibility versus chaining. Int J Flex Manuf Syst 10(4):351–378

    Article  Google Scholar 

  • Simchi-Levi D (2010) Operations rules: Delivering customer value through flexible operations. MIT Press

  • Simchi-Levi D, Wei Y (2012) Understanding the performance of the long chain and sparse designs in process flexibility. Oper Res 60(5):1125–1141

    Article  MathSciNet  MATH  Google Scholar 

  • Simchi-Levi D, Wei Y (2015) Worst-case analysis of process flexibility designs. Oper Res 63(1):166–185

    Article  MathSciNet  MATH  Google Scholar 

  • Smith KA (1999) Neural networks for combinatorial optimization: a review of more than a decade of research. INFORMS J Comput 11(1):15–34

    Article  MathSciNet  MATH  Google Scholar 

  • Song H, Tucker AL, Graue R, Moravick S, Yang JJ (2020) Capacity pooling in hospitals: the hidden consequences of off-service placement. Manage Sci 66(9):3825–3842

    Article  Google Scholar 

  • Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112

  • Tsitsiklis JN, Xu K (2012) On the power of (even a little) resource pooling. Stochastic Syst 2(1):1–66

    Article  MathSciNet  MATH  Google Scholar 

  • Vieira B, Hans EW, van Vliet-Vroegindeweij C, van de Kamer J, van Harten W (2016) Operations research for resource planning and-use in radiotherapy: A literature review. BMC Med Inform Decis Mak 16(1):149

    Article  Google Scholar 

  • Vinyals O, Fortunato M, Jaitly N (2015) Pointer networks. In: Advances in neural information processing systems, pp 2692–2700

  • Wallace RB, Whitt W (2005) A staffing algorithm for call centers with skill-based routing. Manuf Serv Oper Manag 7(4):276–294

    Article  Google Scholar 

  • Wang X, Zhang J (2015) Process flexibility: a distribution-free bound on the performance of k-chain. Oper Res 63(3):555–571

    Article  MathSciNet  MATH  Google Scholar 

  • Werker G, Sauré A, French J, Shechter S (2009) The use of discrete-event simulation modelling to improve radiation therapy planning processes. Radiother Oncol 92(1):76–82

    Article  Google Scholar 

  • Yan Z, Gao SY, Teo CP (2018) On the design of sparse but efficient structures in operations. Manage Sci 64:3421–3445

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON, M5S 3G8, Canada

    Timothy C. Y. Chan & Benjamin G. Potter

  2. Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, ON, M5S 2M9, Canada

    Daniel Letourneau

Authors
  1. Timothy C. Y. Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Letourneau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benjamin G. Potter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin G. Potter.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A: Neural network implementation

1.1 Implementation

We initially considered a convolutional neural network (CNN) with multiple different kernel sizes. We found that when the kernel size equalled the input dimensions, the model had the best performance, thus rendering our CNN implementation equivalent to a fully connected NN. This result is not surprising given the lack of feature locality in our input data.

Our implemented neural network has two hidden layers. The first hidden layer has 1024 \(m \times n\) filters. Each filter is an \(m \times n\) matrix of weights that is multiplied element-wise with the incidence matrix. We found that initializing the filters to be orthogonal matrices with no bias terms improved our model accuracy. The output of each filter is a single value that is stored in a vector of length \(1024 \times 1\). The second hidden layer is a fully connected layer of 128 hidden units, where each unit is connected to every element in the convolutional layer output vector. Dozens of variations of this architecture were evaluated using the validation set before settling on this specification. We used ReLU activation functions.

There are a number of sophisticated open source software packages that support building, training, and deploying neural networks. In this paper, we used the Keras Python package as a wrapper around a Tensorflow back end running the Adam optimizer (Kingma and Ba 2014). All prediction models were trained using an NVIDIA GeForce GTX 1080 Ti GPU. We emphasize the importance of using graphical processing units (GPUs) for both the training of our models and their application within our heuristics. For example, we found that using a GPU resulted in a \(\sim \)25 times speedup during training for the Test Settings from Sect. 4. In addition, in the search stage of both the PS and PSRH heuristics, we are able to create large batches of networks that could be evaluated on a GPU, which greatly increased performance especially for the larger networks in the radiation therapy case study. For training, we used a root mean square error loss function.

Appendix B: PS and PSRH parameter analysis

The following figure illustrates the performance of the PS and PSRH heuristics with varying batch and search size parameters. Each panel is a different test setting and search size. The lines in each plot correspond to a different heuristic and batch size. While there may be performance improvements in going to larger batch and search sizes, generally we see that batch and search sizes of 2 perform quite well. Plus, having smaller batch and search sizes is less computationally expensive. Thus, we use batch and search sizes of 2 in our numerical experiments (Fig. 15).

Fig. 15
figure 15

Comparison of ML-based heuristic performance for different search and batch sizes

Full size image

Appendix C: Treatment network designs

Treatment network designs discussed in Sect. 5 are included here (Figs. 16, 17, 18, 19, 20, 21, 22).

Fig. 16
figure 16

Treatment network used to initialize heuristics in PM case study

Full size image
Fig. 17
figure 17

PSRH-designed 46-arc heterogeneous LINAC network

Full size image
Fig. 18
figure 18

PSRH-designed 56-arc heterogeneous LINAC network

Full size image
Fig. 19
figure 19

PSRH-designed 80-arc heterogeneous LINAC network

Full size image
Fig. 20
figure 20

PSRH-designed 46-arc homogenous LINAC network

Full size image
Fig. 21
figure 21

PSRH-designed 56-arc homogenous LINAC network

Full size image
Fig. 22
figure 22

PSRH-designed 80-arc homogenous LINAC network

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chan, T.C.Y., Letourneau, D. & Potter, B.G. Sparse flexible design: a machine learning approach. Flex Serv Manuf J 34, 1066–1116 (2022). https://doi.org/10.1007/s10696-021-09439-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10696-021-09439-2

Keywords

Search

Navigation