Integrated scheduling of drayage and long-haul operations in synchromodal transport

Abstract

We study the problem of the integrated scheduling of drayage operations and long-haul transport in synchromodality. Although different in time span and characteristics of execution, these two processes have an impact on each other and their interaction has a direct influence on the overall performance of the transport network over time. We propose a simulation based integration of a Mixed-Integer Linear Programming model for the drayage operations and a Markov Decision Process model for the long-haul transport. We analyze the interfaces between these models, outline the challenges of integrating them, and design a heuristic approach to the simulation based integration. In a series of numerical experiments, we evaluate the cost savings compared to a non-integrated approach, using various transport network configurations. We show that our approach achieves average cost savings between 4 and 24% on networks with a majority of pre-haulage freights. Furthermore, we discuss limitations of our model and experiments, and provide guidelines for further research for the integrated scheduling of drayage and long-haul operations in synchromodal transport.

1 Introduction

In recent years, the freight transport industry has faced the challenge of reducing the environmental impact of their operations while staying profitable. To tackle this challenge, Logistic Service Providers (LSPs) have been re-defining performance measures and exploring new approaches to control their operations. One of the new perspectives on performance measurement and control of freight transport operations over a multi-modal network is synchromodality. In synchromodality, the path of a freight from its origin to its destination is not necessarily established up front, but decided using the latest information about the status of the transport network and freight demand. Furthermore, freight can be assigned to any mode and any transfer terminal available in the transport network. This increased flexibility provides LSPs with more opportunities for consolidation and options for efficient transport, throughout the network and throughout time. However, to take advantage of such opportunities and options, transport decisions must encompass the entire network, and performance measurements must span more than a single decision moment. In this paper, we study a network-wide and multi-period decision problem in synchromodality: the integrated scheduling of drayage operations and long-haul transport.

Drayage operations and long-haul transport have usually been scheduled independently due to their difference in time span of execution. On the one hand, drayage operations, also known as pre-/end-haulage and first/last-mile, are carried out by trucks in small geographical areas where each truck can move more than one freight per day. Drayage scheduling involves, simultaneously, the timing of pick-ups and deliveries, the routing of the vehicles, and the selection of the departure terminal for pre-haulage freights, i.e., the intermodal terminal where pre-haulage freights are brought to for the start of their long-haul transport. On the other hand, long-haul transport is carried out by high-capacity modes such as trains and barges, over long geographical areas with traveling times typically lasting more than a day. Long-haul scheduling involves the selection of intermodal services to use for each freight, where an intermodal service is defined by a transport mode and scheduled departure and arrival time. Although the scheduling decisions of each part are mutually independent, schedules of the drayage operations can influence the long-haul transport, and the long-haul schedules can influence the opportunities for the drayage operations, as we exemplify next.

Consider an LSP controlling a synchromodal transport network (one organization able to choose between various carriers) that has to decide which of two intermodal terminals to use as a departure terminal. If the focus is on drayage performance, the terminal closest to the origin of the freight is most likely to be chosen, even if the furthest terminal has better consolidation opportunities for long-haul transport. If the focus is on the long-haul, the terminal with the best consolidation opportunities is most likely to be chosen, even at the expense of unnecessary drayage operations costs. Traditionally, literature on multi-modal long-haul transport has not considered drayage operations as a whole (e.g,. movement of empty containers, pre- and end-haulage freight, etc.), even though the integration of independent scheduling activities using up-to-date network information has been identified as fundamental to improve the performance of multi-modal transport (Crainic et al. 2009).

For an LSP facing the circumstances mentioned before, integrating the scheduling of drayage operations and long-haul transport is difficult for two reasons. First, freights that arrive each day, and their characteristics, are uncertain. Second, the individual drayage and long-haul scheduling decisions have an impact on each other’s feasible decision space. A network-wide focus may signify the deterioration of one of the individual performances, e.g., bringing a freight to a further terminal may be beneficial for the long-haul and the network as a whole but detrimental for the drayage. Balancing such a trade-off requires a careful analysis of the problem characteristics. Considering these challenges, we establish a twofold objective for this paper: (1) to design an approach that integrates the scheduling decisions of drayage operations and long-haul transport and (2) to provide insights into the use of our approach under various freight demand patterns and cost settings.

Following our objective, the contribution of this paper is twofold. First, we design a simulation-based approach to overcome the optimization challenges of integrating the scheduling of drayage operations and long-haul transport. Our approach iteratively solves a Mixed-Integer Linear Programming (MILP) model of the drayage operations and a Markov Decision Process (MDP) model of the long-haul operations. The MILP considers long-haul performance based on the solution of the MDP while the MDP uses freight arrival rates based on the solution of the MILP. To use our approach on large instances, we use a Math-Heuristic (MH) for solving the MILP and an Approximate Dynamic Programming (ADP) algorithm for solving the MDP. Second, we study the use of our approach under various instances of freight demand and drayage/long-haul costs, and evaluate the relation between the decisions and costs of drayage operations and long-haul transport. In addition, we evaluate the gains of integrated scheduling compared to a benchmark heuristic. Based on the evaluations, we gather managerial insights for integrated scheduling of drayage operations and long-haul transport and provide guidelines for further research.

The remainder of this paper is structured as follows. In Sect. 2, we examine literature related to drayage scheduling, long-haul scheduling, and their integration. In Sect. 3, we describe and formulate the problem, along with the MILP and MDP models for the drayage and long-haul scheduling, respectively. In Sect. 4, we present our simulation-based approach and describe the MH and ADP heuristics. In Sect. 5, we present the results of our numerical experiments, and discuss their research and managerial implications. Finally, we present our conclusions in Sect. 6.

2 Literature review

In this section, we briefly review the literature on scheduling drayage operations as well as scheduling long-haul transport in a multi-modal network. The daily scheduling of drayage operations, for the available freights of that day, can be seen as a special case of the Vehicle Routing Problem (VRP): the Full Truckload Pickup-and-Delivery Problem with Time-Windows (FTPDPTW) with flexible tasks (Erera and Smilowitz 2008), where the flexible tasks are the freights whose long-haul terminal is not pre-defined. The scheduling of long-haul transport in a setting with multiple modes, a multi-period horizon, dynamic assignment of freight to services, and stochastic freight arrivals, can be seen as a special case of Dynamic Service Network Design (DSND) problems (SteadieSeifi et al. 2014). In DSND, the assignment of services to freight through time and through different links in a transport network is studied. Since we are interested in the integration of decisions and performance as a perspective to schedule synchromodal transport, we focus on the modeling assumptions of the studies about scheduling multi-modal transport instead of their solution methodologies. For an in-depth review of models and solution methodologies for scheduling drayage operations and long-haul transport, we refer the reader to Erera and Smilowitz (2008) and SteadieSeifi et al. (2014), respectively, noting that the literature about drayage operations is scarce. We start by reviewing the studies on scheduling drayage operations and their relation to synchromodality and long-haul transport. Subsequently, we review studies on scheduling long-haul transport and their relation to synchromodality and drayage operations. We finalize by stating our scientific contribution to integrated scheduling in synchromodal transport.

The impact of drayage operations on the total costs of multi-modal transport is large (Konings 2009), but, research on their scheduling has been limited (Caris et al. 2013). Particularly, studies on the scheduling of drayage operations considering multiple characteristics of synchromodality, such as the flexible choice of drayage destination (i.e., departure terminal for the long-haul) and the use of network information (e.g., including long-haul performance), is limited. Examples that consider and focus on the flexibility in the origin of an empty container to use for a new request can be found in Caris and Janssens (2009), Francis et al. (2007) and Smilowitz (2006). Examples that consider multiple terminals and focus on a homogenous fleet of trucks can be found in Nossack and Pesch (2013) and Braekers et al. (2013), and that consider a heterogeneous fleet without considering time-windows of the requests in Imai et al. (2007). Moreover, most studies about scheduling drayage operations that include some form of network information usually focus on special cases of information that impact decisions rather than network-wide performance. For example, Taylor et al. (2002) include the flexible selection of an initial ramp (i.e., terminal) for freight in truck-rail transport with the objective of minimizing trucking costs. Wang and Regan (2002) and Escudero et al. (2013) consider delays at intermodal terminals as triggers for re-scheduling. Markovi et al. (2014) considers stochastic delays at intermodal terminals that impact the drayage costs. Studies that explicitly include performance on both drayage and long-haul do so for special applications and objectives. For example, Verma and Verter (2010) and Verma et al. (2012) develop a drayage model considering total costs, that includes long-haul costs, for a truck-rail network of hazardous materials and with an additional objective and set of constraints regarding the risk of transporting the hazardous materials. Pérez Rivera and Mes (2017c) consider a terminal assignment costs, which they point out can be used to capture long-haul performance, but does not depend on the combination of freights brought to the terminals. To summarize, scheduling of drayage operations have mostly been studied considering only a few forms of flexibility and without considering long-haul performance that varies through time as new information is revealed.

Compared to drayage operations, scheduling long-haul transport in multi-modal transport has been widely studied. Studies that are closely related to synchromodality are those that include dynamic decisions, flexible assignment to modes, and network-wide performance considering stochastic demand over time. DSND studies have built a strong body of knowledge for modeling the evolution of network-wide decisions and performance through time (Wieberneit 2008). However, the complexity of the models have made it difficult to include stochastic components (SteadieSeifi et al. 2014), even though the need to incorporate uncertainty in demand has been recognized (Lium et al. 2009). Studies that include stochastic demand and anticipate on its realization, such as Lium et al. (2009) and Crainic et al. (2014), usually consider scenario-based models to develop a schedule that is robust to incoming freights over the horizon. Nevertheless, this way of scheduling does not adapt dynamically to a new status of the network. Models that dynamically adapt to new information from the network, such as the two-stage stochastic programming models of Lo et al. (2013) and Bai et al. (2014), and the Markov Decision Process models of Dall’Orto et al. (2006) and Pérez Rivera and Mes (2017b), focus on modeling the complex relations of mode and transfer choices over time, but do not explicitly model the richness of drayage operations and the interaction between the two decision processes (drayage and long-haul). In fact, most DSND studies, independent of whether they include stochastic components, do not explicitly model characteristics of the drayage operations. Some of the studies, such as Crainic et al. (2015), point out that costs from the origin of freights to the initial terminals can be calibrated to mimic drayage operations. Yet, these costs might depend on the status of the network, as we exemplified in Sect. 1. To summarize, scheduling long-haul transport in synchromodality can build upon multi-modal studies with stochastic demand and extend these studies by including the richness of drayage operations.

To conclude, most literature about scheduling drayage operations and scheduling long-haul transport have largely ignored the impact of both decision processes on each other’s operations. Furthermore, some of the key characteristics of synchromodal transport networks have not been included in the modeling assumptions. Considering this, the contribution of our paper to the scientific literature is twofold. First, we design a scheduling approach that focuses on the interaction between drayage and long-haul decisions, considering multiple characteristics of synchromodality such as stochastic arrivals and flexible terminal assignment. Second, we provide insight into how the characteristics of a synchromodal network influence the integration of drayage and long-haul scheduling decisions.

3 Problem description and formulation

We introduce the problem by subsequently describing the characteristics of freight (Sect. 3.1), timing of decisions (Sect. 3.2), drayage operations (Sect. 3.3), and long-haul transport (Sect. 3.4). The latter two parts are based on Pérez Rivera and Mes (2017c) and Pérez Rivera and Mes (2017b) respectively. However, to ease the presentation of the integrated approach, we apply some simplifications to earlier work as we describe in Sect. 3.5.

3.1 Freight characteristics

We study the problem of scheduling drayage operations and long-haul transport in a synchromodal network, over a multi-period horizon ${\mathscr {T}}$, with the objective of minimizing the total expected costs over the network and over the horizon. For simplicity, we refer to a period $t\in {\mathscr {T}}$ as a day in the remainder of this paper, noticing nevertheless, that time can be discretized into any arbitrary interval as long as the modeling considerations mentioned below are applied.

Each day t, new freights ${\mathscr {F}}_t$ arrive into the network. We distinguish between pre-haulage jobs and end-haulage jobs. A pre-haulage job represents freight that starts its long-haul journey and needs to be transported to an initial intermodal terminal by means of a drayage job. An end-haulage job represents freight that finished its long-haul transport and only requires a drayage job to bring it to the customer. Hence, only pre-haulage freights use the long-haul modes and have a destination in the long-haul part of the network. End-haulage freights only use the trucks of the drayage part, and hence have a drayage destination but no long-haul destination. These two considerations make the network asymmetric in paths followed by pre- and end-haulage freights, as exemplified in Fig. 1. In this figure, drayage operations only take place in the left side of the network; in the right side of the network (i.e., around the destinations of the pre-haulage freights), drayage operations are not explicitly modeled, but considered implicitly through the costs of the services running between intermodal terminals and the long-haul destinations.

Each freight $f^\text {N}\in {\mathscr {F}}_t$ has a drayage request type $r^\text {D}\in {\mathscr {R}}$, and depending on the request type, it has drayage destination $d^\text {D}\in {\mathscr {D}}^\text {D}$ (in case of an end-haulage job), or drayage origin $o^\text {D}\in {\mathscr {O}}$ and long-haul destination $d^\text {L}\in {\mathscr {D}}^\text {L}$ (in case of a pre-haulage job). The set of drayage request types ${\mathscr {R}}$ represents all possible drayage jobs, which not only differ in whether they are a pre- or end-haulage job, or whether they use an empty or full container, but also in required driver clearance, truck chassis, etc. The sets of drayage destinations ${\mathscr {D}}^\text {D}$, origins ${\mathscr {O}}$, and long-haul destinations ${\mathscr {D}}^\text {L}$ denote locations in the network, as seen in Fig. 1.

Fig. 1

Simplified example of network locations, pre-haulage jobs, and end-haulage jobs. This is a simple network in the sense that choice of departure terminal determines the arrival terminal, which is not necessarily the case in general networks

In addition to location information, each freight has a drayage and long-haul time-window. The drayage time-window $\left[ e^\text {D},l^\text {D}\right] \in {\mathscr {T}}^\text {D}$ defines the earliest and latest arrival time of a truck to the drayage origin, where the set of drayage times ${\mathscr {T}}^\text {D}\subset {\mathbb {R}}$ represents the time during a single day when trucks are working. The long-haul time-window length $l^\text {L}\in {\mathscr {T}}^\text {L}$ represents the number of days, relative to the current day t, in which a freight must be at its long-haul destination. In other words, a freight with long-haul time-window length $l^\text {L}$ at day t is due on $(t+l^\text {L})$. Using the notation just introduced, we can describe a freight $f^\text {N}\in {\mathscr {F}}_t$ with the tuple $f^\text {N}=\left( r^\text {D},d^\text {D},o^\text {D},e^\text {D},l^\text {D},d^\text {L},l^\text {L}\right)$.

Although freights are not known before they arrive, there is probabilistic knowledge about their arrival intensity and about their characteristics. Between two consecutive days, $f\in {\mathbb {N}}$ freights arrive with probability $p_f^\text {F}$. Each freight has drayage request type $r^\text {D}\in {\mathscr {R}}$ with probability $p_r^\text {R}$ and drayage origin $o^\text {D}\in {\mathscr {O}}$ with probability $p_o^\text {O}$. Depending on the request type $r^\text {D}\in {\mathscr {R}}$, a freight has drayage destination $d^\text {D}\in {\mathscr {D}}^\text {D}$ with probability $p_{r,d^\text {D}}^\text {DD}$, long-haul destination $d^\text {L}\in {\mathscr {D}}^\text {L}$ with probability $p_{r,d^\text {L}}^\text {DL}$, earliest drayage truck arrival $e^\text {D}\in {\mathscr {T}}^\text {D}$ with probability $p_{r,e^\text {D}}^\text {ETD}$, latest drayage truck arrival $l^\text {D}\in {\mathscr {T}}^\text {D}$ with probability $p_{r,l^\text {D}}^\text {LTD}$, and long-haul time-window length $l^\text {L}\in {\mathscr {T}}$ with probability $p_{r,l^\text {L}}^\text {LTL}$. To be concise, we enclose all the probabilities mentioned before, which describe all freights that arrive to the network, in the set ${\mathscr {P}}^\text {D}$.

3.2 Timing of decisions

Each day $t\in {\mathscr {T}}$, the planner must first schedule the drayage operations and then schedule the long-haul transport. Drayage scheduling includes two decisions: (1) the initial terminal where each pre-haulage freight will begin its long-haul, and (2) the route of trucks that execute the drayage operations. Long-haul scheduling includes the assignment decision of each freight in a terminal to an available service at that terminal at the current day, or the postponement of freight to future days (i.e., no service). This assignment decision results in consolidation of certain freight on certain services, and implies part of the route a freight will take, not necessarily the whole route. Although drayage and long-haul scheduling might be done separately, their decisions have an influence on the decision space of each other, and consequently, on the attainable performance over the network and over time. For instance, the assigned initial terminal of pre-haulage freights influences the route of trucks in the drayage operations and the consolidation possibilities in the long-haul modes.

We illustrate the timing of the freight arrivals, transport processes and scheduling decisions using Fig. 2. Consider one terminal with a long-haul mode departing daily at noon and freights that arrive throughout the day. The long-haul schedule $\left( x_{t}^\text {L}\right)$ for day t is created at noon, just before the mode departs. The drayage schedule $\left( x_{t}^\text {D}\right)$ is created at midnight (i.e., the beginning of day t), considering freights that arrived before midnight ${\mathscr {F}}_t$ as well as routing costs and terminal costs $\left( C_{t}^\text {L}\right)$. Drayage operations are completely carried out between midnight and noon. Freights $\left( f_{t}^\text {L}\right)$ to be considered for the schedule of the long-haul at day t$\left( x_{t}^\text {L}\right)$ depend then on freights that were brought by the drayage operations of that day $\left( x_{t}^\text {D}\right)$ in addition to the freights that were not scheduled for the long-haul the day before $\left( x_{t-1}^\text {L}\right)$. This example shows our modeling assumption of alternating one long-haul schedule creation and one drayage schedule creation. We note, however, that this assumption can easily be generalized to a different timing of schedule creation for the two parts of transport, as long as (1) each long-haul scheduling decision depends on all drayage scheduling decisions performed (i.e., long-haul freight brought to intermodal terminals) in between the previous and the current long-haul scheduling decisions, and (2) each time a drayage schedule is created, the impact on the next long-haul schedule is considered, even if the immediately following scheduling is another one for the drayage. The notation given inside parenthesis in the figure is described later on but already included to illustrate their relation. Note that the superscript ‘D’ denotes the parameters related to drayage scheduling and ‘L’ the parameters related to long-haul scheduling.

Fig. 2

Timing of freight arrivals, transport processes, and scheduling decisions

3.3 Drayage operations

To perform all drayage operations, there is a fleet ${\mathscr {K}}$ of heterogeneous trucks. The subset of trucks that can carry out drayage request type $r^\text {D}\in {\mathscr {R}}$ is denoted by ${\mathscr {K}}_r$. The time to execute a drayage request type $r^\text {D}\in {\mathscr {R}}$, e.g., load/unload a container, by truck $k\in {\mathscr {K}}$ is denoted by $T_{r,k}^\text {SD}$. Truck $k\in {\mathscr {K}}$ must begin its route in terminal $B_k$ and finalize it in terminal $F_k\in {\mathscr {H}}$, where ${\mathscr {H}}$ represents all terminals available in the transport network. Each terminal $h\in {\mathscr {H}}$ has a service time $T_{h}^\text {SH}$, which denotes the time to serve a drayage truck. To represent all locations in the drayage, i.e., origins, drayage destinations, and terminals, we define the set ${\mathscr {N}}^\text {D}={\mathscr {O}}\cup {\mathscr {D}}^\text {D}\cup {\mathscr {H}}$ and index it with i and j. We denote the traveling time between locations i and j with $T_{i,j}^\text {TD}$. The routing costs of truck $k\in {\mathscr {K}}$ consist of a fixed cost $C_k^\text {FD}$ for its use and a variable cost $C_{k,i,j}^\text {VD}$ for traveling between locations i and j.

We represent the drayage decisions using the binary variable $x_{k,i,j,t}^\text {D}$, which gets a value of 1 if truck $k\in {\mathscr {K}}$ moves from location $i\in {\mathscr {N}}^\text {D}$ to location $j\in {\mathscr {N}}^\text {D}$ at time t and 0 otherwise. We represent all decisions $x_{k,i,j,t}^\text {D}, \forall k\in {\mathscr {K}}, \, i\in {\mathscr {N}}^\text {D}, \, j\in {\mathscr {N}}^\text {D}$ with a vector $x_t^\text {D}$. The drayage costs at time t, as a function of $x_t^\text {D}$, are defined in (1).

$$\begin{aligned} z_t^\text {D}\left( x_t^\text {D}\right) = \sum _{k\in {\mathscr {K}}}{\left( C^\text {FD}_k\cdot \sum _{j\in {\mathscr {N}}^\text {D}}{x_{k,B_k,j,t}^\text {D}}\right) } + \sum _{k\in {\mathscr {K}}}\sum _{i\in {\mathscr {N}}^\text {D}}{\sum _{j\in {\mathscr {N}}^\text {D}}{C^\text {{VD}}_{k,i,j}\cdot x_{k,i,j,t}^\text {D}}} \end{aligned}$$

(1)

For a complete formulation of the drayage scheduling problem, we refer to the Mixed-Integer Linear Programming (MILP) model from Pérez Rivera and Mes (2017c).

3.4 Long-haul transport

To perform the long-haul transport, there are various high-capacity modes traveling between locations in the long-haul, i.e., long-haul terminals and long-haul destinations. We denote all locations in the long-haul with ${\mathscr {N}}^\text {L}={\mathscr {H}}\cup {\mathscr {D}}^\text {L}$ and index it with i and j. Note that an intermodal terminal is by definition an intermediate terminal and cannot be a destination. We define a service as a specific long-haul transport mode going from location i to j, both in ${\mathscr {N}}^\text {L}$, with a capacity of $Q_{i,j}$ freights and a duration of $T_{i,j}^\text {TL}$, which includes traveling and handling times at the locations. Although there are no capacity limits in departure terminals, the capacity of the long-haul services may prevent freights from using this service and instead use an alternative one or use trucks. Similar to the trucks, we consider the long-haul costs of the service between i and j to consist of a fixed component $C_{i,j}^\text {FL}$ if at least one freight uses this service, and a variable cost $C_{i,j,d}^\text {VL}$ per freight with destination $d\in {\mathscr {D}}^\text {L}$ consolidated in this service. Furthermore, we consider additional long-haul cost $C_{i,j,{\mathscr {G}}}^\text {GL}$ per group of long-haul destinations ${\mathscr {G}}\subseteq {\mathscr {D}}^\text {L}$ visited after, or during, the service between i and j. This cost component can be used to capture, for example, last-mile routing costs after the service between two intermodal terminals, or stopping costs at intermediate destinations visited by the service. Note that in practice, part of the capacity of a service between i and j might be reserved up front, resulting in sunk costs for this service. In principle, this can be handled in our model by duplicating a service, resulting in reserved and unreserved capacity with different prices.

We represent the long-haul transport decision with the integer variable $x_{i,j,d,l,t}^\text {L}$, which counts the number of freights that will be assigned to the service from i to j (both in ${\mathscr {N}}^\text {L}$) at time t, which have long-haul destination $d\in {\mathscr {D}}^\text {L}$ and long-haul time-window length $l\in {\mathscr {T}}$. We represent all decisions $x_{i,j,d,l,t}^\text {L}, \forall i\in {\mathscr {N}}^\text {L}, \, j\in {\mathscr {N}}^\text {L}, \, d\in {\mathscr {D}}^\text {L}, \, l\in {\mathscr {T}}$ with a vector $x_t^\text {L}$. In addition to the $x_{i,j,d,l,t}^\text {L}$ variable, we introduce the auxiliary binary variable $y_{i,j,d,t}^\text {L}$, which gets a value of 1 if there is a freight with destination $d\in {\mathscr {D}}^\text {L}$ consolidated at time t in the service from i to j, both in ${\mathscr {N}}^\text {L}$, and 0 otherwise, as shown in (2).

$$\begin{aligned} y_{i,j,d,t}^\text {L} = {\left\{ \begin{array}{ll} 1, &{} \text {if}\ \displaystyle \sum _{l\in {\mathscr {T}}}{\left( x_{i,j,d,l,t}^\text {L}\right) }>0 \\ 0, &{} \text {otherwise} \end{array}\right. }, \; \forall i\in {\mathscr {N}}^\text {L}, \, j\in {\mathscr {N}}^\text {L}, \, d\in {\mathscr {D}}^\text {L} \end{aligned}$$

(2)

The long-haul transport costs at time t, as a function of $x_t^\text {L}$, are defined in (3).

$$\begin{aligned} \begin{aligned} z_t^\text {L}\left( x_{t}^\text {L}\right) =&\sum _{i\in {\mathscr {N}}^\text {L}}{\sum _{j\in {\mathscr {N}}^\text {L}}{\left( C^\text {{FL}}_{i,j}\cdot \sum _{d\in {\mathscr {D}}^\text {L}}{y_{i,j,d,t}^\text {L}} + \sum _{d\in {\mathscr {D}}^\text {L}}{\left( C_{i,j,d}^\text {VL}\cdot \sum _{l\in {\mathscr {T}}}{x_{i,j,d,l,t}^\text {L}}\right) }\right) }}\\&+\sum _{i\in {\mathscr {N}}^\text {L}}{\sum _{j\in {\mathscr {N}}^\text {L}}{\sum _{{\mathscr {G}}\subseteq {\mathscr {D}}^\text {L}}{\left( C_{i,j,{\mathscr {G}}}^\text {GL}\cdot {\prod _{d\in {\mathscr {G}}}{y_{i,j,d,t}^\text {L}}}\cdot {\prod _{d'\in {\mathscr {G}}\setminus {\mathscr {D}}^\text {L}}{\left( 1-y_{i,j,d',t}^\text {L}\right) }}\right) }}} \end{aligned} \end{aligned}$$

(3)

In (3), the first line captures the economies of scale of long-haul services (through the fixed and variable cost component) and the second line captures the combination of destinations in each service (through the group cost component). This cost definition is a more generic cost function than the one from Pérez Rivera and Mes (2017b), which does not include the group cost component and which includes a reward per freight making it a profit maximization rather than cost minimization problem. However, the decision variable for freight consolidation is equivalent.

We model the scheduling of long-haul transport using an MDP model. The decision is represented by $x_t^\text {L}$. The state is given by freights that are, and will be, present at the terminals. To represent them, we use the integer variable $f_{j,d,l,t}^\text {L}$, which counts the number of freights at terminal $j\in {\mathscr {H}}$ at time t that have long-haul destination $d\in {\mathscr {D}}^\text {L}$, and long-haul time-window length $l\in {\mathscr {T}}$. The state $s_t^\text {L}$ is defined as the vector of these variables, i.e., $s_t^\text {L} = f_t^\text {L} = \left[ f_{j,d,l,t}^\text {L}\right] , \forall j\in {\mathscr {H}}, \, d\in {\mathscr {D}}^\text {L}, \, l\in {\mathscr {T}}$. The state $s_t^\text {L}$ is the result of freights from the previous day $t-1$ that were not consolidated (i.e., difference between the previous state $s_{t-1}^\text {L}$ and previous long-haul decision $x_{t-1}^\text {L}$) and freights that arrived to the terminal in between the previous day $t-1$ and day t. These latter freights are the freights that were brought to the terminals by the drayage operations, which depend on the random freights that arrived to the system (as illustrated in Fig. 2) and are known as the exogenous information of the MDP model. The exogenous information $w_t^\text {L}$ captures the stochasticity of the system, which in turn, converts it into a policy that optimizes the expected performance as will be explained later on. Similar to the freights in the state, we define freights in the exogenous information using the integer variable ${\hat{f}}_{j,d,l,t}^\text {L}$, which counts the number of freights that arrived at terminal $j\in {\mathscr {H}}$ at time t, and that have long-haul destination $d\in {\mathscr {D}}^\text {L}$ and long-haul time-window length $l\in {\mathscr {T}}$. The exogenous information $w_t^\text {L}$ is defined as the vector of these variables, i.e., $w_t^\text {L} = \left[ {\hat{f}}_{j,d,l,t}^\text {L}\right] , \forall j\in {\mathscr {H}}, \, d\in {\mathscr {D}}^\text {L}, \, l\in {\mathscr {T}}$.

For the constraints and equations defining the decision space ${\mathscr {X}}_t^\text {L}$, state space ${\mathscr {S}}$, and transition function $S^\text {M}$, we refer to the MDP model of Pérez Rivera and Mes (2017b). In case of a simplified network as shown in Fig. 1, i.e., separated long-haul lanes, the formulation from Pérez Rivera and Mes (2017a) can be used, replicating the model for each long-haul lane. Although our solution approach is formulated for the generic network setting, we use a simplified network with separated long-haul lanes for our numerical experiments.

The optimal expected cost $V_t\left( s_t^\text {L}\right)$ for each state $s_t^\text {L}\in {\mathscr {S}}$ is given by the Bellman’s recursion (4). This optimal value captures the direct cost of the long-haul $z_t^\text {L}\left( x_t^\text {L}\right)$ plus the expected future costs ${\mathbb {E}}\left[ V_{t+1}\left( s_{t+1}^\text {L}\right) \right]$. In (4b), we use a transition function $S^M$ to define state $s_{t+1}^\text {L}$ in terms of two known vectors at time t: the state $s_t^\text {L}$ and the decision $x_t^\text {L}$; and one unknown vector at time t: the exogenous information $w_{t+1}^\text {L}$, which is partly random and partly due to the drayage decision. Using $p^{\varOmega _{t+1}}_\omega$ as the probability of exogenous information $\omega$, where $\omega$ is a realization of the exogenous information $w_{t+1}^\text {L}$ from the set of all exogenous realizations ${\varOmega _{t+1}}$ following from the drayage-dependent arrival distributions, we can further expand the Bellman’s recursion as shown in (4c).

$$\begin{aligned} V_t\left( s_t^\text {L}\right)&=\min _{x_t^\text {L}\in {\mathscr {X}}_t^\text {L}}{\left( z_t^\text {L}\left( x_t^\text {L}\right) +{\mathbb {E}}\left[ V_{t+1}\left( s_{t+1}^\text {L}\right) \right] \right) }, \quad \forall s_t^\text {L}\in {\mathscr {S}} \end{aligned}$$

(4a)

$$\begin{aligned}&=\min _{x_t^\text {L}\in {\mathscr {X}}_t^\text {L}}{\left( z_t^\text {L}\left( x_t^\text {L}\right) +{\mathbb {E}}\left[ V_{t+1}\left( S^M\left( s_{t}^\text {L},x_{t}^\text {L},w_{t+1}^\text {L}\right) \right) \right] \right) }, \quad \forall s_t^\text {L}\in {\mathscr {S}} \end{aligned}$$

(4b)

$$\begin{aligned}&=\min _{x_t^\text {L}\in {\mathscr {X}}_t^\text {L}}{\left( z_t^\text {L}\left( x_t^\text {L}\right) +\sum _{\omega \in \varOmega _{t+1}}{p^{\varOmega _{t+1}}_\omega \left( V_{t+1}\left( S^M\left( s_{t}^\text {L},x_{t}^\text {L},\omega \right) \right) \right) }\right) }, \quad \forall s_t^\text {L}\in {\mathscr {S}} \end{aligned}$$

(4c)

3.5 Simplifications

Before presenting our integrated model of drayage and long-haul operations, we introduce our main simplifications compared to earlier work. We introduce these simplifications to ease the presentation of the integrated approach.

First, we assume that all freights that arrive at an initial long-haul terminal are “released” for transport (i.e., can be immediately transported in the long-haul after drayage). Second, we assume that trucking of freight between intermodal terminals is not allowed, i.e., once the drayage brings a freight to an intermodal terminal, it departs from that terminal by the long-haul mode. Third, we assume that all services (between intermodal terminals, and between an intermodal terminal and all destinations) have a duration of at most 1 day. Fourth, we assume that every service departs every day. These three assumptions can easily be generalized in the long-haul scheduling model, following the work from Pérez Rivera and Mes (2017b), but in this paper allow us to omit additional time information required in the state definition. Finally, we assume there is only one service between intermodal terminals i and j (both in ${\mathscr {N}}^\text {L}$) and one service between an intermodal terminal $i \in {\mathscr {N}}^\text {L}$ and all destinations $d\in {\mathscr {D}}^\text {L}$. As examples of the latter, we can have a truck performing the last mile delivery from an intermodal terminal i to a destination d, or a train making a round trip from an intermodal terminal i to a number of destinations d.

4 Solution approach

As mentioned earlier, we consider drayage scheduling and long-haul scheduling to be done separately, but with integrated objectives, since their decisions influence the performance of each other in synchromodal transport. In this section, we analyze each component for integration (Sects. 4.1 and 4.2), based on the models described in Sects. 3.3 and 3.4 respectively, and describe the integrated objective function (Sect. 4.3), outline our iterative solution approach (Sect. 4.4), present several challenges of our approach and how to overcome them (Sect. 4.5), and end with a complete overview of our solution approach (Sect. 4.6).

4.1 Modification of the drayage part

The only necessary modification to the drayage part, to integrate the long-haul transport process to drayage decisions, relates to the inclusion of terminal assignment cost for pre-haulage jobs. The terminal assignment costs should capture the consolidation opportunities throughout the network and throughout the time horizon that were mentioned in Sect. 1. Examples of such opportunities include economies of scale (e.g., reduction of cost per freight with an increasing number of freights brought to a terminal) and the combination of freights with similar destinations (e.g., there are freights present at a terminal that go to the same area of new drayage freights). To capture these costs, we use a function that depends on the freights $f_t^\text {L}$ that are already at the long-haul terminals, in addition to freights that are brought to those terminals by the drayage operations. We introduce the function $C_t^\text {L}(f_t^\text {L},x_t^\text {D})$ and re-define the drayage cost considering the long-haul using ${\tilde{z}}_t^{\text {D}} = z_t^\text {D} + C_t^\text {L}\left( f_t^\text {L},x_t^\text {D}\right)$ instead of (1). This modification is key to integrate the long-haul transport performance over time into the drayage operations, as we will elaborate upon later on.

To define $C_t^\text {L}\left( f_t^\text {L},x_t^\text {D}\right)$, we use the increment in the optimal expected costs $V_{t}\left( \cdot \right)$ of the Bellman’s recursion in (4), as shown in (5). This increment captures the “extra” costs in the long-haul when bringing drayage freights to terminals that already have freights. In other words, these marginal costs capture opportunities that arise, for example, when a terminal already has some freight for a specific long-haul destination and it is cheap to bring pre-haulage freights that have the same destination to that terminal. The costs $C_t^\text {L}\left( f_t^\text {D},x_t^\text {D}\right)$ are affected by the timing of drayage decisions (as exemplified in Fig. 2) in two ways. First, the freights $f_t^\text {L}$ at the terminals on day t form the so-called post-decision state $s_{t-1}^\text {L,x}$ of the previous day $t-1$. The post-decision state is the state of the long-haul terminals after a long-haul decision has been made but before the exogenous information arrives, or in our problem, before the drayage operations occur. The post-decision state is a function of the long-haul state and long-haul decision, as shown in (5c) and defined in Pérez Rivera and Mes (2017b). This state has an optimal expected cost of $V_{t}\left( s_{t-1}^\text {L,x}\right)$. Second, the long-haul state $s_{t}^\text {L}$ at day t and its expected downstream costs $V_{t}\left( s_{t}^\text {L}\right)$ depend on the drayage decision $x_t^\text {D}$ at day t and the post-decision state $s_{t-1}^\text {L,x}$ of the previous day $t-1$. Using the transition function $S^\text {M}$, we can define this state based on the post-decision state from the previous day $s_{t-1}^\text {L,x}$ and the drayage-dependent exogenous information $w_t^\text {L,x}$, as shown in (5b). Note that the $\emptyset$ in $S^\text {M}$ refers to the long-haul decision, which takes place after the drayage. Remind that $w_t^\text {L,x}$ is a function of the new freights ${\mathscr {F}}_t$ that arrived, randomly, for drayage and the drayage scheduling decision $x_t^\text {D}$ that brought them to the terminals, as shown in (5d).

$$\begin{aligned} C_t^\text {L}\left( f_t^\text {L},x_t^\text {D}\right)= & {} C_t^\text {L}\left( s_{t-1}^\text {L,x},x_t^\text {D}\right) = V_{t}\left( s_{t}^\text {L}\right) - V_{t}\left( s_{t-1}^\text {L,x}\right) \end{aligned}$$

(5a)

$$\begin{aligned}&\text {where}\nonumber \\ s_{t}^\text {L}= S^\text {M}\left( s_{t-1}^\text {L,x},\emptyset ,w_t^\text {L,x}\right) \end{aligned}$$

(5b)

$$\begin{aligned} s_{t-1}^\text {L,x}= S^\text {M,x}\left( s_{t-1}^\text {L},x_{t-1}^\text {L}\right) \end{aligned}$$

(5c)

$$\begin{aligned} w_t^\text {L,x}= S^\text {M,w}\left( {\mathscr {F}}_t,x_{t}^\text {D}\right) \end{aligned}$$

(5d)

4.2 Modification of the long-haul part

The modification of the long-haul part lies in the stochastic arrival process of freights to the network considered, i.e., the exogenous information process $\left[ \varOmega _t\right] _{\forall t\in {\mathscr {T}}}$. In our problem, freights arrive randomly to the drayage part of the network, and subsequently, after the decisions of the drayage, to the long-haul terminals. Consequently, the exogenous information process (i.e., freight arrivals) for the long-haul scheduling is stochastic and drayage-dependent. To model such an exogenous information process, we define probabilities per intermodal terminal in the drayage, as it is considered in the model of Pérez Rivera and Mes (2017b), with the relation that one “origin” in their model is one departure terminal in ours. These probabilities capture the drayage dependency of arrivals. In other words, the probabilities in our model capture the probability that a certain freight type is brought to a certain terminal, opposed to certain freights becoming known to the system. Using our previously defined probability distributions of freight arrivals, we introduce the set of drayage-dependent probabilities ${\mathscr {P}}^\text {L}$ that is used in the MDP model and encloses the following probabilities. In between two consecutive days $t-1$ and t, a total of $f\in {\mathbb {N}}$ freights arrive with probability $p^\text {FL}_{f,j,t}$ to terminal $j\in {\mathscr {N}}^\text {L}$. A freight that arrives between days $t-1$ and t to terminal $j\in {\mathscr {N}}^\text {L}$ has destination $d\in {\mathscr {D}}^\text {L}$ with probability $p^\text {DL}_{d,j,t}$, and time-window length $l\in {\mathscr {T}}^\text {L}$ with probability $p^\text {LTL}_{l,j,t}$. The probabilities in ${\mathscr {P}}^\text {L}$ describe all freights that arrive to the long-haul based on the probabilities in ${\mathscr {P}}^\text {D}$ and decision $x_t^\text {D}$. The integration of the long-haul scheduling and the drayage scheduling is completely described by the relation between ${\mathscr {P}}^\text {L}$ and decision $x_t^\text {D}$.

4.3 Integrated problem

In the long-haul scheduling model, drayage performance is captured indirectly through the exogenous information process $\left[ \varOmega _t\right] _{\forall t\in {\mathscr {T}}}$. The exogenous information $\omega \in \varOmega _t$, and its corresponding probability $p^{\varOmega _{t}}_\omega$, is partly due to the stochastic arrival of freights to the drayage (i.e., ${\mathscr {P}}^\text {D}$) and the drayage decision (i.e., $x_t^\text {D}$). Therefore, the solution to the Bellman equations in (4) is dependent on the drayage decisions. In turn, drayage decisions are dependent on the solution of the Bellman equations required for the long-haul decisions. The challenge to solve the integrated scheduling problem considering this recursion is to find a set of long-haul drayage-dependent probabilities ${\mathscr {P}}_{\pi }^{\text {L}}$ for a policy $\pi \in \varPi$ that has two properties: (1) it results in a stable recursion, i.e., a solution to the Bellman equations that does not modify the drayage scheduling decisions that gave rise to its input long-haul drayage-dependent probabilities, and (2) it minimizes the overall costs (i.e., drayage plus long-haul) defined in (6a). We now discuss the components of this objective function and the recursion therein.

$$\begin{aligned}&\min _{\pi \in \varPi }{\left. {\mathbb {E}}\left[ \sum _{t\in {\mathscr {T}}}{\left( z_t^\text {D}\left( x_{t,\pi }^{\text {D}}\right) + z_t^\text {L}\left( x_{t,\pi }^\text {L}\right) \right) }\right| s_0^\text {L},{\mathscr {P}}^\text {D},\varGamma \right] }\end{aligned}$$

(6a)

$$\begin{aligned}&\quad \text {where}\nonumber \\&x_{t,\pi }^{\text {D}} = \mathop {\hbox {arg min}}\limits _{x_{t}^{\text {D}}\in {\mathscr {X}}_t^\text {D}}{\left[ {\tilde{z}}_{t,\pi }^{\text {D}}\left( x_{t}^{\text {D}}\right) \right] } \end{aligned}$$

(6b)

$$\begin{aligned}&{\mathscr {P}}_{\pi }^{\text {L}} = \varGamma \left( {{\mathscr {P}}^\text {D},{\left[ x_{t,\pi }^{\text {D}}\right] _{\forall t\in {\mathscr {T}}}}}\right) \end{aligned}$$

(6c)

In the integrated objective function (6a), there are two new components to both the drayage and long-haul scheduling models: the drayage decisions dependent on a long-haul policy $\left( x_{t,\pi }^{\text {D}}\right)$, and the function that defines the long-haul probabilities from the drayage decisions $\left( \varGamma \right)$. The drayage decision $x_{t,\pi }^{\text {D}}$ at time $t\in {\mathscr {T}}$ given policy $\pi \in \varPi$ is defined as the decision that minimizes the drayage cost considering the long-haul ${\tilde{z}}_t^{\text {D}}$, as shown in (6b). Note that ${\tilde{z}}_t^{\text {D}}$ has a subscript $\pi$ to denote that the Bellman equation costs used are those of that policy, i.e., $V_{t,\pi }$. Note also that the costs we consider in the integrated objective in (6a) are the drayage costs only, i.e., $z_t^{\text {D}}$, such that the long-haul costs are not added twice since they are already included in $z_t^{\text {L}}$. The function $\varGamma$ maps the probabilities of the drayage ${\mathscr {P}}^\text {D}$, and the drayage decisions that consider the long-haul $\left[ x_{t,\pi }^{\text {D}}\right] _{\forall t\in {\mathscr {T}}}$ under policy $\pi$, to the probabilities of the long-haul ${\mathscr {P}}_{\pi }^{\text {L}}$ under policy $\pi$, as shown in (6c). It is in these two components that the recursion lies, as exemplified in Fig. 3: to compute ${\mathscr {P}}_{\pi }^\text {L}$, we need $x_{t,\pi }^{\text {D}}$, which depends on $V_{t,\pi }$, which in turn depends on ${\mathscr {P}}_{\pi }^\text {L}$. Note that the recursion occurs only when computing the value functions $V_{t,\pi }$ and that, once the value functions are computed, they do not change with daily long-haul decision-making in practice (i.e., the recursion is not present when implementing in practice). Considering this recursion, we have two approaches to solve the problem: either we consider one optimization problem with a large system of equations for the evolution of the network and constraints for drayage and long-haul scheduling, or we consider an iterative approach where we split the problem into several steps and iterations. Since a close-form solution for the overall optimization problem may only exist under limiting assumptions (e.g., the definition of $\varGamma$), we focus on an iterative approach. In the following section, we describe the iterative method for computing ${\mathscr {P}}_{\pi }^\text {L}$ and solving the recursion.

Fig. 3

Components of the recursion between drayage and long-haul decisions

4.4 Iterative integration

To solve the optimization problem in (6a), we propose a simulation-based approach connecting the MILP and MDP models for the drayage and long-haul scheduling, respectively. In this approach, we iteratively define ${\mathscr {P}}_{\pi }^{\text {L}}$ using the costs and decisions from policy $\pi$. Our approach, which we call iterative integration, consists of four steps:

Step (A) :

Define the long-haul probabilities ${\mathscr {P}}_{\pi }^\text {L}$ using $\varGamma$ as in (6c). If there are observed drayage decisions (i.e., we are arriving at this step from Step D), then $\varGamma$ is used as an updating function.

Step (B) :

Solve the Bellman equations in (4) using the exogenous information from the probabilities ${\mathscr {P}}_{\pi }^\text {L}$ resulting from Step (A). Store the expected long-haul costs $V_{t,\pi }\left( s_t^\text {L}\right)$ for all states and days in the horizon.

Step (C) :

Simulate the drayage and long-haul scheduling policies resulting from the Bellman equations, for the entire horizon, and for a number of replications. Each replication yields an observation of drayage and long-haul costs, ${\hat{z}}_{\pi }^\text {D}$ and ${\hat{z}}_{\pi }^\text {L}$, respectively, as well as an observed set of drayage decisions $\left[ {\hat{x}}_{t,\pi }^{\text {D}}\right] _{\forall t\in {\mathscr {T}}}$.

Step (D) :

Decide whether to continue to the next iteration and go back to Step (A) or to stop.

The four steps of the iterative integration method are shown in Fig. 4. In addition, we show a second approach which we call the sequential integration method. This method consists of Steps (A) and (B) of the iterative integration only, or in other words, only a single iteration. This method is computationally simpler and can be used, for instance, when there is an analytical expression to define the drayage-dependent arrival rates for each terminal, or when these probabilities can be defined based on experience of human planners or historical data. We consider both approaches for our experiments, as we explain later on in Sect. 5. For now, we focus on the challenges of the iterative integration.

Fig. 4

Proposed solution methods to the integrated scheduling model

Each step of the iterative integration has its own challenges. Steps (A) and (D) have challenges that are intrinsic to the simulation-based approach connecting the MILP and MDP models. First, defining long-haul probabilities ${\mathscr {P}}_{\pi }^\text {L}$ involves defining the function $\varGamma$. Second, deciding whether to stop or to continue involves the assessment of whether the policy $\pi$ is good enough, or if it converged (i.e., continuing does not have any effect). Steps (B) and (C) involve the computational challenges due to increasing complexity of the MILP and MDP models with increasing problem size. In the following sections, we elaborate on these challenges and propose ways to overcome them.

4.5 Challenges

Steps (A) and (D) are the fundamental steps where our iterative approach defines the information parameters that integrate drayage and long-haul decisions shown in Fig. 3: the long-haul freight arrival information ${\mathscr {P}}_{\pi }^{\text {L}}$ and the drayage terminal assignment cost $\left[ V_{t,\pi }\right] _{\forall t\in {\mathscr {T}}}$. Remind that ${\mathscr {P}}_{\pi }^{\text {L}} = \varGamma \left( {{\mathscr {P}}^\text {D},{\left[ x_{t,\pi }^{\text {D}}\right] _{\forall t\in {\mathscr {T}}}}}\right)$. The general objective in these steps is to define the parameters that achieve the overall objective in (6).

In Step (A), a function $\varGamma$ that maps the probabilities of drayage freight arrivals ${\mathscr {P}}^\text {D}$ to the probabilities of the long-haul freight arrivals ${\mathscr {P}}_{\pi }^{\text {L}}$ must be defined. The challenge is twofold. First, the dependence of ${\mathscr {P}}_{\pi }^{\text {L}}$ in the drayage decisions $\left[ x_{t,\pi }^{\text {D}}\right] _{\forall t\in {\mathscr {T}}}$ that consider the long-haul is recursive, and hence the mapping complex to define. Second, the influence of ${\mathscr {P}}_{\pi }^{\text {L}}$ on the overall costs/objective is indirect and difficult to measure. For the long-haul costs, the MDP model yields direct expected costs for a given ${\mathscr {P}}_{\pi }^{\text {L}}$, but for the drayage costs and the integrated costs, the MILP and the MDP model interact in a difficult way to measure and derive costs. This difficulty is also present in Step (D), where a decision whether to continue to the next iteration must be made. The main criteria is whether the policy $\pi$ from ${\mathscr {P}}_{\pi }^{\text {L}}$ results in sufficiently low overall costs. However, as stated before, this is difficult to measure and other criteria may be necessary, such as convergence of costs or convergence of the policy.

In contrast to the modeling challenges in Steps (A) and (D), Steps (B) and (C) have computational challenges with increasing problem size due to the complexity in solving the individual MILP and MDP models. Per iteration, the MDP model has to be solved once, and the MILP model $\left| {\mathscr {T}}\right| *N$ times, where N is the number of simulation runs and $\left| {\mathscr {T}}\right|$ the length of the planning horizon, per iteration. To overcome these challenges, we use the Math-Heuristic (MH) for the MILP model from Pérez Rivera and Mes (2017c) and the Approximate Dynamic Programming (ADP) algorithm for the MDP model from Pérez Rivera and Mes (2017b). We apply two modifications to the MH and one modification to the ADP algorithm, which we elaborate upon next. For all details on these heuristics, we refer the reader to the aforementioned papers.

In the ADP algorithm, we modify the Value Function Approximation (VFA) by designing a new set of characteristics ${\mathscr {B}}_t$, or basis functions $\phi _{b}\,\forall b\in {\mathscr {B}}_t$, that approximate the downstream rewards in the Bellman’s equations. This new VFA provides a better fit with the integrated problem and works as follows. We categorize freights based on a common freight description used in transport problems: must go and may go freights. On a given day t, must go freights are those freights due at t, i.e., time-window length equal to zero, and may go freights are those freights that can be postponed, i.e., time-window length greater than zero. Based on this categorization, we include the following four basis functions per terminal $j\in {\mathscr {H}}$: (1) number of must go freights, (2) count of must go destinations, (3) number of may go freights, and (4) count of may go destinations, as shown in (7a) to (7d), respectively. To count the must go destinations, we use the binary variables $u_{j,d,t}$, which get a value of 1 if there is a must go freight to destination $d\in {\mathscr {D}}^\text {L}$ in terminal $j\in {\mathscr {H}}$ and 0 otherwise. Similarly, we use the binary variable $v_{j,d,t}$ to count the may go destinations. In addition to the must go and may go freight basis functions, we include the total number of freights of a post-decision state in (7e), a constant in (7f), and all post-decision state variables, as shown in (7g). In total, we now have $\left| {\mathscr {B}}_t\right| =\left| s_{t}^\text {L}\right| +6$ characteristics (also denoted by features), indexed from $\left[ 1,2,\ldots ,\left| {\mathscr {B}}_t\right| \right]$, using the auxiliary function $b(d,l,t)\mapsto \left[ 7,8,\ldots ,7+\left| s_{t}^\text {L}\right| \right]$.

$$\begin{aligned} \phi _{1,j}\left( s_{t}^\text {L}\right)= & {} {\sum _{d\in {\mathscr {D}}^\text {L}}{{f_{j,d,0,t}^\text {L}}}} \end{aligned}$$

(7a)

$$\begin{aligned} \phi _{2,j}\left( s_{t}^\text {L}\right)= & {} \sum _{d\in {\mathscr {D}}^\text {L}}{u_{j,d,t}} \end{aligned}$$

(7b)

$$\begin{aligned} \phi _{3,j}\left( s_{t}^\text {L}\right)= & {} {\sum _{d\in {\mathscr {D}}^\text {L}}{\sum _{l\in {\mathscr {T}}\setminus 0}{f_{j,d,l,t}^\text {L}}}} \end{aligned}$$

(7c)

$$\begin{aligned} \phi _{4,j}\left( s_{t}^\text {L}\right)= & {} \sum _{d\in {\mathscr {D}}^\text {L}}{v_{j,d,t}} \end{aligned}$$

(7d)

$$\begin{aligned} \phi _{5,j}\left( s_{t}^\text {L}\right)= & {} {\sum _{d\in {\mathscr {D}}^\text {L}}{\sum _{l\in {\mathscr {T}}}{f_{j,d,l,t}^\text {L}}}} \end{aligned}$$

(7e)

$$\begin{aligned} \phi _{6,j}\left( s_{t}^\text {L}\right)= & {} 1 \end{aligned}$$

(7f)

$$\begin{aligned} \phi _{b(d,l,t),j}\left( s_{t}^\text {L}\right)= & {} f_{j,d,l,t}^\text {L}, \quad \forall d\in {\mathscr {D}}^\text {L}, \, l\in {\mathscr {T}} \end{aligned}$$

(7g)

The VFA is then the product of the basis functions and weights $\theta _{b,j,t}$ as shown in (8). This VFA is used in the drayage costs that consider the incremental long-haul costs shown in (5). $V_t\left( \cdot \right)$ captures the expected long-haul performance for all terminals, but can also be used for individual terminals, as we see next.

$$\begin{aligned} V_{t}\left( s_{t}^\text {L}\right) - V_{t}\left( s_{t-1}^\text {L,x}\right) \approx \sum _{j\in {\mathscr {H}}}{\sum _{b\in {\mathscr {B}}}{\theta _{b,j,t}\cdot \phi _{b,j}\left( s_{t}^\text {L}\right) }}-\sum _{j\in {\mathscr {H}}}{\sum _{b\in {\mathscr {B}}}{\theta _{b,j,t}\cdot \phi _{b,j}\left( s_{t-1}^\text {L,x}\right) }} \end{aligned}$$

(8)

For the MH approach to solve the MILP model, we include an additional Math-Heuristic Operator (MHO). This MHO is based on the long-haul performance resulting from the VFA described before, as shown in (9). For pre-haulage jobs $F\in {\mathscr {F}}_t$, we fix the arc to the terminal with lowest integrated costs (i.e., drayage costs and expected increase in long-haul costs), as shown in (9a). If the pre-haulage job needs an empty container, we use one from the closest terminal, as shown (9b).

$$\begin{aligned}&\sum _{k\in {\mathscr {K}}}{x_{k,i,j',t}}=1\nonumber \\&\quad \text {such that}\nonumber \\&j'=\mathop {\hbox {arg min}}\limits _{j\in \delta ^{+}(i)}\left( T_{i,j}\cdot C_{k,i,j}^\text {VD}+\sum _{j\in {\mathscr {H}}}{\sum _{b\in {\mathscr {B}}}{\theta _{b,j,t}\cdot \phi _{b,j}\left( s_{t}^\text {L}\right) }}-\sum _{j\in {\mathscr {H}}}{\sum _{b\in {\mathscr {B}}}{\theta _{b,j,t}\cdot \phi _{b,j}\left( s_{t-1}^\text {L,x}\right) }}\right) \end{aligned}$$

(9a)

$$\begin{aligned}&\sum _{k\in {\mathscr {K}}}{x_{k,j',i,t}}=1\left| j'=\mathop {\hbox {arg min}}\limits _{j\in \delta ^{-}(i)}{\left( T_{j,i}\cdot C_{k,i,j}^\text {VD}\right) }\right. \end{aligned}$$

(9b)

4.6 Heuristic approach

With the MH and ADP heuristics, our iterative integration method works as shown in Fig. 5. First, ADP learns the VFA (i.e., weights $\theta$ of the basis functions described in the previous sections) based on the initial long-haul arrival probabilities ${\mathscr {P}}^\text {L}$. Then, a simulation is carried out where freights arrive randomly for the drayage. These freights are scheduled using the MH described in the previous section, considering the learned VFA. After N simulation runs, the observed long-haul arrivals $w^\text {L,x}$ for all days in the horizon and simulation runs are given to the probability updating function $\varGamma$, which yields new long-haul arrival probabilities. The process is repeated M times. Note that in Fig. 5, parameters that vary per simulation run have a subscript n, and those that vary per iteration of the integrated approach (but remain constant through the simulation) have a subscript m. Furthermore, the ADP algorithm runs for a number of internal-iterations at each iteration m and the MH runs for a number of internal-iterations at each time t, of replication n, and of iteration m.

Fig. 5

Graphical representation of the iterative integration method

5 Numerical experiments

In this section, we study the performance of our integrated scheduling approach through a series of numerical experiments. We explore the use of our integrated approach and evaluate its implications under various problem characteristics. This section is structured as follows. We begin by describing the transport network and the three phases of experiments we carry out. Then, we present the results of each experimental phase and our observations. Finally, we discuss our results and provide ideas for future research.

5.1 Experimental setup

We consider a transport network with a point-to-point topology and three long-haul services, as shown in Fig. 6. The network spans an area of 1000 $\times$ 500 km, to resemble the distances that make consolidation (e.g., using barges and trains) for the long-haul desirable in Europe (Woxenius 2007). The origin of pre-haulage freights, as well as all other drayage request locations, occurs in an area of 200x500km around Terminals 1, 2, and 3. There are 25 trucks, of which 9 are located at Terminal 1, and 8 at each of the Terminals 2 and 3. There are six long-haul destinations for pre-haulage freights, which are evenly distributed between Terminals 4, 5, and 6. There is the possibility to hire additional trucks for the long-haul, at a much larger expense than the trains and barges in the network. We use a literature-based cost structure, similar as the one in Pérez Rivera and Mes (2017b), where drayage operations, on average, contribute 40% to the total transport costs. The cost details and parameters are shown in “Appendix 2”.

Fig. 6

Locations and transport modes of the experimental network

We consider a planning horizon of 5 days, i.e., one working week. Each day, new drayage freights arrive according to a right-truncated Poisson distribution with mean 20 and truncated at 25. Each freight that arrives has four random characteristics: (1) drayage location, (2) drayage request type, (3) long-haul destination, and (4) long-haul time-window length. The drayage location is uniformly distributed in an area of 200 $\times$ 500 km around Terminals 1, 2, and 3, as exemplified in Fig. 6. For the Drayage Request (DR) type, we consider four categories. Category DR1 represents an end-haulage request starting with a full container at a known terminal, transporting it to a customer (i.e., drayage location), unloading it, and then finishing with the transport of the empty container to any terminal. Category DR2 represents an end-haulage request starting with a loaded container at a known terminal and finishing with its transport to the customer. Category DR3 represents a pre-haulage request starting with an empty container in any of the terminals, transporting it to a customer, loading it, and then finishing with the transport of the loaded container to any terminal. Category DR4 represents a pre-haulage request starting with a loaded container at a customer and finishing with its transport to any terminal. A drayage request is category DR1 with probability 0.1, DR2 with 0.1, DR3 with 0.4, and DR4 with 0.4. Each pre-haulage freight that arrives, independent of its type (i.e., DR3 or DR4), has long-haul destination 7, 8, 9, or 10 with probability 0.05 each, and destination 11 or 12 with probability 0.4 each. Furthermore, each pre-haulage freight has a long-haul time-window length of 1 or 3 days with probability 0.1 each, and 2 days with probability 0.8.

We use a total of 8 combinations of network characteristics, which we call instances. These instances are based on the network described above, considering variations to three of the stochastic characteristics considered. For the drayage location distribution, we study two settings: the random one described before and a clustered one consisting of drayage locations distributed uniformly around Terminal 3 in a square of 150 $\times$ 150 km. For the drayage type distribution, we study two settings: the original setting, which we denote by pre-haulage dominant, and an end-haulage dominant setting where drayage requests are of type DR1 or DR2 with probability 0.4 each, and are type DR3 or DR4 with probability 0.1 each. For the long-haul destination, we study two settings: the unbalanced distribution described before and a balanced distribution where each destination has probability of 1 / 6. These network characteristics, and the notation we use to refer to them, are shown in Table 1.

Table 1 Network characteristics and their settings

The goal of our numerical experiments is to study freight demand characteristics that have a direct impact on the integration of drayage and long-haul transport scheduling and how an integrated approach compares to a non-integrated one. For this reason, we consider that only the aforementioned freight demand characteristics are random and that all others settings are fixed. The drayage time-window length, maximum driving time of drayage trucks, and terminal opening times are equal to 1 day. Furthermore, all pre-haulage freights that arrive are released for the long-haul, i.e., once arrived at the long-haul, all pre-haulage freights are allowed to be taken by the first long-haul service departing. It is important to note that our models and algorithms can handle these characteristics as stochastic, but that in our experiments we consider them deterministic. We divide the experiments into three phases where we use the sequential integration method (see Fig. 4) in the first two phases, and the iterative integration method in the third phase. Remind that the sequential integration method is a special case of the iterative integration method in Fig. 5 with $M=1$.

Experiments with the integrated scheduling method require both a learning and an evaluation phase. In the learning phase, the approach from Figs. 4 or 5 is carried out, and results in a policy that is tested in an evaluation phase, using simulation of 100 weeks. To prevent randomness in the learning phase (i.e., simulation used in ADP) from affecting the methods, the same policy is learned for the three methods. To prevent randomness in the evaluation phase, the same freight arrivals over the horizon of the problem (i.e., 1 week) and over the simulation runs (i.e., 100 weeks) are used for evaluating the four scheduling methods. Furthermore, for reducing the statistical noise of learning the policies, we replicate the whole learning and evaluation phase 10 times for the first two experimental phases (i.e, with the sequential integration method). Across the replications, we observed an average relative error of 0.03 (at a t confidence level of $\alpha =0.05$) for all costs performance indicators.

Our integration method, both in its sequential and iterative form, requires careful calibration of various parameters. Before performing our experimental phases, we study the calibration of four parameters: (1) ADP initial probability distributions, (2) ADP time horizon, (3) ADP initial weights, and (4) ADP updating step size. These parameters have a direct influence on the expected long-haul costs for the planning horizon, which affect the terminal assignment decision in the drayage and consequently the freights available for consolidation in the long-haul. We describe the details of this calibration study in “Appendix 4”.

We now discuss the purpose of each of the experimental phases of our numerical study. First, we perform an exploratory phase where we study the relative costs savings of our integrated approach compared to the costs of using a non-integrated approach commonly found in practice (Sect. 5.2). Second, we perform an evaluation phase where we delve into the specific consequences of our integrated approach towards each part of the transport process: the drayage operations and the long-haul transport (Sect. 5.3). Third, we perform an convergence test phase where we study the behavior of our iterative integrated approach throughout the iterations (Sect. 5.4). After presenting the results of the three phases, we discuss the limitations of our experiments and provide future research ideas to overcome them in Sect. 5.5.

5.2 Phase I: exploratory experiments

In the exploratory experiments, we evaluate the relative cost savings of our integrated scheduling approach over a non-integrated scheduling approach using various cost setups. The non-integrated scheduling approach we use as a benchmark consists of two independent heuristics, one for the long-haul and one for the drayage. The long-haul heuristic consolidates freights that must be sent today with the lowest cost possible (i.e., combination of high-capacity modes and hired trucks), and, if there is capacity left in the high-capacity mode, it schedules freights that do not add extra costs, such as freights with the same destination. The drayage heuristic uses the logic from Caris and Janssens (2009): first, it pairs drayage requests that are close to each other (i.e., assign closest terminal), second it adds the pairs to a route using cheapest insertion, and finally schedules all remaining requests using cheapest insertion.

We evaluate the savings in the eight instances obtained by the combination of network characteristics from Table 1. Furthermore, for each of the eight instances, we test two additional cost setups: (1) low drayage-cost, where drayage costs make up, on average, 20% of the total costs instead of the 40% in the normal setup, and (2) high drayage-cost, where drayage costs make up, on average, 80% of total costs. The results for the normal, low and high drayage-cost setup are shown in Tables 2, 3, and 4 respectively.

Table 2 Percentage difference with the benchmark in normal drayage-cost setup

In Table 2, we observe that our approach achieves total costs savings between 2 and 6% in 7 out of the 8 instances. Two interesting observations arise. First, the relative savings are larger in instances with clustered drayage locations (C). In these instances, all drayage freights are closer to Terminal 3 than any other terminal. However, this might not always be the departure terminal. Hence, we observe that with larger total costs savings there are increased drayage costs due to pre-haulage freights not being brought to their closest terminal. Second, we observe that instances with end-haulage dominant drayage types (E) have a lower long-haul utilization (of barges and trains). These instances have significantly less pre-haulage freights compared to the others. Our approach seems to postpone more freights for their long-haul transport than the benchmark. Although it is intuitive to postpone as long as possible to seek a larger number of freights per barge or train when not many pre-haulage freights arrive, it may result in poor long-haul utilization during the postponement periods and higher costs when there are suddenly more freights than barge/train capacity available at a terminal (e.g., Instances R–E–U and C–E–U).

Table 3 Percentage difference with the benchmark in low drayage-cost setup

Table 4 Percentage difference with the benchmark in high drayage-cost setup

In Table 3 with the low drayage-cost setup, similar observations to those from Table 2 arise. The only difference is in the magnitude, with larger savings and less pre-haulage freights assigned to their closest terminal being the intuitive result of drayage operations not contributing significantly to the total costs. In such a cost setup, the choice of transporting pre-haulage freights to the best departure terminal is certainly more important than transporting it to the closest departure terminal.

In Table 4, showing the results of the experiments with high drayage-cost setup, we observe two important differences. First, we observe that the total cost savings in all clustered drayage origin instances (C) are significantly larger than in the other cost setups. These savings seem to arise due to the increased assignment of closer departure terminals to the pre-haulage freights. On the one hand, since more freight are being brought to the same departure terminal (Terminal 3 in this case), the chances of not having enough capacity in the long-haul service of that terminal, and thus having to use the expensive truck, are higher. This explains the increased long-haul costs in the C–P instances. On the other hand, if not so many freights arrive, as in the case of the end-haulage dominant drayage instances C–E, savings for the long-haul are still possible. Second, our approach achieves total cost savings in only 1 out of the 4 random drayage origin (R) instances tested, in instance R–E–U, which was the instance where no savings were achieved in the previous cost setups. The savings are a result of drayage improvement, where significantly more pre-haulage freights were brought to their closest departure terminal than in the other R instances. Although these observations may be due to the drayage costs being much higher than the long-haul costs (e.g., the drayage operations should have a stronger emphasis than in our approach), one can argue that if our approach would properly learn the VFA, and a reasonable policy for the drayage scheduling using the VFA, then it would not perform worse than the non-integrated benchmark. In Sect. 5.5, we discuss these observations and their implications.

In the exploratory experiments above, we observed that the network-wide relative savings of our integrated approach have different implications for different network characteristics. In the following phase, we study in more detail what these implications are in absolute terms.

5.3 Phase II: evaluation experiments

In the evaluation experiments, we analyze the specific consequences of our integrated approach towards drayage operations and long-haul transport, and compare them to those of the non-integrated benchmark heuristic of the previous section. To further analyze the impact of our integrated approach on the individual scheduling of drayage operations or of long-haul transport, we compare to the scheduling case when only one of the two policies of our approach is being used, and in the other part, the benchmark heuristic. In Table 5, we show the two-letter notation used for each scheduling method, where the first letter denotes the scheduling method of the drayage operations and the second one the scheduling method for the long-haul transport. We use H to denote the individual non-integrated heuristics described in the previous section, M for matheuristic policy from our integrated approach (i.e., drayage part), and A for the ADP policy from our integrated approach (i.e., long-haul part).

Table 5 Different scheduling methods for evaluation

Similar to the previous experiments, we use the eight instances obtained by the combination of network characteristics from Table 1 under the normal drayage-cost setup. For each instance, we test the four scheduling methods of Table 5. Methods H–A, M–H, and M–A require our integrated approach to learn the two output policies, even if only one will be used for the actual scheduling, i.e., H–A and M–H. In all three cases, we must learn the VFA of the long-haul transport: in H–A we use it for the long-haul decisions, in M–H we use it in drayage decisions, and in M–A we use it in both.

Fig. 7

Average total (drayage+long-haul) costs for the pre-haulage dominant (P) instances

In Fig. 7, we observe the average costs of each scheduling method for the pre-haulage dominant instances. In the exploratory experiments of these instances (previous section), we saw that our integration method (M–A) achieved savings between 2 and 6% compared to the benchmark (H–H). In these evaluation experiments, we observe similar savings with two additional observations on which of the policies helps the most. First, we observe that for the instances with random drayage locations (R), the methods with the ADP policy for the long-haul (i.e., H–A and M–A) are performing the best; whereas in the instances with clustered drayage locations (C), the methods with the matheuristic policy for the drayage (i.e., M–H and M–A) are performing the best. In the R instances, it seems that more emphasis should be put on when to use the long-haul services once a freight is brought to a departure terminal, since pre-haulage freight arrivals are evenly distributed among the departure terminals. In the C instances, choosing the best departure terminal is more important for a good long-haul and network-wide costs, even at the expense of the drayage costs, since pre-haulage freight arrivals are clustered in one terminal.

Table 6 Performance of different scheduling methods for the pre-haulage dominant (P) instances

In Table 6, we observe further implications of the emphasis on the drayage and long-haul policies of our integrated approach. Two implications stand out. First, in all instances, having a long-haul policy that is good (i.e., low number of long-haul trucks and low cost per freight) is crucial for the best network-wide performance. In fact, in three out of the four instances, the scheduling method achieving the lowest network-wide costs used the lowest number of long-haul trucks. Second, in the R instances, both the drayage and long-haul policy should achieve low costs per freight for their part to achieve the best network performance; whereas in the C instances, the emphasis is on organizing the drayage in a way to allow the long-haul transport to have a better performance. This result is intuitive when considering that myopic drayage decisions in the C instances will most of the time bring pre-haulage freight to the closest terminal, resulting in overflow of freight and high use of trucks when long-haul (barge or train) capacity is exceeded. Finally, Table 6 also indirectly provides insight into the utilization of the long-haul services through the number of trucks required for the long-haul and the number of requests assigned to the closest terminal. For example, in Instance C–P–B, our integrated approach (M–A) uses three times less long-haul trucking compared to the benchmark (H–H) at the expense of choosing further terminals in the drayage (i.e., smaller number of requests to closest terminal). This implies a “distributed” preference to terminals (and thus long-haul services) instead of the closest one to the cluster, which results in better consolidation opportunities, less postponement of freight in terminals, and better long-haul costs.

In the case of end-haulage dominant (E) instances, whose results are shown in “Appendix 5”, there are no significant differences among the total cost performance of the scheduling methods. Similar to the P instances, the scheduling methods with the matheuristic (M) drayage policy are assigning less freights to their closest terminal, or in other words, trying to achieve better future consolidation by bringing freight to a further terminal. In contrast to the P instances, a lower cost per freight of the long-haul policy does not relate to a lower number of long-haul trucks used. A possible explanation for this might be the excessive postponement of freight, as discussed previously. A critical point and improvement could be the use of thresholds (such as when operators call to see if there is space or not in a terminal) to balance more things and avoid the postponement or too-crowded terminal. Finally, in the end-haulage dominant (E) instances, the relative savings of the integrated approach are smaller than in the pre-haulage dominant instances. In instances R–E–U and C–E–U, the benchmark heuristic seems to slightly outperform our approach. Since the integrated approach typically steers drayage decisions towards efficient consolidation for the long-haul, instances where drayage cost dominate do not benefit as much from an integrated approach.

Overall, our sequential integrated approach, when used with the best scheduling method (displayed in bold in Table 6), results in total cost savings with respect to a non-integrated approach (scheduling method H–H) between 4 (R–P–B) and 24% (C–P–B) in the P instances. The savings in the long-haul cost per freight in these instances range between 5 (R–P–B) and 39% (C–P–B), without any increase in drayage cost in both instances with random drayage locations (R–P–U and R–P–B) but with a 48% increase in drayage cost in both instances with clustered drayage locations (C–P–U and C–P–B). In the following section, we explore the use of our iterative integrated approach to overcome some of the limitations of the sequential one.

5.4 Phase III: convergence test

In this convergence test, we study the behavior of our iterative integrated approach throughout the iterations. For the iterative approach, depicted in Fig. 5, we start by using the same sequential approach of the previous experimental phases and use 100 iterations ($M=100$), with the difference that, from iteration $m=2$ onwards, we initialize the long-haul policy (i.e., basis function weights of the ADP) at each iteration with the long-haul policy from the previous iteration. Furthermore, throughout the iterations we need a probability updating function $\varGamma$, which we describe next.

We define the long-haul probabilities ${\mathscr {P}}^\text {L}_m$ at every iteration m based on the observed distribution of arrivals across the departure terminals and the network-wide distribution of arrivals to the entire network. The intensity of arrivals to Terminal $i\in {\mathscr {J}}$ at iteration $m|1<m<M$ is updated to be a right-truncated Poisson distribution with mean $\lambda _{i,m}$ and truncated at 25. The mean $\lambda _{i,m}$ is defined by Eq. 10, where $a_{i,m-1}$ represents the total arrivals to Terminal i at iteration $m-1$. The probability $p^{\text {D}}_{i,d,m}$ of an arrival at Terminal $i\in {\mathscr {H}}$ having destination $d\in {\mathscr {D}}^\text {L}$ at iteration m is defined by Eq. 11, where $o^{\text {SD}}_{i,d,m-1}$ represent the scaled arrivals to Terminal i at iteration $m-1$ that had destination d and is defined by Eq. 12. The probability of an arrival at a terminal having a long-haul time-window length l is computed similarly to that of the destination.

$$\begin{aligned} \lambda _{i,m}= & {} 20\frac{a_{i,m-1}}{\displaystyle \sum \nolimits _{j\in {\mathscr {H}}}a_{j,m-1}} \end{aligned}$$

(10)

$$\begin{aligned} p^{\text {D}}_{i,d,m}= & {} \frac{o^{\text {SD}}_{i,d,m-1}}{\displaystyle \sum \nolimits _{d\in {\mathscr {D}}^\text {L}}{o^{\text {SD}}_{i,d,m-1}}}\end{aligned}$$

(11)

$$\begin{aligned} o^{\text {SD}}_{i,d,m}= & {} o^{\text {D}}_{i,d,m}\frac{p^\text {DL}_{r,d^\text {L}}\displaystyle \sum \nolimits _{d\in {\mathscr {D}}^\text {L}}{\sum \nolimits _{j\in {\mathscr {H}}}{o^{\text {D}}_{j,d,m}}}}{\displaystyle \sum \nolimits _{j\in {\mathscr {H}}}{o^{\text {D}}_{j,d,m}}} \end{aligned}$$

(12)

Similar to the previous phases, we use the eight instances obtained by the combination of network characteristics from Table 1 under the normal drayage-cost setup. In Table 7, we show the initial ($m=1)$ and final ($m=100$) performance indicators of the iterative method and their percentage difference. Two observations stand out. First, in the end-haulage dominant (E) instances, the iterative method always improves the total cost, and in the pre-haulage dominant (P) not. Second, although all E instances have lower total costs and long-haul cost per freight at the end of the iterations, the closest terminal assigned decreases in the random drayage location (R) instances and increases in the clustered drayage location (C) instances. In the previous phases, E instances were the instances where we observed the lowest savings and where the combination of scheduling policies did not differ much. The results of this phase show that with an iterative method, the performance of our integrated approach can be improved further. We now analyze two of the instances in more detail.

Table 7 Initial and final performance indicators of the iterative method

Fig. 8

Iterative method in R–E–U

Fig. 9

Iterative method in C–P–B

In Figs. 8 and 9, we show two indicators throughout the iterations for Instance R–E–U and C–P–B, respectively. We show the long-haul cost per freight, which is an indicator of the long-haul policy, and the assignments of pre-haulage freights to their closest terminal, which is an indicator of the drayage policy. In Fig. 8, the R–E–U instance, we observe that our approach gradually improves over the iterations in terms of long-haul cost and that it converges in terms of pre-haulage freights assigned to the closest terminals. In this instance, cost are more heavily fluctuating than assignments. In Fig. 9, the C–P–B instance, we observe that the costs significantly deteriorate with the iterations and that the fluctuations of the long-haul cost per freight (i.e., long-haul policy), and the closest terminal assigned (i.e., drayage policy) are significantly larger than in the R–E–U instance. Particularly, we see that large fluctuations in a small range of iterations (e.g., 35–40 and 90–95) are present. A possible explanation of this lies in the interaction between the long-haul and drayage policy in our approach. Adjusting the long-haul policy a little bit might have a huge impact on the drayage policy. For example, a small increase in the assignment cost to a Terminal at one iteration (i.e., VFA being updated at each iteration) may result in most freights not being assigned to that terminal in the following one. We might need a mechanism to actually avoid this continuous switching of departure terminals and in this way reduce the drastic fluctuations in the resulting policy. For example, at each iteration m, the initial weights of the basis functions in the ADP approach for each terminal can be defined considering the final ADP weights of all other terminals during the previous iteration $m-1$.

5.5 Discussion

The results of our numerical experiments raised a series of discussion points about the integrated scheduling of drayage operations and long-haul transport. In this section, we elaborate on these points, distinguishing between experimental limitations, modeling limitations, and guidelines for further research.

5.5.1 Experimental limitations

We observed that our integrated approach is capable of achieving significant savings even when used in “sequential” mode (Phase I and II of the experiments). In instances with high drayage-costs, where we did not achieve significant savings in 3 out of the 8 instances tested, the influence of the initial probability distribution is likely the cause for under-performance. The initial probability distribution of the long-haul made the ADP algorithm learn a VFA not accurately representing the expected (i.e., downstream) costs in the long-haul and therefore resulting in wrong decisions for the scheduling of both parts. In addition, we observed larger percentage savings in the total costs of instances with end-haulage dominance. In these instances, our approach significantly reduced the long-haul costs, percentage wise and compared to the non-integrated benchmark, but significantly increased the drayage costs. One might argue that the scale, or weight, of the costs for the two different parts may influence the conclusion, and hence for the possible implementation in practice by an LSP, additional performance measurements are needed.

5.5.2 Modeling limitations

In our experiments, we observed that our model and solution approach performed best in instances with clustered drayage locations and with pre-haulage dominance. This may be a result of our method of integration, where the drayage scheduling decisions have an explicit (i.e., direct) link to the long-haul performance but long-haul decisions only have an implicit link to the drayage. Although we did not calibrate the implicit link (i.e., probability distributions for the long-haul through the function $\varGamma$), and this might be the reason of the under-performance in instances with other configurations, it is possible that our approach requires an explicit link for instances with random drayage locations and end-haulage dominance.

5.5.3 Further research

The first guideline for further research is to improve the iterative integrated approach. This involves three challenges: (1) how to make the computational burden smaller, (2) how to improve the function $\varGamma$ in Step (A) such that it avoids the heavy fluctuations in drayage policy, and (3) how to decide whether to continue to the next iteration in Step (D) analyzing the convergence of probabilities, costs, and policy. The second guideline is to explore other characteristics that may have an influence on the cost saving opportunities of our integrated approach. In our experiments, the ratio between pre-haulage or end-haulage jobs had a larger influence than the location of the requests or the destination of the long-haul. Perhaps other time-related characteristics, which we assumed fixed in our study, have an important influence on the savings that our integrated scheduling approach can achieve.

6 Conclusions

In this paper, we designed a simulation-based approach to integrate a MILP model for scheduling drayage operations and an MDP model for scheduling long-haul transport. With our approach, we studied the relation between scheduling decisions, timing, and network-wide performance of drayage and long-haul operations in synchromodal transport considering stochastic demand and multiple freight characteristics. In addition, we tested how our integrated scheduling approach performed compared to a benchmark heuristic over various network configurations and cost structures. Furthermore, we analyzed the impact of our approach on the individual scheduling policies of drayage operations and long-haul transport, and how the performance of our approach can iteratively be improved.

In extensive numerical experiments, we showed that our integrated approach results in significant savings compared to a non-integrated approach, ranging between 4 and 24% in instances with a majority of pre-haulage freights. However, we also showed that under certain network characteristics, such as a majority of end-haulage freights, our approach performs approximately the same as non-integrated heuristics. These experiments helped us identify two areas for further research. First, the calculation of the long-haul probabilities for the ADP algorithm must be improved, such that the right emphasis is given on the drayage operations. Second, time and cost characteristics of the freight demand must be studied in more detail to provide insight into the applicability of our method in practice. We observed that the savings of our approach had significant implications to either the drayage operations or the long-haul transport. Studying these implications further will achieve improvements for the practice of synchromodal transport through the integrated scheduling of drayage operations and long-haul transport.

Appendices

Appendix 1: Notation

In this appendix we present all mathematical notation used throughout the paper. In all of the notation, a superscript ‘D’ at the end indicates that the parameter or variable is related to drayage scheduling, while the superscript ‘L’ at the end indicates a relation to long-haul scheduling. We divide the notation into four categories: (1) sets, (2) probabilities, (3) parameters and variables for the long-haul part of the network, and (4) parameters and variables for the drayage part of the network (Tables 8, 9, 10 and 11).

Table 8 Sets and their elements/indices ($^*$indicates miscellaneous elements)

Table 9 Probabilities

Table 10 Parameters and variables of the MDP model for the long-haul transport

Table 11 Parameters and variables of the MILP model for the drayage operations

Appendix 2: Cost setup

The drayage and long-haul cost setups are shown in Tables 12 and 13, respectively. The drayage fixed cost $C_k^\text {FD}$ and variable cost $C_{k,i,j}^\text {VD}$ are the same for all vehicles $k\in {\mathscr {K}}$. The drayage variable costs $C_{k,i,j}^\text {VD}$ are multiplied by the traveling time $T_{i,j}^\text {TD}$. There are two long-haul variable costs $C_{i,j,d}^\text {VL}$ per terminal $i\in {\mathscr {H}}$: the cost for using the high-capacity service (i.e., barge or train) to terminal j and the cost for hiring an additional truck directly to destination d. Furthermore, there are three groups ${\mathscr {G}}$, with a group setup cost of $C_{i,j,{\mathscr {G}}}^\text {GL}$ only for the high-capacity services (i, j). This group setup costs captures the fixed cost component, hence we use $C_{i,j}^\text {FL}=0$.

Table 12 Drayage cost setups

Table 13 Long-haul costs per terminal, for all instances

Appendix 3: Initial long-haul probability distributions

In this appendix, we show the “even” and “cost-oriented” initial probability distributions for the long-haul terminals. network-wide, new drayage freights arrive to the system according to a right-truncated Poisson distribution with mean 20 and truncated at 25. In both, the even and cost-oriented initial distributions, we use as initial arrival distribution for terminals a right-truncated Poisson distribution with mean $\frac{20}{3}$ and truncated at 25. The network-wide destination distribution for pre-haulage freights, for the unbalanced and balanced instances, is shown in Table 14. For the even initial probability distribution $p^\text {DL}_{d,j,t}$ per terminal j per time t, we use the same values of the unbalanced and balanced network-wide distributions, depending on the instance naturally. For the cost-oriented initial probability distribution, in contrast to the even, we use the distribution shown in Table 14, for all instances.

Table 14 Long-haul destination probabilities

Appendix 4: Calibration experiments

In this appendix, we study the relation between the tunable parameters of our integrated scheduling approach and the total costs. We do not strive to find the best setting possible for each parameter, but strive to study the sensitivity of the total costs for each parameter, under different network characteristics. Before presenting the results, we describe the tunable parameter values tested.

We test a total of 72 combinations of tunable parameters as follows. For the initial probability distributions for each terminal, we test two possible settings: an even distribution, where the number of freights are split evenly among the terminals and all other characteristics are the same as the network-wide distributions, and a “cost-oriented” distribution, which differs from the even distribution only in the distribution of destinations per terminal. We describe this distribution as cost-oriented because destinations closer to the end of the long-haul service of a terminal, i.e., cheapest destination to visit from a terminal, obtain 80% of the probability mass. These distributions can be found in “Appendix 3”. For the ADP time horizon, we use two settings: equal time horizon (i.e., equal to the problem horizon of 5 days), or a larger time horizon of 7 days. For the initial weights of the VFA in ADP, we use a “slope-oriented” setting, where all weights are equal to 1, or an “intercept-oriented” setting, where all weights are equal to 0 except for the constant, which is equal to $1.2\,\times \,10^4$. For the step size $\lambda$ used by ADP in the non-stationary least square updating method, we test 9 values, equally distributed between 0.1 and 0.9. These tunable parameters, and the notation we use to refer to them in the results, are shown in Table 15. The ADP algorithm runs for a total of 500 internal-iterations and the matheuristic runs for a single internal-iteration with the additional operator described in Sect. 4. Note that this iterations are from the heuristic themselves, and not with the iterations M of the iterative integration method.

Table 15 Tunable parameters and their values

For each instance combination in Table 1, we test the 72 combinations of tunable parameters using the sequential integration method. First, we run the ADP algorithm with a given set of parameters, and learn the weights of the basis functions. Second, we run a simulation of 100 replications of the time horizon and include the so-called “end effects”, which are costs for the long-haul freights that were postponed at each terminal at the end of the horizon, when running each replication of a given policy. Each step uses common random numbers across parameter combinations in order for performance differences to arise from the difference in tunable parameters. We measure three performance indicators: drayage costs, long-haul costs, and total costs (i.e., the sum of the last two). In the following, we summarize the results of these $8\,\times \,72$ experiments. We refer to an instance using three letters, e.g., R–P–U stands for an instance with random drayage locations, pre-haulage dominant drayage types, and unbalanced long-haul destinations.

Fig. 10

Total costs, per instance, for the best ADP settings of the even and cost-oriented probability distributions among terminals

In Fig. 10, we show the differences between the initial probability distributions for ADP. For each instance, we show the best settings of the remaining three tunable parameters, i.e., the combination of parameters that achieves the lowest total costs. Two important observations arise. First, the differences between the two initial probability distributions is not large. However, the even distribution is slightly better in 5 out of the 8 instances. It seems that, given that all other ADP parameters are tuned, the initial distribution does not play a significant role in achieving the lowest total costs. We come back to this issue in Sect. 5.5. Second, total costs are significantly larger, on average 49%, in instances with pre-haulage dominant drayage types than in end-haulage drayage types. Naturally, pre-haulage dominant instances have larger long-haul costs than the end-haulage dominant instances because of the longer paths of freights (see Fig. 1). To study the relation between tunable parameters of ADP for the long-haul cost and for the drayage cost, we look at three individual instances with the even initial distribution: Instance R–P–U, C–P–U, and C–E–U. We look into the relation of all other tunable parameters to the drayage, long-haul, and total costs in Figs. 11, 12, 13, 14, 15 and 16.

For Instance R–P–U, we observe in Fig. 11 that having an equal ADP time horizon (T5) and constant initial weights (W1) are the best settings to achieve lower total costs. In Fig. 12, we observe that T5W1 obtain the lowest total costs in R–P–U by achieving significantly lower long-haul costs than the other settings without increasing the drayage costs. In a similar way, settings T5W1 achieve the lowest total costs for Instance C–P–U, as shown in Fig. 13. However, in contrast to R–P–U, we observe that T5W1 obtains the lowest total costs by increasing drayage costs compared to other settings. This behavior is to be expected, considering that this instance has clustered drayage locations around Terminal 3, while freights to some long-haul destinations are transported cheaper in other terminals. In other words, drayage operations have to incur additional travel costs to bring freight to other terminals in order to have lower costs for the long-haul of some destinations.

For Instance C–E–U, in Fig. 15, we observe that differences in total cost among different tunable parameters are not as large as in the previous instances (around 1000 rather than 5000) and that the best settings are those with equal time horizon (T5) and slope-oriented initial weights (W2). In Fig. 16, we observe that the savings in total costs come again from the long-haul costs without significantly increasing drayage costs. Furthermore, we observe that long-haul costs are lower than drayage (which is to be expected due to more end-haulage requests), explaining the smaller total costs differences across settings than in the previous instances. In the remaining instances, we observe similar behavior with respect to long-haul and drayage cost relation and best settings to achieve the lowest total costs. For all instances, having an equal ADP time horizon (T5) was the best setting. For instances with pre-haulage dominant drayage types, the constant initial weights (W1) and a step-size of 0.4 were the best settings; while for instances with end-haulage dominant types, the slope-oriented initial weights (W2) and a step-size of 0.1 were the best settings.

Fig. 11

Total Costs R–P–U

Fig. 12

Individual Costs R–P–U

Fig. 13

Total Costs C–P–U

Fig. 14

Individual Costs C–P–U

Fig. 15

Total Costs C–E–U

Fig. 16

Individual Costs C–E–U

Appendix 5: Results evaluation experiments for the end-haulage dominant (E) instances

In this appendix, we show the results of the evaluation phase of experiments for the end-haulage dominant instances (Fig. 17; Table 16).

Fig. 17

Average total (drayage+long-haul) costs for the end-haulage dominant (E) instances

Table 16 Performance of different scheduling methods for the end-haulage dominant (E) instances