Effective Cross-Region Courier-Displacement for Instant Delivery via Reinforcement Learning

Abstract

With the rapid development of mobile phones and the Internet of Things, instant delivery services (e.g., UberEats and MeiTuan) have become a popular choice for people to order foods, fruits, and other groceries online, especially after the impact of COVID-19. In instant delivery services, it is important to dispatch massive orders to limited couriers, especially in rush hours. To meet this need, an efficient courier displacement mechanism not only can balance the demand (picking up orders) and supply (couriers’ capacity) but also improve the efficiency of order delivery by reducing idle displacing time. Existing studies on fleet management of rider-sharing or bike rebalancing cannot apply to courier displacement problems in instant delivery due to unique practical factors of instant delivery including region difference and strict delivery time constraints. In this work, we propose an efficient cross-region courier displacement method Courier Displacement Reinforcement Learning (short for CDRL), based on multi-agent actor-critic, considering the dynamic demand and supply at the region level and strict time constraints. Specifically, the multi-agent actor-critic reinforcement learning-based courier displacement framework utilizes a policy network to generate displacement decisions considering multiple practical factors and designs a value network to evaluate decisions of the policy network. One month of real-world order records data-set of Shanghai collecting from Eleme (i.e., one of the biggest instant delivery services in China) are utilized in the evaluation and the results show that our method offering up to 36% increase in courier displacement performance and reduce idle ride time by 17%.