Article “Local Motion Simulation using Deep Reinforcement Learning” authored by Dong Xu, Xiao Huang, Zhenlong Li, Xiang Li is accepted for publication in Transactions in GIS.
Abstract: Traditional local motion simulation largely focuses on avoiding the collision in the next frame. However, due to the lack of forward-looking, the global trajectory of agents usually seems not reasonable. As a method of optimizing the overall reward, Deep Reinforcement Learning (DRL) can better correct the problems existed in the traditional local motion simulation method. In this paper, we propose a local motion simulation method integrating Optimal Reciprocal Collision Avoidance (ORCA) and DRL, referred to as ORCA-DRL. The main idea of ORCA-DRL is to perform local collision avoidance detection via ORCA and smooth the trajectory at the same time via DRL. We use Deep Neural Network (DNN) as the state-to-action mapping function, where the state information is detected by virtual visual sensors and the action space includes two continuous spaces: speed and direction. To improve data utilization and speed up the training process, we use the Proximal Policy Optimization (PPO-Clip) based on the Actor-Critic (AC) framework to update the DNN parameters. Three scenes (Circle, Hallway, and Crossing) are designed to evaluate the performance of ORCA-DRL. The results reveal that, compared with the ORCA, our proposed ORCA-DRL method can 1) reduce the total number of frames, leading to less time for agents to reach their destination, and 2) effectively avoid local optimum, evidenced by smoothened global trajectory.
