In context
In late 2023, mobile robot navigation in dynamic, complex environments remained a challenge for traditional map-based frameworks, which rely on SLAM and separate planning modules. These systems often struggle with moving obstacles, sensor limitations, and the overhead of maintaining accurate maps. Deep reinforcement learning (DRL) offered a promising alternative, but existing models typically assumed fully observable states, limiting their real-world applicability.
What was reported
Researchers from Northeastern University, China, introduced TS-TD3 (time series twin delayed deep deterministic policy gradient), a DRL method for end-to-end mapless navigation. The approach formulates navigation as a partially observable Markov decision process (POMDP), acknowledging that sensors like 2D LiDAR provide only partial environmental information. To handle hidden states, the model integrates historical information processed by long short-term memory (LSTM) components, which capture temporal correlations in navigation actions and help infer dynamic obstacle motion.
The actor-critic architecture uses a twin delayed DDPG framework with clipped double Q-learning to reduce overestimation. A novel historical information baseline is introduced into the deterministic policy gradient, guiding exploration toward hidden state information. Additionally, an expert experience network provides initial action guidance during early training, accelerating convergence and ensuring natural, stable trajectories.
In simulations, TS-TD3 outperformed mainstream DRL methods, producing smooth, continuous paths and effectively handling intersections of multiple dynamic obstacles. In real-world tests, the model was deployed directly in unknown environments without adjustment, demonstrating strong generalization and navigation performance.
Why it mattered
This work advances DRL-based navigation toward practical deployment in dynamic industrial settings, where robots must operate safely alongside humans and moving machinery. By addressing partial observability and temporal action correlation, TS-TD3 reduces reliance on pre-built maps and complex planning pipelines, potentially lowering system costs and improving responsiveness in unstructured environments.
“The motion trajectory obtained by the proposed method is natural, stable and continuous in the simulation experiment, the intersection of multiple dynamic obstacles can be dealt with, and the overall navigation performance is optimal.”
Source: 《机器人》期刊 (robot.sia.cn) · Published 2023-11-10 · “基于TS-TD3的动态环境端到端无地图导航方法”