In context
As service robots move from structured factory lines to open environments like homes and warehouses, long-horizon task planning—where a robot must execute a sequence of sub-tasks over extended periods—has become a critical challenge. Traditional reinforcement learning and skill-based hierarchical methods often struggle with sparse rewards, low sample efficiency, and limited perception input, especially when tasks are described in natural language.
What was reported
Researchers from Xi'an Jiaotong University proposed MMDPP (Multi-modal Diffusion Policy for Long-horizon Task Planning), a method that integrates a large vision-language model (Qwen2.5-VL) to parse natural language instructions into structured task elements. These elements are embedded and combined with low-dimensional state vectors and RGB image features via a multi-modal fusion module that uses a bottleneck attention mechanism to reduce gradient conflicts.
The fused representation conditions a denoising diffusion model, which generates action sequences end-to-end from language input to action prediction. The approach was evaluated in three settings: a self-constructed MuJoCo-Kitchen-Image environment, where it significantly outperformed baselines; the Robosuite-Kitchen benchmark, where it surpassed the SiMPL method by 2.4% in success rate; and a UR5 physical robot performing table-top rearrangement tasks, achieving an 80% success rate.
The method addresses limitations of prior skill-based approaches that rely on precise state inputs and fixed-duration skill templates, offering better adaptability to task changes and visual-driven deployment.
Why it mattered
This work demonstrates a viable paradigm for using diffusion models in long-horizon robot planning, potentially enabling more robust and flexible automation in unstructured environments where tasks are specified by natural language and require visual perception.
“The adaptability of action policy learning to task changes in long-horizon tasks is significantly enhanced by the proposed method, providing an effective paradigm for long-horizon robot planning based on diffusion modeling.”
Source: 《机器人》期刊 (robot.sia.cn) · Published 2025-08-19 · “基于多模态扩散策略的长时任务规划”