In context
In July 2024, deep learning-based stereo matching was widely used in robotics navigation, autonomous driving, and virtual reality, yet convolutional neural networks (CNNs) struggled with limited receptive fields and weight sharing, leading to poor accuracy in challenging image regions. This paper from Hunan University addressed that gap by combining local and global feature extraction.
What was reported
The study introduced a binocular stereo matching algorithm with a three-branch hybrid feature extractor, integrating a CNN branch (ResNet-50), a Swin Transformer branch (Swin-B), and a fusion branch in parallel. This design preserves CNN's local feature strength and Swin Transformer's global modeling, while reducing data dependency and accelerating convergence.
The fusion branch uses multi-stage cross-branch global-local information adapters, combining channel, spatial, and category attention to propagate features across stages, effectively screening strong correlation features for weak-textured and detailed regions.
Tested on SceneFlow, KITTI 2012, and KITTI 2015 datasets, the method achieved an end-point error of 0.652 pixels on SceneFlow and a 0.79% error rate (disparity >5 pixels) in non-occluded regions on KITTI 2012, demonstrating excellent accuracy.
Why it mattered
This work highlighted a practical path to improve 3D perception for industrial robots and automated systems, where robust depth estimation in low-texture environments is critical for navigation and manipulation. The hybrid architecture offered a balanced solution between computational efficiency and accuracy, potentially influencing future stereo vision designs.
The parallel branch setting effectively preserves the local feature expression ability of CNN and the global feature expression ability of Swin Transformer framework.
Source: 《机器人》期刊 (robot.sia.cn) · Published 2024-07-20 · “基于三分支混合特征提取的双目立体匹配算法”