In context
By mid-2023, humanoid robots had advanced in mechanical design and control but lagged in emotional expression, a key barrier to natural human-robot interaction. Researchers sought methods to transfer human facial emotions to robots with high fidelity and smoothness, addressing the limitations of prior frame-by-frame approaches that ignored temporal dynamics and motor constraints.
What was reported
Researchers proposed RFEFormer, a robotic facial emotion transfer network combining a Transformer architecture with cubic B-spline smoothing constraints. The network comprises a facial deformation encoding subnetwork and an actuation sequence generation subnetwork. The encoding subnetwork uses intra-domain deformation attention and inter-domain cooperative attention within a Transformer encoder to capture spatial features across facial regions (head, eyebrows, eyes, mouth) at multiple granularities. The generation subnetwork employs a Transformer decoder for cross-attention between facial spatio-temporal sequences and historical motor actuation sequences, enabling multi-step prediction of future motor commands. A cubic B-spline smoothing constraint regularizes the predicted sequences to reduce mechanical jitter.
Experiments on a 47-motor humanoid robot (with 11 head motors) demonstrated a motor actuation deviation of 3.21%, facial deformation fidelity of 89.48%, and motor motion smoothness of 90.63%. The system achieved real-time facial emotion transfer at over 25 frames per second, outperforming related methods in fidelity and smoothness while maintaining real-time performance.
Why it mattered
This work addressed a critical gap in humanoid robotics: achieving emotionally expressive faces that are both realistic and mechanically feasible. By modeling temporal sequences and enforcing smoothness, RFEFormer advanced the state of the art in affective computing and human-robot interaction, offering a practical approach for integrating emotional intelligence into service and companion robots.
Compared with the related methods, the proposed RFEFormer network not only satisfies the real-time performance, but also improves the time sequence-based indexes such as fidelity and smoothness, which are more sensitive and concerned by human senses.
Source: 《机器人》期刊 (robot.sia.cn) · Published 2023-07-08 · “基于Transformer架构和B样条平滑约束的机器人面部情感迁移网络”