The ability to remotely control robots in real-time, also known as teleoperation, could be useful for a broad range of real-world applications. In recent years, some engineers have been trying to develop teleoperation systems that allow users to guide the actions of humanoid robots, which have a body structure resembling that of humans, getting the robots to precisely imitate their whole-body movements.
Researchers at Stanford University and Simon Fraser University recently introduced TWIST (teleoperated whole-body imitation system), a new system that allows humanoid robots to closely imitate the whole-body motions of human users in real-time, successfully completing various real-world tasks.
This system, outlined in a paper posted to the preprint server arXiv, leverages motion capture (MoCap) data, along with reinforcement learning and imitation learning approaches.
“We want humanoids to have the same level of whole-body dexterity as humans. Imagine a messy kitchen,” Yanjie Ze, first author of the paper, told Tech Xplore. “Humans can hold things with two hands and use their feet to move obstacles, such as a basket on the ground; humans can also open the door using the sides of their bodies or their elbows. We want to make humanoids achieve the same by imitating humans directly.”
TWIST, the system developed by Ze, Karen Liu and their colleagues, utilizes data captured by MoCap devices, technologies that precisely track the body movements of humans. Compared to many teleoperation systems introduced in the past, TWIST leverages joints across the entire bodies of humanoid robots to closely replicate human movements, while also ensuring that the motions of different limbs are coordinated.
“TWIST is a system that teleoperates humanoid robots using the whole-body movement of a person in real time,” explained Ze. “We accurately capture human motion and then use AI to map it to commands that humanoids may execute. Our system has much higher accuracy in whole-body control than prior work and enables diverse motions and skills that cannot be accomplished before.”
Ze and his colleagues evaluated their teleoperation system in a series of real-world experiments, using the humanoid robot G1, developed by Unitree Robotics. They found that their system successfully enabled the teleoperation of this robot and could also be applied to other humanoid robots, such as the T1 robot created by Booster Robotics.
“The most notable finding is that whole-body human motion data is powerful enough to control humanoids (including their legs, feet, waists, knees, elbows, etc.,),” said Ze. “Our system allows all these body parts to move as those of humans do, which enables robots to exhibit human-like, whole-body dexterity. The immediate application is to use humans to control humanoids to collect large-scale data to train robotic foundation models.”
In the future, the TWIST system could be improved further, for instance, by reducing its reliability on MoCap systems, which are not portable and thus restrict its applications. Eventually, it could be deployed in real-world settings, for instance to allow robots to complete manual tasks in hazardous working environments or automate industrial processes that require high levels of precision.
“We want humanoid robots to be truly intelligent and capable in accomplishing real-world tasks,” added Ze. “Our next step will be to scale up data collection and then to enable robots to learn autonomous skills.”
More information:
Yanjie Ze et al, TWIST: Teleoperated Whole-Body Imitation System, arXiv (2025). DOI: 10.48550/arxiv.2505.02833
arXiv
© 2025 Science X Network
Citation:
Whole-body teleoperation system allows robots to perform coordinated tasks with human-like dexterity (2025, May 15)
retrieved 15 May 2025
from https://techxplore.com/news/2025-05-body-teleoperation-robots-tasks-human.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
