Aedu AI
All posts

MANUS Gloves6 min read

How MANUS Gloves Are Used to Train Robot Hands in 2026

Robot hands have caught up; the data to train them has not. Here is how teams now use MANUS data gloves to record human demonstrations, drive dexterous robot hands in NVIDIA Isaac Teleop, and feel contact through haptic fingertips.

Aedu AI Team
How MANUS Gloves Are Used to Train Robot Hands in 2026

Dexterous robot hands have improved quickly. Five-finger hands from Inspire, Allegro, Shadow Robot, Xynova and others are now reliable enough for research labs and humanoid companies to buy off the shelf. What holds them back is no longer the hardware. It is the data needed to teach them.

At ICRA 2026, MANUS summed up what it saw across the exhibition floor: the hard problem has become "generating the large-scale, high-quality data needed to train dexterous manipulation systems", because "imprecise motion capture propagates through the learning pipeline and results in lower-quality policies." In other words, a robot hand can only learn to grasp as well as the human demonstrations it is shown.

This article explains how teams are using MANUS data gloves to close that gap: what the gloves capture, how that motion reaches a robot hand, and how leading labs are turning it into training data.

Why robot hands learn from human hands

Most of today's dexterous manipulation policies are trained by imitation learning. A person performs a task (opening a drawer, picking up a cup, turning a valve) while their hand motion drives a robot hand. Each attempt is recorded as an episode: the robot's joint positions, camera images and, increasingly, touch signals. Thousands of these episodes become the dataset a policy learns from.

Two things decide whether that works:

  • Tracking quality. If the captured finger motion jitters, drifts or loses a finger behind an object, the robot repeats those errors and the policy learns them.
  • Throughput. Teams need to record many clean episodes quickly, across many objects and tasks, without recalibrating constantly.

Camera-based hand tracking struggles exactly where manipulation happens: when fingers wrap around an object and disappear from view.

What a MANUS glove actually measures

MANUS Metagloves Pro use electromagnetic field (EMF) sensors at the fingertips instead of cameras. Because the sensors read a magnetic field rather than an image, there is no occlusion when the hand closes around an object, and no drift over a long session.

MANUS provides the hand data at three levels, so it can be mapped onto very different robot hands:

  • Fingertip sensor positions, five 6-DoF sensors per hand
  • A calibrated 25-node skeletal model of the full hand
  • Joint angles for each finger

The gloves sample at 120 Hz and run wired over USB-C or wirelessly. The Metagloves Pro Haptic version adds a linear resonant actuator on each fingertip, which matters for the next part.

From glove to robot: retargeting in NVIDIA Isaac Teleop

A human hand and a robot hand rarely share the same proportions, joint limits or number of joints. Retargeting converts human finger motion into commands the robot hand can execute.

MANUS is the officially supported data glove in NVIDIA Isaac Teleop and Isaac Lab. The MANUS plugin captures the hand skeleton from the MANUS SDK and injects it into Isaac Teleop's standard hand-tracking layer. Any retargeter built for that layer can therefore use MANUS data without extra work. A retargeter takes the glove data and, using the robot hand's URDF model, solves for the robot's joint angles frame by frame. NVIDIA's documentation, for example, includes a retargeter that maps a MANUS glove onto the Sharpa hand through optimisation-based inverse kinematics.

The practical benefit for a lab is that the same glove and the same pipeline work in simulation and on hardware. A team can start with simulated hands in Isaac Lab, then connect a real dexterous hand later without changing how operators work.

Feeling the grasp: haptic feedback in the loop

Teleoperating a robot hand without touch is like typing with numb fingers. Operators squeeze too hard, or not hard enough, and those mistakes end up in the dataset.

In September 2026, MANUS announced that Isaac Teleop and Isaac Lab now support per-finger haptic feedback with Metagloves Pro Haptic. Contact forces are measured on each finger of the simulated robot hand as it touches the grasped object, and that signal is sent back to the glove's five fingertip actuators. Operators feel when the robot makes contact and when an object starts to slip. MANUS notes this produces "trajectories that already reflect real contact dynamics", which is exactly what contact-rich tasks need.

Who is using it

The pattern is the same across very different organisations:

  • Peking University and the Beijing Academy of Artificial Intelligence built RoboBrain-Dex, a vision-language-action model for dexterous manipulation. They used MANUS gloves with VIVE trackers for wrist pose to record egocentric human data and to teleoperate a Unitree G1 humanoid with Inspire dexterous hands. Their model achieved the highest average success rate across six real-world dexterous tasks in their tests.
  • Seoul National University and RLWRLD created HRDexDB, a paired human-robot grasping dataset: 1,400 episodes across 100 objects and four embodiments, with synchronised hand trajectories, object poses, touch sensing and egocentric RGB-D video. MANUS gloves provided the finger tracking that let operators reproduce human grasps on the robots under identical conditions.
  • ABB Robotics and PSYONIC used MANUS gloves to capture natural hand motion as demonstration data for robot dexterity, with Haply providing wrist tracking and force feedback.
  • Robot hand makers at ICRA 2026, including AgiLink, Inspire, Allegro and Xynova, used MANUS gloves to map human hand motion onto their hands live on the show floor.

Three ways to start collecting data

You do not need a humanoid on day one. Most labs grow through three stages:

  1. Simulation first. One pair of MANUS gloves, an XR headset and a GPU workstation running Isaac Lab. Operators learn to teleoperate simulated hands, and the team builds its recording pipeline before buying a robot.
  2. One dexterous station. Add a collaborative arm with a dexterous hand, wrist tracking and two or three RGB-D cameras. Each demonstration is recorded as synchronised episodes for imitation learning. The haptic gloves pay off here.
  3. Scaled, bimanual collection. Two arms and two hands, or a humanoid upper body, with several stations recording in parallel and a data management workflow behind them.

Whatever the stage, the same rules hold: keep the glove calibration consistent between operators, synchronise every stream to one clock, and label episodes as you record them rather than afterwards.

How Aedu AI helps

Aedu AI designs and installs Physical AI Labs for robot teleoperation and demonstration data collection. We supply MANUS Metagloves Pro and Pro Haptic, set up NVIDIA Isaac Teleop and Isaac Lab, connect your robot through ROS 2, and train your team to record clean data from the first week.

See the Physical AI Lab configurations and MANUS Metagloves specifications, or get in touch to plan your lab.

Key points

  • Dexterous robot hands have matured; high-quality human demonstration data is now the bottleneck.
  • MANUS gloves track fingers with electromagnetic sensors, so there is no occlusion or drift when the hand closes on an object.
  • MANUS is the officially supported glove in NVIDIA Isaac Teleop and Isaac Lab, and the same pipeline works in simulation and on real robot hands.
  • Metagloves Pro Haptic return per-finger contact forces from the robot hand to the operator.
  • Teams from Peking University and BAAI, Seoul National University and RLWRLD, and ABB with PSYONIC use MANUS gloves to build dexterous manipulation datasets.

FAQ

Do I need a real robot to start?

No. Isaac Lab runs simulated robot hands, so a team can teleoperate and record in simulation first, then add a real hand with the same gloves.

Which robot hands can MANUS gloves control?

Any hand with a retargeter in the pipeline. Teams use them with Inspire, Allegro, Sharpa, PSYONIC, Xynova and other hands; a URDF model of the hand is the starting point.

Why not use camera-based hand tracking?

Cameras lose fingers that are hidden behind the object being grasped, which is exactly when manipulation data matters most. Electromagnetic tracking keeps tracking through those moments.

What does haptic feedback add?

Operators feel contact and slip, so they grip more naturally and the recorded demonstrations reflect realistic contact forces.

Sources

  • MANUS
  • Physical AI
  • Robot Teleoperation
  • Data Collection
  • NVIDIA Isaac
Keep readingAll posts →

Work with us

Tell us what you want to train, simulate, or visualize, and we will follow up by email.

or email support@aedu.ai