For the past three years, the standard way to teach a robot a new skill has been teleoperation: a person controls the robot remotely and every movement is recorded as a demonstration. It produces exactly the data a robot needs, but each operator can drive only one robot at a time, and hours of data are expensive.
This week brought three announcements that point to what comes next. None of them replaces teleoperation. Together they show how human video, teleoperation and simulation are starting to work as one pipeline.
What happened this week
Agile Robots opened a Robot Academy. On 8 October, Agile Robots CEO Zhaopeng Chen presented the company's Robot Academy, a physical facility in Munich. Cameras record workers performing industrial tasks, hand-tracking software extracts the motion, and the motion is mapped onto robot joints and practised in simulation before it reaches a real robot. Teleoperation stays in the mix, alongside video, direct human demonstrations, fleet data from deployed robots and simulation.
Google released Gemini 4 Argon for Robotics. Announced on 13 October, the model can watch a real robot teleoperation episode and build a digital twin of the scene, or take an egocentric (first-person) human video of a manipulation task and retarget the hand motion onto a simulated humanoid. In one of Google's examples, Argon generated more than 4,500 varied training episodes in simulation, and adding that data raised real-world success on the task from 56% to 75%.
KITECH built a full-body human motion platform. On 14 October, the Korea Institute of Industrial Technology described its Human-to-Humanoid Intelligence Platform (HHIP). It records people with optical motion capture, force plates, glove sensors for finger movement and EMG sensors for muscle activity, all on a single time axis, to train humanoid control models.
What each data source is good at
The three methods are complementary, not competing.
- Human video brings scale and variety. People can be filmed doing real work in real places, without a robot in the room. What video lacks is the robot's own actions, the forces involved and what happens when fingers hide behind an object.
- Teleoperation brings precision. Every recording pairs what the robot saw with the exact commands it executed, which is what imitation learning needs to get contact-rich tasks right. Data gloves keep tracking the fingers when cameras lose them.
- Simulation multiplies both. A single teleoperated episode or human video can become thousands of varied training episodes in a digital twin, as Google's Argon example shows.
The pattern across Agile Robots, Google and KITECH is the same: capture human movement as precisely as possible, use teleoperation where precision matters, and let simulation do the multiplying.
Why precise hand capture matters more, not less
Video-first approaches still depend on knowing exactly what the human hand did. Agile Robots extracts hand motion with tracking software; KITECH adds glove sensors and force plates; Google retargets hand motion from video onto a robot hand. The cleaner that hand data, the better everything downstream.
This is where data gloves fit. MANUS gloves track each finger with electromagnetic sensors, so they keep working when fingers close around an object and disappear from a camera's view. The same gloves can record human demonstrations on their own or drive a robot hand live through NVIDIA Isaac Teleop. Since July, NVIDIA and Hugging Face have also brought Isaac Teleop and the GR00T 1.7 model into the open-source LeRobot library, so teams can go from recorded demonstrations to a trained policy with open tools.
Reviewing data and digital twins in 3D
As labs collect more episodes and build more digital twins, a new step appears: people need to look at the data and decide. Is this demonstration clean enough to train on? Does the simulated scene match the real cell? Which tasks need more data?
These are 3D questions, and they are hard to answer on a flat monitor. A holographic display lets a team see the scene with real depth, without putting on VR headsets:
- For one engineer or researcher, a desktop holographic display such as ILLUMETRY IO shows a 3D scene floating above a 24-inch screen, with a 6DoF stylus to pick up and inspect objects close up.
- For a team making decisions, a hologram table such as ILLUMETRY EOS presents a 65-inch 3D scene that a group can stand around: a lab layout, a robot cell, a digital twin of a production line or campus, during design reviews, project briefings and funding presentations.
ILLUMETRY works with Unity and Unreal Engine content, and Aedu AI can prepare or port content from existing 3D models.
What this means for a Physical AI lab
A Physical AI lab built today should not bet on one data source. In practice that means:
- Precise human capture: data gloves and tracking that work with or without a robot.
- Teleoperation stations: for the contact-rich tasks where video falls short.
- Simulation: NVIDIA Isaac Sim and Isaac Lab to turn a few real demonstrations into many.
- A way to review the results: from a workstation to a holographic display for the team.
You do not need all of it on day one. Most labs start with gloves and simulation and add robots later; we set out the phases in How to Build a Physical AI Teaching Lab.
Aedu AI designs, supplies and installs Physical AI Labs and supplies holographic displays in Singapore and Southeast Asia. If you are planning a lab or a 3D review space, .
Key points
- Agile Robots (8 October), Google (13 October) and KITECH (14 October) each showed new ways to turn human movement into robot training data.
- Human video brings scale, teleoperation brings precision, and simulation multiplies both. Leading teams combine all three.
- Google reports that simulated episodes generated by Gemini 4 Argon raised one real-world task's success rate from 56% to 75%.
- Precise hand capture with data gloves matters more as video and simulation grow, because every downstream step depends on it.
- Holographic displays such as ILLUMETRY IO and EOS let engineers and decision-makers review 3D data and digital twins without headsets.
FAQ
Will human video replace robot teleoperation?
Not on current evidence. Agile Robots and Google both use video and simulation alongside teleoperation, not instead of it. Video gives scale; teleoperation gives the exact robot actions and contact that precise tasks need.
What is egocentric video in robot learning?
Video recorded from the performer's own point of view, usually with a head-mounted camera, so the footage shows the hands and objects the way a robot's cameras would see them.
What is a Real2Sim digital twin?
A simulated copy of a real scene, built from camera footage or recorded robot episodes, in which a robot can practise many variations of a task before trying it in the real world.
How do holographic displays help with robot data?
They show 3D scenes, robot cells and digital twins with real depth, so an engineer can inspect a scene up close on a desktop display and a team can review it together around a hologram table, without VR headsets.
Who supplies Physical AI labs and holographic displays in Singapore?
Aedu AI, based in Singapore, designs and installs Physical AI Labs for robot teleoperation and demonstration data collection, and supplies ILLUMETRY holographic displays, for universities, research institutes and companies across Singapore and Southeast Asia.
Sources
- Tech Times, "Agile Robots Unveils Robot Academy: Worker Video Generates Training Data Without Teleoperation" (13 October 2026)
- Google for Developers Blog, "Gemini 4 Argon for Robotics" (13 October 2026)
- Seoul Economic Daily, "Korean Institute Builds Platform to Turn Human Motion Into Humanoid AI Data" (14 October 2026)
- Hugging Face, "NVIDIA Isaac Teleop and GR00T 1.7 Open VLA Model Available in LeRobot" (July 2026)
- Cover image: egocentric human demonstrations with MANUS data gloves, courtesy of MANUS.
- Physical AI
- Robot Teleoperation
- Robot Learning
- Digital Twin
- Holographic Display
