Skip to content

XTac UMI · Wearable multimodal data collection

Give robot datasets
a sense of touch

Two-handed touch, vision and pose in sync
One demonstration, a training-ready dataset

XTac UMI G1 grippers with the Pico4 Ultra Enterprise headset
in a Backpack Kit or a Developer Kit

About XTac UMI G1 Choose an edition

Leader gripper

Leader: hand-held capture

Follower gripper

Follower: isomorphic end effector

Touch Ignites Physical Intelligence

8 image streams 4 tactile + 2 wrist + 2 headset

3 × 6-DoF poses both grippers + head

5ms sync positioning < 3mm

MCAP · LeRobotDataset v3 raw and training formats

Highlights

  • Visuotactile fusion: tri-color fingertip sensors capture contact location, grasp stability and slip
  • Close to the hand: the wearable leader handles grasping, insertion and thin-part picking
  • Precise sync: 5ms multi-device time sync, < 3mm positioning
  • Open formats: LeRobot / MCAP output, compatible with mainstream data ecosystems

More highlights

Two editions for different settings

Both editions use the same grippers and headset and produce the same data. The Backpack Kit is a turnkey unit whose collection software is not open source; the Developer Kit's collection software and gripper SDK are open source and free to customize.

  • The XTac UMI data collection backpack

    Backpack Kit

    XTac UMI Backpack

    • The backpack runs all collection; no computer needed
    • A tablet browser is the console; gripper buttons start and stop recording
    • One person wears it, suited to field work and long sessions
    • Upload datasets to ModelScope in one click, as MCAP or LeRobotDataset v3

    Turnkey, closed-source collection software; for large-scale collection teams and data factories

    Quickstart About the backpack

  • XTac UMI G1 visuotactile gripper

    Developer Kit

    XTac UMI G1 Developer Kit (open source)

    • Grippers and headset connect straight to your collection PC
    • Built on open-source LeRobot; recording produces a LeRobotDataset
    • Open-source collection software and gripper SDK, free to customize
    • Supports the follower gripper for executing and replaying motions on a robot

    For research and algorithm teams with their own training pipelines

    Quickstart Compare the two editions

The data you get

Where each of the eight streams sits on the hardware, and its dataset key

Sensor positions and dataset keys

Rerun live preview: four visuotactile streams, headset stereo, both wrist cameras and action curves

Live Rerun preview while recording (Developer Kit)

xense-lerobot-viewer 3D replay: gripper and headset tracks, wrist and headset views, four visuotactile streams

3D replay in xense-lerobot-viewer

Open dataset: TacVerse

The three TacVerse scene families: office, home and workbench

Visuotactile manipulation data collected with XTac UMI G1 grippers, released as LeRobotDataset v3 under CC BY-SA 4.0 and ready to download and train on.

  • 122 tasks across office, home and workbench scenes
  • 17,690 demonstrations: 370 hours, about 40 million frames
  • Two-handed touch: four tactile streams, two wrist views and gripper poses per frame

Hugging Face ModelScope

Developer Kit open-source repositories

The Developer Kit's collection repo, gripper SDK, tracker service and viewer are open source under Apache-2.0, see the list; the Backpack Kit's software is closed.