Ant open-sources LingBot-Touch to give robot hands a universal sense of touch

LingBot-Touch is an open-source tactile foundation model from Ant Group that converts raw data from diverse robotic fingertip sensors into universal tactile tokens. It effectively acts as a plug-and-play translator, enabling embodied manipulation models to understand physical contact without hardware-specific retraining.

What it does

While cameras guide a robotic arm toward an object, vision often fails the moment fingers make contact due to occlusion. LingBot-Touch steps in to handle contact dynamics, detecting slippage, force vectors, and resistance during delicate manipulation tasks like unscrewing bottle caps, zipping bags, or inserting keys into locks.

In real-world benchmarks across 26 manipulation tasks, attaching the 5.32-million-parameter LingBot-Touch-Tiny to standard policy models like π0.5, T-Rex, and LingBot-VLA increased overall parameter counts by only around 0.15%, while boosting average task success rates by 21 to 24 percentage points. For example, in a bag-zipping task, success jumped from 39% with pure vision to 59% with LingBot-Touch. It also enabled contact-dependent behaviors such as feeling the pop of a straw penetrating a cup seal and blind sorting Mahjong tiles by touch alone.

Crucially, the representations generalize to unseen hardware. When evaluated zero-shot on an XHand equipped with a magnetic tactile sensor absent from the training set, keeping LingBot-Touch frozen still raised the manipulation policy's average success rate from 48% to 74.17%.

How it works and specifications

Tactile sensors function under completely different physical principles, spanning vision-based elastomer deformation, magnetic field changes, and piezoresistive resistance shifts. Directly feeding these sparse, noisy raw signals into action policies often degrades performance. LingBot-Touch solves this by giving each sensor modality a lightweight front-end encoder, then mapping all contact points into a unified 5D finger coordinate space via rotary position embeddings.

Instead of reconstructing noisy raw numbers where background zeros dominate, LingBot-Touch uses a Tactile-JEPA architecture. During training, 90% of contact points are masked, forcing the network to predict contact representations directly in feature space. This brings blind localization contact error down to 0.12 mm, compared to 2.19 mm in existing baselines like FTP-1.

The model is pre-trained on 4,983 hours (approximately 1.33 billion frames) of real tactile data. Two-thirds of this dataset was collected via humans wearing tactile data gloves during daily chores, with the remaining third sourced from teleoperated bimanual robots. Scaling behavior is evident: while the Tiny variant saturates around 4,000 hours, the larger Small model continues to improve through 5,000 hours.

Getting started

The technical report, codebase, and model weights are fully open-sourced. You can find the source repository on GitHub under Robbyant/lingbot-touch, while pre-trained checkpoints are hosted on Hugging Face and ModelScope. The repository includes integration examples showing how to route raw tactile arrays into the tokenizers and feed the resulting embeddings directly into standard policy architectures.

Why it matters

If you build robotic manipulation pipelines or work with dexterous hands, LingBot-Touch is worth benchmarking immediately. Historically, changing a tactile sensor or switching hand hardware meant re-collecting contact datasets and retraining end-to-end controllers from scratch. LingBot-Touch decouples tactile intelligence from physical hardware into a reusable abstraction layer.

Compared to earlier attempts like FTP-1 that relied on custom hardware adapters, LingBot-Touch demonstrates that touch experience gathered from human gloves transfers zero-shot to completely unseen robotic hands. We think this opens an affordable path to scale physical AI data without relying entirely on expensive teleoperation rigs.