Google DeepMind's SL2T lets Pixel 11 users sign directly into apps

Dictating a quick text or firing off a voice search while walking down the street is second nature for hearing users. For roughly 70 million Deaf and hard of hearing people worldwide, interacting with a smartphone has meant being tethered to a tiny on-screen keyboard.

Now, Google DeepMind wants to bring the same conversational freedom to signers. The company just introduced SL2T, short for sign-language-to-text, a multilingual model designed to bridge the gap between visual physical languages and digital text.

The system is making its real-world debut on the new Pixel 11, powering sign-to-text dictation right inside Gboard and Live Transcribe at no extra cost. It kicks off with American Sign Language (ASL) into English, with more devices and languages planned down the road.

Skipping the camera feed for pure coordinates

Teaching a machine to read sign language is wildly different from standard speech transcription. Sign languages are entirely independent natural languages with their own complex grammars, facial expressions, and spatial rules. They are not simply spelled-out spoken words.

DeepMind trained SL2T on over 100,000 hours of video across more than 50 sign languages, with ASL accounting for roughly a quarter of the dataset. Instead of converting signs into intermediate glosses or literal word markers, the model maps bodily movement straight into fluid English text. On the FLEURS-ASL benchmark, SL2T posted a zero-shot score of 70 BLEURT, easily topping prior public benchmarks.

To sidestep massive privacy landmines, Google is not beaming live video from the camera to its cloud servers. Instead, an on-device vision model extracts geometric pose landmarks from the user hands, arms, torso, and face, immediately throwing away the raw video. Only the coordinate data travels upstream for translation.

The team also tuned the model for real-life messiness. That includes tuning it for the 10 percent of left-handed signers, handling one-handed signing when someone is holding their phone in the other hand, and suppressing hallucinations when a user is just moving naturally without actually signing.

Signing directly to apps and Gemini

In daily use, the feature treats signing just like voice dictation. Deaf users can position their phone and sign to fire off web searches, draft work documents, reply to chats, or prompt Gemini to execute multi-step tasks.

In the Live Transcribe app, users can now sign their side of an in-person conversation, which turns into clean English text on the screen instead of forcing them to type back and forth. DeepMind built the technology alongside Deaf Googlers and an external advisory committee of global Deaf organizations, ensuring the tool actually reflects community needs rather than academic assumptions.

Why it matters

We think this is one of the most practical and thoughtful deployments of multimodal AI in recent memory. For many Deaf individuals, ASL is their first language, meaning typing in English requires constant mental translation. Giving users the option to express themselves natively in the interface cuts down massive friction across everyday phone tasks.

From an engineering standpoint, Google's architectural bet looks sharp. Converting raw pixels to skeletal landmarks on-device simultaneously solves network bandwidth limits and neutralizes creepy room-recording privacy concerns. The obvious hurdle will be lighting and visual occlusion: if a noisy environment trips up landmark tracking, translation errors can snowball quickly.

For the wider tech ecosystem, this sets a high bar. Accessibility features are often relegated to niche settings, but integrating sign language translation directly into the default system keyboard treats it as a first-class input method. It puts clear pressure on Apple and Microsoft to treat sign language parity as an essential standard rather than a backburner research project.