Nvidia launches AI platform to teach sign language

American Sign Language is the third most common language in the United States, trailing only English and Spanish. Yet when you look at today's booming AI landscape, ASL data is practically non-existent compared to text and audio corpora.

NVIDIA is teaming up with the American Society for Deaf Children (ASDC) and creative agency Hello Monday to change that with Signs, a new interactive web platform designed to teach ASL while quietly crowdsourcing a massive training dataset.

Mirroring movements with AI

The web tool is straightforward and accessible right now via your browser. A 3D avatar guides users through gestures, while an AI model tracks hand movements and finger positions through your webcam, giving real-time corrective feedback.

Users can already start practicing an initial set of 100 common signs. Hello Monday, who previously built the viral alphabet tool Fingerspelling.xyz with the ASDC, handled the platform's visual and interactive design.

Beyond teaching, Signs functions as a crowdsourcing engine. Anyone can record and submit clips of themselves signing specific words. NVIDIA aims to collect 400,000 video clips covering 1,000 signs, with every submission verified by fluent ASL users and professional interpreters to maintain high fidelity.

Once completed, NVIDIA plans to open-source the dataset to the public, offering raw material for developers building accessible digital humans, meeting tools, and real-time translation agents.

Looking beyond the hands

There is still a significant technical hurdle: ASL is not just about hands. Facial expressions, eyebrow raises, and head tilts provide critical grammatical markers and emotional nuances.

Right now, Signs exclusively tracks finger and hand orientation. The team acknowledges this limitation and is actively researching ways to capture non-manual signals in upcoming releases.

They are also consulting with researchers at the Rochester Institute of Technology's Center for Accessibility and Inclusion Research to tackle regional dialects, slang, and general user experience for deaf and hard-of-hearing communities.

Why it matters

Most deaf children are born to hearing parents, who often scramble to learn ASL while navigating busy lives. Having a friction-free, browser-based tool allows families to start communicating with kids as early as six to eight months old without waiting on formal classes.

For the AI industry, high-quality multimodal training data is hitting a wall. Sign language is particularly tricky because it involves continuous 3D spatial dynamics over time. By putting 400,000 verified clips into the public domain, NVIDIA is essentially laying down the foundational rails for real-time sign language interfaces.

We think turning a consumer-facing learning tool into a community data flywheel is clever. It generates realistic, diverse user data far better than hiring a handful of actors in a studio. The catch is that sign language without facial expressions is only half a conversation; until computer vision integrates those non-manual cues, true conversational agents will remain out of reach.