Skild AI's S1 robot foundation model learns new tasks from a single video

Training a robot to perform a task has traditionally felt like stepping back into deep learning's dark ages: collect dozens of hours of teleoperation data, run expensive post-training, and pray the policy doesn't glitch.

Skild AI launched S1, a robotics foundation model that throws away task-specific fine-tuning entirely. Instead of collecting new weights for every action, S1 treats robotic manipulation just like a frontier language model treats text prompts: show it a single video demonstration, and it gets straight to work.

Robotics finally gets its GPT-3 moment

Skild AI pointed out that robotics has remained trapped in the "BERT era." While modern neural nets can learn complex moves, teaching a robot a new task still demands tens to hundreds of hours of deployment data to fine-tune a specialist policy.

Research has already shown that when post-training data is dense enough, policies trained from scratch easily match pre-trained models. That raised an uncomfortable question across the industry: what is the actual point of pre-training if you still have to train from scratch anyway?

Skild argues pre-training should enable in-context learning, just as the leap from BERT to GPT-3 did for NLP. Exactly one year after releasing an in-context learner for locomotion, the team built S1 from the ground up to bring that exact zero-fine-tuning adaptability to manipulation.

Learning coffee and pancakes from a video prompt

Language prompts work fine when you tell a robot to hand you a cup. But when tasks become nuanced, delicate, or long, language breaks down. Humans don't learn to fold fitted sheets or flip pancakes through spoken instructions; they watch someone do it.

S1 accepts task specifications purely through video demonstrations. Crucially, Skild evaluated the model on tasks that were entirely absent from its pre-training dataset and spanned up to 10 minutes in duration, such as making coffee, potting plants, and flipping pancakes.

On completely novel workflows, S1 outpaced conventional vision-language-action (VLA) models. In fact, running off a single video prompt matched the benchmark performance of existing VLAs that required 50 to 100 hours of fine-tuning data. The model is now rolling out to select industrial partners.

Why it matters

For industrial automation and enterprise deployments, ditching fine-tuning is huge. Re-tooling an assembly station or introducing a new packaging workflow usually costs days of engineering time and expensive data logging. If an operator can show an off-the-shelf arm a two-minute video and have it running reliably, deployment bottlenecks collapse.

We think Skild's bet on video-prompted in-context learning is exactly where robotics needed to go. It directly addresses the dirty secret of physical AI: standard pre-training hasn't been transferring well to novel tasks. By stretching horizon lengths to 10 minutes on genuinely out-of-distribution chores, S1 sets a much more realistic bar for general-purpose machines.

That said, video alone doesn't capture tactile feedback, resistance, or mass. While visual imitation can guide rough trajectories, high-precision assembly and delicate handling will quickly test whether a pure visual in-context learner can survive unexpected real-world friction without physical tuning.