Hugging Face launches AI WebTV powered by open-source models

AI WebTV is an experimental open-source project that chains generative models together to broadcast an uninterrupted, automated video stream. It takes a basic premise, uses a language model to expand it into shot-by-shot prompts, synthesizes video clips, smooths out the motion, pairs them with generated music, and streams the finished reel live 24/7.

How the pipeline works

The entire setup is written in Node.js and TypeScript, leaning on open-source models hosted across Hugging Face Spaces.

The pipeline starts with ideation. A human provides a base theme, and ChatGPT breaks it down into multiple detailed video prompts to keep the sequence rolling with variety.

Video generation relies on Zeroscope V2, an open-source model based on ModelScope. The system runs a two-step generation pass: first with zeroscope_v2_576 to produce a 576x320 clip, followed by zeroscope_v2_XL to upscale the result to 1024x576 using the exact same prompt.

Instead of hosting everything locally from scratch, the prototype calls Gradio apps on Hugging Face Spaces via the @gradio/client package, downloading the rendered takes directly to local storage.

Once a clip is upscaled, post-processing kicks in. The pipeline runs the footage through FILM, a frame interpolation algorithm that smooths out jerky diffusion frames and stabilizes camera pans. In parallel, it calls MusicGen to generate accompanying background audio.

Finally, FFmpeg stitches the mp4 video and m4a audio clips into a playlist. It converts the feed into an FLV stream and pushes it over RTMP to an instance of node-media-server or directly to Twitch.

Quirks and limitations

While the combination of an upscaling pass and FILM frame interpolation visibly cleans up output, the underlying models still produce plenty of glitches.

Movement direction is a recurring issue. Clips sometimes play in reverse, and specific modifiers like color keywords are often overlooked by the diffusion process.

The models also tend to take text prompts too literally. For example, adding the word llama generated a llama along with burning letters spelling out llama in the scene. Mentioning camera gear like Canon EOS or IMAX can cause actual cameras or watermark-like text to appear in the frame.

To counter this, the project recommends writing camera-specific prompts with explicit directions, angles, and speed settings, while repeating consistent details across consecutive shots to maintain visual continuity.

How to try it

You can watch the live broadcast directly on the AI WebTV Space on Hugging Face, or through its Twitch mirror stream on mobile devices.

If you want to build and run your own channel, all core components and pipeline scripts are open. You will want to duplicate the public Gradio Spaces for Zeroscope and MusicGen to your own account to avoid public queue limits and timeouts.

Why it matters

AI WebTV proves that an end-to-end, fully autonomous generative broadcasting pipeline can be built today using off-the-shelf open-source models without proprietary video APIs. For developers building synthetic media pipelines, automated video workflows, or experimental streams, this architecture provides a practical blueprint.

That said, it remains an early tech demo. The clips are short, narrative coherence is virtually nonexistent, and the compute overhead of chaining multiple diffusion and interpolation passes makes real-time production demanding.