OpenMOSS drops MOSS-VL-Realtime for continuous live video streams
OpenMOSS just open-sourced MOSS-VL-Realtime, an 11B parameter vision-language model tailored for live streaming video. Instead of waiting for an entire clip to load before answering, it processes incoming frames on the fly, allows users to interrupt with questions at any timestamp, and stays silent when visual evidence is insufficient.
MOSS-VL-Realtime is an open-source 11B vision-language model built specifically to ingest continuous video streams and answer queries on the fly.
Traditional video models usually force you into an offline workflow: record the whole clip, load all frames into memory, and wait for the model to digest before asking a question. OpenMOSS is taking a completely different approach by treating video as an unending stream of timestamped frames where perception and text generation happen side by side.
What it can do
Its headline feature is interruptibility. As your screen, camera feed, or video file streams in, you can throw a question at any timestamp. The model immediately responds using the visual context observed up to that exact second.
It also knows when to shut up. If nothing interesting happens on screen or visual evidence is still thin, the model outputs a silence token and keeps watching instead of hallucinating.
When subsequent frames bring new evidence, it dynamically revises its earlier output. Under the hood, it pairs a decoupled cross-attention architecture with a massive 256K context window. Using Cross-attention Rotary Position Embedding (XRoPE), it maps visual patches and text tokens across time, height, and width to keep track of event duration and sequence.
Getting started
The model weighs in at 11B parameters in BF16 precision. By default, it operates at 1.0 FPS with a window of up to 256 frames, supporting one active realtime session per model instance.
OpenMOSS has published the code on GitHub and the model weights on Hugging Face alongside a technical report on arXiv. You will need Python 3.12, transformers, accelerate, and ideally FlashAttention 2 for snappy inference.
For integration, it exposes two clean interfaces: a session-based API where you push frames and prompts into an ongoing stream, and a queue-based generator designed for multi-threaded backend services. Standard offline video inference is also supported if you need to run quick benchmarks.
Why it matters
If you are building camera-based agents, live stream commentary bots, or real-time desktop copilots, this checkpoint deserves a spot in your testing pipeline. At 11B parameters, it runs comfortably on modern single-GPU setups without forcing you into costly closed-source streaming APIs.
We think baking proactive silence and on-the-fly correction right into the model weights is a clever design choice that spares engineers from writing brittle heuristics. The 1.0 FPS default is on the conservative side for fast-paced action, but for ambient monitoring and interactive assistants, it hits a very practical sweet spot.
Sources (1)
- huggingface.co(opens in a new tab)Originalhuggingface.co