Omnimat Zero cleans up video reflections and shadows
Omnimat Zero just open-sourced a video tool that erases objects along with their tricky reflections and ground shadows. It can also isolate foreground subjects into transparent layers for background swaps, making clean video cleanup far less tedious.
Anyone who edits video knows that erasing an object is rarely the hardest part. The real nightmare is cleaning up the lingering ground shadows and subtle water reflections that instantly ruin the shot.
Omnimat Zero tackles this specific headache head-on. The open-source tool doesn't just cut out targeted elements from video footage; it tracks down and wipes out the accompanying shadows and reflections too.
Handling reflections and tricky occlusions
In side-by-side tests, the tool shows impressive contextual awareness. When removing a cat, it seamlessly erases the reflection shimmering in a puddle. It pulls off the same trick with swans on water and erases ground shadows when removing people or a pack of dogs.
It also handles ridiculously fine details. In one demo, removing just the colon from a digital clock face also removed its faint reflection on a glossy tabletop. When tasked with deleting a person in the background while keeping a foreground pole intact, rival tools accidentally cut into the pole, while Omnimat Zero preserved it cleanly.
Beyond deletion, it can isolate foreground subjects into transparent layers for compositing. The complete codebase and local setup instructions are already out on GitHub.
Why it matters
For video editors, visual cleanup like tracking masks across reflections has always eaten up hours. Having a model that natively understands secondary light and shadows can turn a tedious rotoscoping job into a quick export.
That said, early feedback notes that its background-replacement composites still look rough and unnatural. We think you shouldn't rely on it for finished multi-layer composites just yet. Treat it as a dedicated, high-precision cleanup assistant to wipe out unwanted clutter before moving on to final grading.
Sources (3)
- arxiv.org(opens in a new tab)Originalarxiv.org
- dvirsamuel.github.io(opens in a new tab)Reportdvirsamuel.github.io
- github.com(opens in a new tab)Reportgithub.com