Tsinghua's Falcon reads lips to isolate voices in noisy calls
Researchers from Tsinghua University just open-sourced Falcon, a lightweight model that isolates target voices in noisy rooms by tracking mouth movements. With just 0.5M trainable parameters, it drastically cuts latency compared to prior baselines. The accompanying Falcon Studio app runs on a laptop and feeds the clean audio straight into Zoom or Tencent Meeting.