Manifest AI drops attention from Qwen3 for just $4,000
Manifest AI just unveiled Brumby-14B-Base, a retrained variant of Qwen3 that completely ditches the transformer’s signature attention mechanism. Instead, it uses a recurrent architecture called Power Retention, keeping per-token compute costs flat across long contexts. The team trained it on 32 H100s for just $4,000 to match baseline performance, though they admit the cost-saving trick only works by building on existing transformer weights.