Meta's CM3leon handles both text and images with 5x less compute
Meta just unveiled CM3leon, a single foundation model capable of both text-to-image and image-to-text generation. By adapting recipes from text-only language models, it achieved a state-of-the-art 4.88 FID score on zero-shot MS-COCO while requiring five times less compute than previous transformer methods. It marks a big step toward unified autoregressive multimodality.