Hugging Face welcomes Falcon, the open LLM beating LLaMA

Falcon is an open-source family of large language models developed by Abu Dhabi's Technology Innovation Institute (TII) and distributed under the Apache 2.0 license, meaning it is completely free for commercial use. The release includes two base models, Falcon-40B and Falcon-7B, along with experimental instruct-tuned variants for conversational tasks. At launch, Falcon-40B claimed the top spot on the Hugging Face Open LLM Leaderboard, outperforming LLaMA-65B.

Along with the model weights and code, TII also open-sourced an extract of 600 billion tokens from its custom RefinedWeb dataset, giving developers direct access to the training foundation.

How it works

The driving factor behind Falcon's performance is clean data. Over 80% of its pretraining data comes from RefinedWeb, an extensive pipeline built on CommonCrawl that uses aggressive deduplication and strict heuristic filtering. Falcon-7B was trained on 1.5 trillion tokens, while Falcon-40B was trained on 1 trillion tokens with roughly half the compute used by LLaMA-65B.

On the architecture side, Falcon introduces multiquery attention. Instead of allocating distinct keys and values to every attention head, it shares a single key and value across all heads. This trick leaves pretraining performance intact while shrinking the inference KV-cache by 10x to 100x. For a 2,048-token context, Falcon-40B's cache requires just 240MB of memory compared to 5,400MB for LLaMA-65B, while Falcon-7B sits at a tiny 20MB.

Hardware requirements and deployment

Falcon-7B requires around 15GB of GPU memory, making full-precision inference and fine-tuning feasible on consumer GPUs. Hugging Face also demonstrated a Core ML build running directly on an M1 MacBook Pro.

Falcon-40B is heavier, requiring roughly 90GB of VRAM in native bfloat16, which spills over a single 80GB A100. However, using 8-bit quantization brings it down to about 45GB, fitting inside a 48GB Nvidia A6000. Under 4-bit quantization, memory usage drops to roughly 27GB, opening the door for 30GB to 40GB GPUs.

To try it out, models are available on Hugging Face Spaces and the Text Generation Inference (TGI) stack. When loading through transformers, users need to enable trust_remote_code=True to fetch the custom model files and stick to bfloat16 for optimal precision.

Why it matters

Until Falcon, developers seeking top-tier open models were largely handcuffed by restrictive research-only licenses. Falcon-40B proves that truly open models can match and beat closed or research-gated alternatives like LLaMA-65B without legal gray areas, making it an immediate default for commercial builders.

We think the real breakthrough here is inference efficiency. By drastically cutting the memory footprint of the KV-cache, Falcon makes real-world production serving far cheaper at scale. If you are building enterprise assistants or self-hosted reasoning pipelines, Falcon is the first model you should test right now.