TII drops Falcon 180B, an open-source giant rivaling PaLM 2

Falcon 180B is an open-access language model developed by the Technology Innovation Institute (TII), packing a massive 180 billion parameters to claim the crown as the largest openly released LLM to date.

It is 2.5 times the size of Meta's Llama 2 70B and chewed through 3.5 trillion tokens on 4,096 GPUs, racking up roughly 7 million GPU hours on Amazon SageMaker. In benchmark evaluations, it beats Llama 2 70B and OpenAI's GPT-3.5 on MMLU while matching Google's proprietary PaLM 2-Large across standard NLP benchmarks.

Under the hood

Architecturally, Falcon 180B scales up the foundation laid by Falcon 40B, relying heavily on multiquery attention to keep memory overhead manageable and improve inference throughput.

The training corpus leans heavily on TII's RefinedWeb dataset, which accounts for around 85% of the mix. The remaining slice combines curated technical papers, conversations, and roughly 3% code. Because the dataset is so vast, running through 3.5 trillion tokens did not even complete a full single epoch.

TII released both a base pretrained model and an instruction-tuned chat variant. The base model serves as a clean canvas for custom fine-tuning, while the chat version uses a straightforward prompt template that alternates between user and bot turns.

Hardware specs and performance

On the Hugging Face Open LLM Leaderboard at release, Falcon 180B grabbed top honors with a score of 68.74. Across benchmarks like HellaSwag, LAMBADA, Winogrande, and ARC, it lands comfortably between GPT-3.5 and GPT-4 in overall capability.

The trade-off is brutal hardware requirements. Running inference in native BF16 or FP16 requires 640GB of VRAM, translating to a cluster of eight 80GB A100 GPUs. Even aggressive 4-bit INT4 quantization demands 320GB of memory, or eight 40GB A100 cards.

Training demands are equally steep. Full fine-tuning requires 64 A100 80GB cards, while LoRA with ZeRO-3 needs 16 cards. The most accessible route is QLoRA, which can run on two 80GB A100s. Crucially, community testing shows 4-bit and 8-bit quantized weights retain almost identical benchmark accuracy compared to the unquantized release.

How to run it and licensing terms

The model is available directly via the Hugging Face Hub under tiiuae/falcon-180B. Starting with Transformers 4.33, you can pull weights using native pipelines, safetensors, and bitsandbytes integration. Those without an enterprise GPU farm can try the interactive Falcon Chat demo hosted on Hugging Face Spaces.

Keep an eye on the legal terms. Falcon 180B allows commercial use, but under restrictive conditions that explicitly prohibit any hosting services. If your business model involves renting out model access as an API, this license shuts the door.

Why it matters

Falcon 180B proves that open-weight models can brute-force their way to parity with premier proprietary systems like PaLM 2. It resets expectations for what open community weights can achieve when paired with massive compute and thoroughly curated web data.

That said, this is an enterprise-grade toy rather than a daily driver for independent developers. Between the requirement for eight enterprise GPUs just to generate responses and the strict restrictions on commercial hosting, it is built for well-funded research teams and corporate labs looking to escape proprietary vendor lock-in.