Cohere drops 100M multilingual Wikipedia embeddings

If you have ever tried building a neural search engine over Wikipedia, you already know the pain. You have to scrape millions of pages, split them into digestible chunks, run them through an embedding model, and pray your API bill or cloud compute bill does not explode along the way.

Cohere took that entire chore off your plate. The AI company released The Embedding Archives, a massive dataset packing around 100 million multilingual embedding vectors across Wikipedia articles, and made it completely free for anyone to download.

Pre-chunked Wikipedia in 10 languages

The archive spans 10 major languages: English, German, French, Spanish, Italian, Japanese, Arabic, Chinese, Korean, and Hindi, along with Simple English.

Cohere broke the Wikipedia articles down into individual passages, attached the article title to the start of each text block, and passed them through its multilingual-22-12 model. The result is a clean 768-dimensional float vector for every single snippet.

The entire collection lives on Hugging Face Datasets. You can pull down the passages, vectors, and metadata with a single line of Python code, completely skipping the messy parsing stage.

Zero-download search with Weaviate

If you do not want to download dozens of gigabytes of raw vectors onto your local machine, Cohere also partnered with vector database Weaviate to host a ready-to-query version online.

Weaviate hosts a public 10-million vector subset featuring 1 million entries for each of the 10 languages. You can fire natural language queries like plot twists directly at it and get semantic matches back in milliseconds, with optional language filtering if you only want results in Japanese or German.

Because the underlying model operates in a shared cross-lingual space, concepts that mean the same thing in different languages end up clustered close together. You can query in one language and easily retrieve relevant context written in another.

Why it matters

For developers and startups experimenting with RAG and semantic search, this removes one of the steepest upfront friction points. Getting clean, pre-embedded world knowledge without paying a cent in compute lets builders validate prototypes on day one.

Strategically, this is Cohere playing the long game with style. Giving away millions of precomputed embeddings looks like open-source philanthropy, but it anchors developers directly into Cohere's embedding ecosystem. The moment you need to embed incoming user queries to search that archive, you are pinging Cohere's API.

The obvious catch is freshness. Wikipedia updates every minute, and static vector dumps are snapshots frozen in time. You cannot rely on this for breaking news or volatile information. But as a massive, ready-made playground for multilingual neural retrieval, it is about as generous as developer tooling gets.