EleutherAI drops massive 8TB dataset of licensed and public domain text
EleutherAI just released The Common Pile v0.1, an 8-terabyte AI training dataset built entirely from licensed and public domain text over two years. Alongside it, the team dropped two 7B Comma models that perform on par with early Llama models, pushing back on the industry narrative that frontier AI requires unpermitted scraping.