A language model bigger than GPT-3 has arrived with a bold ambition: freeingAIfrom Big Techs clutches.
Its also multilingual unlikeGooglesLaMDA and OpenAIs GPT-3 an unusual feature in an English-dominated field.
It’s free, every week, in your inbox.
These features could democratize access to technology thats set to make a deep impact on society.
Powerful AI models can be trained and released in an open way.
Theyre also adeptat producing harmful content and their future capabilities aredifficult to predict.
BLOOM gives researchers a unique chance to explore their risks and benefits.
said Teven Le Scao, co-lead of BLOOMs training, in a statement.
Opening AI
LLMs are prohibitively expensive to create and run.
Training GPT-3, for instance, was estimated to costup to $27.6 million.
Inevitably, tech companies want to protect such large investments particularly when they provide competitive advantages.
Its therefore unsurprising that LLMs are rarely open-sourced with some notable exceptions.
Meta has produced the most prominent anomaly.
In May, the companyoffered access to the 175-billion parameter OPT system.
The full model, however, is only available upon request and for non-commercial use.
BLOOM ramps up the accessibility.
The 176-billion-parameter model is available for free to any individual or institution who agrees tothe systemsResponsible AI License.
Anyone canpublicly viewthe meeting notes, discussions, and code behind the model.
The initiative is bootstrapped and led by AI startupHugging Face.
The training corpus aligned with our values.
They trained the modelon theJean Zay supercomputerin Paris, France.
That meant we were able to mitigate ethical concerns without compromising on performance or scale.
The size is certainly imposing.At 176 billion parameters, BLOOM is larger than OpenAIs GPT-3 and MetaAIs OPT.
The model can generate text in 46 natural languages and dialects and 13 programming languages.
For many of them, its the first-ever language model with over 100B parameters.
Its also uniquely affordable.BigScience saysresearchers can use BLOOM for less than $40/hr on a cloud provider.
Story byThomas Macaulay
Thomas is the managing editor of TNW.
He leads our coverage of European tech and oversees our talented team of writers.
Away from work, he e(show all)Thomas is the managing editor of TNW.
He leads our coverage of European tech and oversees our talented team of writers.
Away from work, he enjoys playing chess (badly) and the guitar (even worse).
