The notoriously secretiveMetahas set a milestone for transparency.
The company this week offered the entire research community access to a fully-trained large language model (LLM).
Named the Open Pretrained Transformer (OPT), the system mirrors the performance and size ofOpenAIsvaunted GPT-3 model.
This mimicry is deliberate.
It’s free, every week, in your inbox.
No other company has ever provided this level of access to an LLM.
Such openness mayappear uncharacteristic.
After all, Meta is often accused of concealing its algorithms andtheir harmful impacts.
Yet the move maynot be entirely altruistic.
The companys public embrace of transparency could also dampen criticism of its secrecy.
Mutual benefits
Metas researchers acknowledge that OPT has major shortcomings.
They note that the system doesnt work well with declarative instructions or point-blank interrogatives.
It also has a tendency to generate toxic language and reinforce harmful stereotypes even when fed relatively innocuous prompts.
The move will hopefully show that businesses and society both benefit from transparency.
you could get the OPT open-source code and small-scale pre-trained modelshere.
He leads our coverage of European tech and oversees our talented team of writers.
Away from work, he e(show all)Thomas is the managing editor of TNW.
He leads our coverage of European tech and oversees our talented team of writers.
Away from work, he enjoys playing chess (badly) and the guitar (even worse).
