The Game is Over!
My opinion: Its all about scale now!
The Game is Over!
My opinion: Its all about scale now!
The Game is Over!
Research focused on these problems, eg S4 for greater memory, is needed.
Philosophy about symbols isnt.
I learned it from [Google researcher Geoffrey Hinton] @geoffreyhinton a decade ago.
Geoff predicted what was predictable with uncanny clarity.
It’s free, every week, in your inbox.
How did we get here?
DeepMind recently released a research paper andpublished a blog poston its new multi-modal AI system.
Its easy to confuse something like Gato with AGI.
The difference, however, is that a general intelligence could learn to do new things without prior training.
Its not a general AI, its a bunch of pre-trained, narrow models bundled neatly.
Thats not a bad thing, if thats what youre looking for.
Doctor de Freitas disagrees.
Hesdebated and discussedhis ideas with everyone from Facebooks Yann LeCun to the University of Montreals Yoshua Bengio.
Of course, that name, Scaling-Uber-Alles, is not entirely fair.
Marcus goes on to describe the problem of incomprehensibility that inundates the AI industrys giant-sized models.
While thats certainly worth a chuckle, theres a serious undertone there.
NeitherGato, DALL-E, nor GPT-3 are robust enough for unfettered public consumption.
Its unclear how scaling, even coupled with breakthrough logic algorithms, could fix these issues.
That doesnt mean giant-sized models arent useful or worthy endeavors.
What DeepMind, OpenAI, and similar labs are doing is very important.
Its science at the cutting-edge.
But to declare the game is over?
To insinuate that AGI will arise from a system whose distinguishing contribution is how it serves models?
Gato is amazing, but that feels like a stretch.
Theres nothing in de Freitas spirited rebuttal to change my opinion.
Gatos creators are obviously brilliant.
Im not pessimistic about AGI because Gato isnt mind-blowing enough.
Quite the opposite, in fact.
I fear AGI is decades more away centuries, perhaps because of Gato, DALL-E, and GPT-3.
They each demonstrate a breakthrough in our ability to manipulate computers.
to the Gandalfian sorcery of AGI weve been promised.
It is sheer hubris to believe otherwise.
