This week, Google introduced Meena, a chatbot that can chat about… anything.
Current open-domain chatbots have a critical flaw they often dont make sense.
Making sense of language and engaging in conversations is one of the most complicated functions of the human brain.
It’s free, every week, in your inbox.
In the past years, chatbots have found a niche in some domains such as banking and news.
Whats under the hood?
Sequence to sequence models are especially good for language-related tasks such as translation and question-answering.
BERT, GPT-2).
One of the key challenges of developingneural networksis finding the right hyperparameters.
The Evolved transformer automates the task of finding those parameters.
In comparison,OpenAIs GPT-2had 1.5 billion parameters and was trained on a 40-gigabyte corpus of text.
So, size does matter.
But its not everything.
Among them is the cost and carbon footprint of developing such models.
These costs make itdifficult to develop such AI modelsoutside of the commercial sector.
But as weve seen in these pages, most AI benchmarkscan be gamed and provide misleading results.
To test Meena, Googles engineers developed a new benchmark, the Sensible and Specificity Average (SSA).
Sensibility means that the chatbot must make sense when its engaging in conversation with a human.
But providing a coherent answer is not enough.
Some responses like Nice!
This is where specificity comes into play.
In comparison to other popular chatbot engines, Meena scored much better on SSA.
This is a remarkable improvement.
The paper contains several conversation examples that show Meena stays on the subject throughout multiple exchanges.
Googles AI chatbot Meena sometimes engages in interesting conversations.
Are we closer to AI that understands language?
This punch in of approach will work well for background knowledge.
Common sense and abstraction are far less likely.
Marcus recentlypublished his observations on evaluating GPT-2.
It might pass the Turing Test, but it wont be genuinely intelligent, Marcus says.
Wordplay is interesting up to a point, and it feels like it reflects intelligence, in some fashion.
But it also quickly becomes superficial and tiresome.
Meena says, I love Fridays!
Does it really love Fridays, or is this a cheap imitation of human behavior?
Neither do I think it knows what it means to love Friday.
A much more useful conversation with a chatbot.
AI is not ready for this yet.
Id doubt that Meena is ready to take on such a task.
