Even if I become the number one, there is an entity that cannot be defeated.
Uninformed analysts have been picking up on these successes to suggest that AI is becoming smarter than humans.
This begs the question, does mastering a game prove anything?
And if not, how can you measure the level of intelligence of an AI system?
Take the following example.
In the picture below, youre presented with three problems and their solution.
Theres also a fourth task that hasnt been solved.
Can you guess the solution?
(Source: Arxiv.org)
Youre probably going to think that its very easy.
Chollet published this paper a few weeks before Le-sedol declared his retirement.
In it, he provided many important guidelines on understanding and measuring intelligence.
Ironically, Chollets paper did not receive a fraction of the attention it needs.
Unfortunately, the media is more interested incovering exciting AI news that gets more clicks.
Whats wrong with current AI?
In fact, the obsession with optimizing AI algorithms for specific tasks has entrenched the community innarrow AI.
Chollets observations are in line with those made by other scientists on thelimitations and challenges of deep learning systems.
The same can be seen in other fields,such as self-driving cars.
What is intelligence?
One of the key challenges that the AI community has struggled with is defining intelligence.
The keyhere is achieve goals and wide range of environments.
Chollet then examines the two dominant approaches in creating intelligence systems: symbolic AI and machine learning.
This approach requires human engineers to meticulously write the rules that define the behavior of an AI agent.
The AI examines the data and develops a mathematical model that represents the common traits of cancer patterns.
It can then process new slides and outputs how likely it is that the patients have cancer.
Neural internet-based models, also known as connectionist AI, are named after their biological counterparts.
We see the world through the lens of the tools we are most familiar with.
Truly intelligent systems should be able to develop higher-level skills that can span across many tasks.
Francois Chollet breaks down intelligence into a hierarchy of three layers.
Current AI systems are still struggling at the bottom rung of this ladder.
Interestingly, this is a missing component of both symbolic and connectionist AI.
For instance, consider Stockfish, the best rule-base chess-playing program.
The brightest human cant memorize tens of thousands of chess rules.
Likewise, no human can play millions of chess games in a lifetime.
ARC is a set of problem-solving tasks that tailored for both AI and humans.
One of the key ideas behind ARC is to level the playing ground between humans and AI.
For instance, it doesnt involve language-related problems, whichAI systems have historically struggled with.
The system does not provide access to vast amounts of training data.
A training example in the ARC dataset: The red object must be made adjacent to the blue box.
Optimizing for evaluation sets is a popular cheating method in data science and machine learning competitions.
In his experiments with ARC, Chollet has found that humans can fully solve ARC tests.
But current AI systems struggle with the same tasks.
