The challenges of deep learning
Deep learningis often compared to the brains of humans and animals.
In their paper, Bengio, Hinton, and LeCun acknowledge these shortcomings.
The model is trained to find recurring patterns in examples that have similar labels.
It then uses the learned patterns to associate new examples with the right labels.
Supervised learning is especially useful for problems where labeled examples are abundantly available.
Reinforcement learningis another branch of machine learning, in which an agent learns to maximize rewards in an environment.
The agent starts by taking random actions.
As it receives feedback from its environment, it finds sequences of actions that provide better rewards.
In both cases, as the scientists acknowledge, machine learning models require huge labor.
assumption, which supposes that real-world data has the same distribution as the training data.
From the early days, theoreticians of machine learning have focused on the iid assumption…
Unfortunately, this is not a realistic assumption in the real world, the scientists write.
And we know that humans dont suffer from the problems of current deep learning systems.
Scientists provide various solutions to end the gap between AI and human intelligence.
Symbol manipulation is a very important part of humans ability to reason about the world.
It is also one of the great challenges of deep learning systems.
Bengio, Hinton, and LeCun do not believe in mixing neural networks and symbolic AI.
But our work suggests otherwise.
One of the benefits of Transformers is their capability to learn without the need for labeled data.
More recently, researchers have shown that Transformers can be applied to computer vision tasks as well.
When combined withconvolutional neural networks, transformers can predict the content of masked regions.
This is an intriguing approach and seems to be much closer to what the human mind does.
The paper also touches upon system 2 deep learning, a term borrowed from Nobel laureate psychologist Daniel Kahneman.
And we expect there to be radically new ideas, Hinton told ACM.
you could read the original articlehere.
