This article is part of our coverage of the latest inAI research.
What is the next step toward bridging the gap between natural and artificial intelligence?
Scientists and researchers are divided on the answer.
LeCun has been thinking and talking aboutself-supervised and unsupervised learningfor years.
The latter is referred to as out-of-distribution generalization or sensitivity to edge cases.
It’s free, every week, in your inbox.
Those are problems that humans and animals learn to solve very early in their lives.
You dont need to drive off a cliff to know that your car will fall and crash.
You know that when an object occludes another object, the latter still exists even if cant be seen.
We learn most of these things without being explicitly instructed,purely by observation and acting in the world.
We then use these basic building blocks to accumulate more complex knowledge.
The more popular branch of ML issupervised learning, in which models are trained on labeled examples.
First, supervised ML models require enormous human effort to label training examples.
And second, supervised ML models cant improve themselves because they need outside help to annotate new training examples.
It is like a supervised learning system that does its own data annotation.
The self-supervised learning paradigm is much more attuned to the way humans and animals learn.
Being able to train ML models on huge stores of unlabeled data has many applications.
In recent years, self-supervised learning has found its way into several areas of ML, includinglarge language models.
The model must venture to predict the missing parts.
The trained model will learn solid representations of how text is structured.
It can be used for tasks such as text generation or fine-tuned on downstream tasks such as question answering.
Scientists have also managed to apply self-supervised learning to computer vision tasks such asmedical imaging.
The web connection adjusts its parameters until its output remains consistent across different variations of the same image.
The model can then be fine-tuned on a downstream task with fewer labeled images.
High-level abstractions
More recently, scientists have experimented with pure self-supervised learning on computer vision tasks.
This is an extremely difficult problem, LeCun says.
Images are very high-dimensional spaces.
There are near-infinite ways in which pixels can be arranged in an image.
If theres a wall in front of it, you expect it to bounce back.
You know this because you have knowledge of intuitive physics and you know how rigid and soft bodies work.
Similarly, when a person is talking to you, you expect their facial features to change across frames.
But you dont expect their mouth and ears to suddenly switch places.
This makes it possible for the model to make inferences on high-dimensional information such as visual data.
The world model will be a key component of this architecture.
But it will also need to coordinate with other modules.
Among them is a perception module that receives and processes sensory information from the world.
An actor module turns perceptions and predictions into actions.
A short-term memory module keeps track of actions and perceptions and fills the gaps in the models information.
A cost module helps evaluate the intrinsicor hardwiredcosts of actions as well as the task-specific value of future states.
What will human-level AI do?
Most discussions ofhuman-level AIare about machines that replace natural intelligence and perform every task that a human can.
Scientists are widely divided on the outlook of artificial general intelligence.
Is consciousness a prerequisite for AGI?
Will AGI have its own goals and desires?
Can we create a brain in a vat and without a physical shell?
But a more practical direction of research is creating AI that is compatible with human intelligence.
This, I think, is the promise that LeCuns area of research holds.
it’s possible for you to read the original articlehere.
