Reinforcement learning is one of the exciting branches of artificial intelligence.
It plays an important role in game-playing AI systems, modern robots,chip-design systems, and other applications.
There are many different types ofreinforcement learning algorithms, but two main categories are model-based and model-free RL.
They are both inspired by our understanding of learning in humans and animals.
Nearly every book on reinforcement learning contains a chapter that explains the differences between model-free and model-based reinforcement learning.
But seldom are the biological and evolutionary precedents discussed in books about reinforcement learning algorithms for computers.
40% off TNW Conference!
To escape, the cat had to manipulate a series of gadgets such as strings and levers.
Over time, the cat became faster and faster at escaping the box.
Thorndike concluded that the cat learned from the reward and punishments that its actions provided.
The law of effect is also the basis for model-free reinforcement learning.
In model-free reinforcement learning, an agent perceives the world, takes an action, and measures the reward.
The agent usually starts by taking random actions and gradually repeats those that are associated with more rewards.
Thats basically what model-free reinforcement learning is.
The simplest thing you might imagine.
In model-free reinforcement learning, theres no direct knowledge or model of the world.
The RL agent must directly experience every outcome of each action through trial and error.
During his experiments, Tolman realized that animals could learn things about their environment without reinforcement.
Tolman called this latent learning.
This is also the basis of model-based reinforcement learning.
In model-based reinforcement learning, you develop a model of the world.
With model-free reinforcement learning, the new information would not be of any use to you.
In some cases, creating a decent model of the environment is either not possible or too difficult.
Computationally, model-based reinforcement learning is a lot more elaborate.
On the contrary, in model-based reinforcement learning, the complexity will depend on the model you build.
Basically, neither model-based nor model-free reinforcement learning is a perfect solution.
But it also does forward search, which is model-based.
But despite remarkable achievements, progress in reinforcement learning is still slow.
As soon as RL models are faced with complex and unpredictable environments, their performance starts to degrade.
Other tasks such asrobotic hand manipulationalso require huge amounts of training and trial-and-error.
And we have much more than just model-free and model-based reinforcement learning, Lee believes.
When you have multiple learning algorithms, they become useless if you turn some of them off.
Even if youre relying on one algorithmsay model-free RLthe other algorithms must continue to run.
Some interesting work in AI research shows how this might work.
The information that we get from the environment is very sparse.
And using that information, we have to generalize.
Theres a lot more that we need to understand.
