Called AlphaCode, the model is based on Transformers, the same architecture OpenAI uses in itscode-generation models.
Programming is one of the promising applications of deep learning and large language models.
And in this regard, AlphaCode surely impresses.
It has managed to solve complicated programming challenges that typically require hours of planning, coding, and testing.
It might one day become a good tool to turn problem descriptions into working code.
But it certainly isnt the equivalent of a human programmer of any level.
Its a totally different approach to creating software, one that isnt complete without human thinking and intuition.
40% off TNW Conference!
AlphaCode, on the other hand, aims to solve competitive programming problems.
Finally, their results are evaluated based on performance on hidden tests that werent available during implementation.
A coding challenge can have other conditions such as timing and memory constraints.
It is much more difficult than synthesizing a source code excerpt based on previously seen examples.
This kind ofdeep learning systemis generally known as a sequence-to-sequence model (seq2seq).
Seq2seq algorithms take a sequence of values (letters, pixels, numbers, etc.)
as input and produce another sequence of values.
According to DeepMinds paper, AlphaCode uses an encoder-decoder Transformer architecture.
The encoder part of AlphaCode creates a numerical representation of the natural language description of the problem.
In the pretraining phase, AlphaCode went throughunsupervised learningon 715 gigabytes of data extracted from GitHub.
The model is trained by trying to predict the missing parts of a language or code snippet.
The pretrained model is then fine-tuned onCodeContests, an annotated dataset created by the DeepMind team.
The model is trained to transform the textual description of the challenge into the resulting source code.
Its results are evaluated with test cases and compared to the correct submissions.
This made sure that the ML model would not generate memorized results when faced with coding challenges.
When AlphaCode processes a new problem, it generates many solutions.
It then uses a filtering algorithm to choose the best 10 candidates and submits them to the competition.
If at least one of them is correct, then the problem is considered solved.
The samples are then filtered to only include those that pass the tests included in the problem statement.
This removes approximately 99 percent of the generated samples, according to the paper.
But this still leaves thousands of valid samples.
To optimize the sample-selection process, a clustering algorithm is used to divide the solutions into groups.
According to the researchers, the clustering process tends to group the working solutions together.
However, some publications have mistaken this claim for AI coding being as good as human programmers.
This is the fallacy ofcomparing narrow AI with the general problem-solving capabilities of humans.
Two prime examples are DeepBlue andAlphaGo, the AI systems that beat the world champions at chess and Go.
While both systems were terrific achievements of computer science and artificial intelligence, they only excelled at one task.
The same thing can be said about competitive programming.
A human programmer who reaches a competitive level in coding challenges has spent years studying.
In a nutshell, these competitions have been designed for humans.
This is why many companies use these challenges to make hiring decisions.
AlphaCode, on the other hand, is a shortcut for competitive programmingalbeit a brilliant one.
It creates novel code.
It doesnt copy-paste from its training data.
But it is not the equivalent of an average programmer.
Human programmers use their intuition to direct their limited computing resources in the direction of the right solution.
They use debugging, analysis, and review to refine their code.
In contrast, AlphaCode generates thousands of samplessometimes up to 100,000and filters them to find the ones that work.
As computer science professorErnest Davis observes, There is a substantial component of monkeys typing Hamlet going on here.
It then produces 10 candidates, and considers it a success if one of those is correct.
This is not an attack against AlphaCode.
However, we must also acknowledge the limits of this approach.
First, as Davis notes, the problem becomes extremely harder as the solution becomes longer.
Such tools can have a tremendous impact on the productivity of human programmers.
But human programmers will still be in control.
They have to learn toharness the power and limits of AI-generated code.
It should also be recognized for what it is not: the digital equivalent of a human programmer.
you’ve got the option to read the original articlehere.
