The secret to good programming might be to ignore everything we know about writing code. At least for AI.
It seems preposterous, but DeepMind’s new coding AI just trounced roughly 50 percent of human coders in a highly competitive programming competition. On the surface the tasks sound relatively simple: each coder is presented with a problem in everyday language, and the contestants need to write a program to solve the task as fast as possible—and hopefully, free of errors.
But it’s a behemoth challenge for AI coders. The agents need to first understand the task—something that comes naturally to humans—and then generate code for tricky problems that challenge even the best human programmers.
AI programmers are nothing new. Back in 2021, the non-profit research lab OpenAI released Codex, a program proficient in over a dozen programming languages and tuned in to natural, everyday language. What sets DeepMind’s AI release—dubbed AlphaCode—apart is in part what it doesn’t need.
Unlike previous AI coders, AlphaCode is relatively naïve. It doesn’t have any built-in knowledge about computer code syntax or structure. Rather, it learns somewhat similarly to toddlers grasping their first language. AlphaCode takes a “data-only” approach. It learns by observing buckets of existing code and is eventually able to flexibly deconstruct and combine “words” and “phrases”—in this case, snippets of code—to solve new problems.
When challenged with the CodeContest—the battle rap torment of competitive programming—the AI solved about 30 percent of the problems, while beating half the human competition. The success rate may seem measly, but these are incredibly complex problems. OpenAI’s Codex, for example, managed single-digit success when faced with similar benchmarks.
“It’s very impressive, the performance they’re able to achieve on some pretty challenging problems,” said Dr. Armando Solar-Lezama at MIT, who was not involved in the research.
The problems AlphaCode tackled are far from everyday applications—think of it more as a sophisticated math tournament in school. It’s also unlikely the AI will take over programming completely, as its code is riddled with errors. But it could take over mundane tasks or offer out-of-the-box solutions that evade human programmers.
Perhaps more importantly, AlphaCode paves the road for a novel way to design AI coders: forget past experience and just listen to the data.
“It may seem surprising that this procedure has any chance of creating correct code,” said Dr. J. Zico Kolter at Carnegie Mellon University and the Bosch Center for AI in Pittsburgh, who was not involved in the research. But what AlphaCode shows is when “given the proper data and model complexity, coherent structure can emerge,” even if it’s debatable whether the AI truly “understands” the task at hand.
Language to Code
AlphaCode is just the latest attempt at harnessing AI to generate better programs.
Coding is a bit like writing a cookbook. Each task requires multiple tiers of accuracy: one is the overall structure of the program, akin to an overview of the recipe. Another is detailing each procedure in extremely clear language and syntax, like describing each step of what to do, how much of each ingredient needs to go in, at what temperature and with what tools.
Each of these parameters—say, cacao to make hot chocolate—are called “variables” in a computer program. Put simply, a program needs to define the variables—let’s say “c” for cacao. It then …….