AlphaCode - with the authors!

An interview with the creators of AlphaCode!

Paper review video here: https://youtu.be/s9UAOmyah1A

OUTLINE:

0:00 - Intro

1:10 - Media Reception

5:10 - How did the project go from start to finish?

9:15 - Does the model understand its own code?

14:45 - Are there plans to reduce the number of samples?

16:15 - Could one do smarter filtering of samples?

18:55 - How crucial are the public test cases?

21:55 - Could we imagine an adversarial method?

24:45 - How are coding problems even made?

27:40 - Does AlphaCode evaluate a solution's asymptotic complexity?

33:15 - Are our sampling procedures inappropriate for diversity?

36:30 - Are all generated solutions as instructive as the example?

41:30 - How are synthetic examples created during training?

42:30 - What were high and low points during this research?

45:25 - What was the most valid criticism after publication?

47:40 - What are applications in the real world?

51:00 - Where do we go from here?

Paper: https://storage.googleapis.com/deepmi...

Code: https://github.com/deepmind/code_cont...

Abstract: Programming is a powerful and ubiquitous problem-solving tool. Developing systems that can assist programmers or even generate programs independently could make programming more productive and accessible, yet so far incorporating innovations in AI has proven challenging. Recent large-scale language models have demonstrated an impressive ability to generate code, and are now able to complete simple programming tasks. However, these models still perform poorly when evaluated on more complex, unseen problems that require problem-solving skills beyond simply translating instructions into code. For example, competitive programming problems which require an understanding of algorithms and complex natural language remain extremely challenging. To address this gap, we introduce AlphaCode, a system for code generation that can create novel solutions to these problems that require deeper reasoning. Evaluated on recent programming competitions on the Codeforces platform, AlphaCode achieved on average a ranking of top 54.3% in programming competitions with more than 5,000 participants. We found that three key components were critical to achieve good and reliable performance: (1) an extensive and clean competitive programming dataset for training and evaluation, (2) large and efficient-to-sample transformer-based architectures, and (3) large-scale model sampling to explore the search space, followed by filtering based on program behavior to a small set of submissions.

Authors: Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu and Oriol Vinyals

Links:

Merch: store.ykilcher.com

TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick

YouTube: https://www.youtube.com/c/yannickilcher

Twitter: https://twitter.com/ykilcher

Discord: https://discord.gg/4H8xxDF

BitChute: https://www.bitchute.com/channel/yann...

LinkedIn: https://www.linkedin.com/in/ykilcher

BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):

SubscribeStar: https://www.subscribestar.com/yannick...

Patreon: https://www.patreon.com/yannickilcher