Data Skeptic
A Survey Assessing Github Copilot
Nov 20, 2023
Jenny Liang, a PhD student at Carnegie Mellon University, discusses her recent survey on the usability of AI programming assistants. She shares some questions and takeaways from the survey, as well as the major reasons developers don't want to use code-generation tools. Concerns about intellectual property and the access code-generation tools have to in-house code are discussed.
26:25
Episode guests
AI Summary
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- Developers struggle with controlling code generation models like GitHub Copilot, leading to generated code that often doesn't meet requirements.
- GitHub Copilot is valued by developers for completing shorter and line code completions, but they are less inclined to use it for longer snippets or uncertain tasks.
Deep dives
Developers' struggle with controllability of code generation tools
The study found that developers found it difficult to control code generation models like GitHub Copilot. They were unsure about what input would cause a specific output. This lack of controllability was seen as a problem, especially since developers reported that generated code often did not meet functional or non-functional requirements. Controllability was identified as an important concept to address in code generation tools.
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
