Jenny Liang, a PhD student at Carnegie Mellon University, discusses her recent survey on the usability of AI programming assistants. She shares some questions and takeaways from the survey, as well as the major reasons developers don't want to use code-generation tools. Concerns about intellectual property and the access code-generation tools have to in-house code are discussed.
Read more
AI Summary
AI Chapters
Episode notes
auto_awesome
Podcast summary created with Snipd AI
Quick takeaways
Developers struggle with controlling code generation models like GitHub Copilot, leading to generated code that often doesn't meet requirements.
GitHub Copilot is valued by developers for completing shorter and line code completions, but they are less inclined to use it for longer snippets or uncertain tasks.
Deep dives
Developers' struggle with controllability of code generation tools
The study found that developers found it difficult to control code generation models like GitHub Copilot. They were unsure about what input would cause a specific output. This lack of controllability was seen as a problem, especially since developers reported that generated code often did not meet functional or non-functional requirements. Controllability was identified as an important concept to address in code generation tools.
Value of code completion and shorter completions in GitHub Copilot
The survey revealed that developers valued GitHub Copilot's ability to complete code for them, particularly with shorter completions and line completions. This aligns with the concept of acceleration mode, where developers know what they want and use Copilot as a faster way to reach their desired code. The study also highlighted that developers were less inclined to use Copilot for generating longer snippets of code or for uncertain or exploratory tasks.
Future directions and concerns in AI-powered programming tools
The study indicated that AI-powered programming tools like GitHub Copilot and GPT4 have the potential to be integrated at multiple stages of development, including code review, testing, and more. Personalization, capturing individual programming styles, and real-time updates were suggested as areas for improvement. Concerns regarding intellectual property infringement and the need for AI tools to keep up with changing programming languages were also raised. The research suggested that the landscape of programming tools is evolving and that natural language interfaces may play an increasingly significant role in programming assistance.
In this episode, we are joined by Jenny Liang, a PhD student at Carnegie Mellon University, where she studies the usability of code generation tools. She discusses her recent survey on the usability of AI programming assistants.
Jenny discussed the method she used to gather people to complete her survey. She also shared some questions in her survey alongside vital takeaways. She shared the major reasons for developers not wanting to us code-generation tools. She stressed that the code-generation tools might access the software developers' in-house code, which is intellectual property.
Learn more about Jenny Liang via https://jennyliang.me/
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode