Data Skeptic cover image

Data Skeptic

A Survey Assessing Github Copilot

Nov 20, 2023
26:25
Snipd AI
Jenny Liang, a PhD student at Carnegie Mellon University, discusses her recent survey on the usability of AI programming assistants. She shares some questions and takeaways from the survey, as well as the major reasons developers don't want to use code-generation tools. Concerns about intellectual property and the access code-generation tools have to in-house code are discussed.
Read more

Podcast summary created with Snipd AI

Quick takeaways

  • Developers struggle with controlling code generation models like GitHub Copilot, leading to generated code that often doesn't meet requirements.
  • GitHub Copilot is valued by developers for completing shorter and line code completions, but they are less inclined to use it for longer snippets or uncertain tasks.

Deep dives

Developers' struggle with controllability of code generation tools

The study found that developers found it difficult to control code generation models like GitHub Copilot. They were unsure about what input would cause a specific output. This lack of controllability was seen as a problem, especially since developers reported that generated code often did not meet functional or non-functional requirements. Controllability was identified as an important concept to address in code generation tools.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode