Josh Pigford, Founder of Flipper Cloud, joins Matt to discuss the importance of automated testing in software development, challenges and uncertainties in testing, measuring productivity and challenges in attribution, balancing craft and business in engineering, the future of test automation with AI, and the benefits of automated software testing.
The importance and extent of testing in a development project should be determined by the context, customer impact, and business goals.
AI-driven testing can help automate the process of writing tests, providing speed and efficiency, but human touch and judgment are still necessary to refine and customize the generated tests.
Deep dives
The trade-offs of automated testing
Automated testing is a topic that sparks debate among developers. While some argue that all code should be thoroughly tested, others believe that the context and goals of a project should determine the need for testing. The discussion delves into the trade-offs of automated testing, highlighting how it can be time-consuming and may not always align with business objectives. The speakers mention examples from their own experiences, such as the challenges of testing non-deterministic APIs and the potential benefits of allowing small bugs to provide touchpoints with customers. Ultimately, it is important for developers to consider the context, customer impact, and business goals when deciding on the level of testing necessary for a project.
The influence of context on testing
Context plays a crucial role in determining the importance and extent of testing in a development project. The conversation explores how different projects demand varying levels of testing, depending on factors like the type of product, the scale of the business, and the specific needs of customers. The speakers discuss the nuances of testing, considering the challenges of developing meaningful tests for rapidly changing front-end code, the unpredictability of certain APIs, and the varying complexities of different projects, including those involving AI algorithms. They emphasize the need for developers to understand and adapt to the specific context of their projects when determining the necessity of testing.
Automating tests with AI
The discussion touches on the potential of using AI, specifically GPT-like models, to automate the process of writing tests. The speakers share their experiences with AI-generated tests, which can provide a starting point for developers and help reduce the time and effort required to write tests from scratch. The conversation acknowledges the benefits of AI-driven testing in terms of speed and efficiency, but also recognizes the importance of human touch and judgment in refining and customizing the generated tests. They discuss the potential future impact of AI in generating tests and the implications it may have on the testing landscape.
The importance of business objectives
Throughout the conversation, the speakers emphasize the need to align testing practices with larger business objectives. They emphasize that what matters most is whether a product is meeting customers' needs and providing value, rather than adhering to strict testing standards. They explore how different factors, such as the type of product or the life expectancy of a project, may influence testing priorities. Ultimately, the discussion encourages developers to consider the broader context of their work, including business objectives, customer impact, and the potential trade-offs associated with testing, to make informed decisions about the level of testing required for a project.
Matt and Josh talk about automated testing and try to answer the question "do we need them?"
This season of YAGNI was made possible by our friends at Flipper Cloud - Are big launches stressing you out? Then you need feature flags! Flipper Cloud helps your team deploy the code now and then roll out features when you’re good and ready. Get started for free at flippercloud.io