Andrew Colombi, Co-founder of Tonic.ai, and Adam Kamor discuss creating realistic synthetic data, data de-identification, Tonic's subsetting engine, and more. They delve into challenges of obtaining valid test data for technology platforms and Tonic's mission to convert production data into secure testing data.
Read more
AI Summary
Highlights
AI Chapters
Episode notes
auto_awesome
Podcast summary created with Snipd AI
Quick takeaways
Accessing valid test data for sensitive information is a common challenge addressed by Tonic AI through synthetic data creation and de-identification processes.
Tonic AI's tool, Ephemeral, offers lightweight and cost-effective database solutions designed for quick setup and shutdown, catering to CI/CD automation needs.
Tonic Validate serves as a crucial testing tool for LLM-based applications, providing insights into application quality and performance improvements.
Deep dives
Challenges in Accessing High-Quality Test Data
Accessing valid and high-quality test data poses a challenge, especially for sensitive data. Tonic AI, founded in 2018, offers tools to transform production data into safe testing data. The focus is on realistic synthetic data creation, data de-identification, and the Tonic sub-setting engine.
Founding Stories of Tonic's Co-Founders
Adam and Andrew, co-founders of Tonic, share their paths to co-founding the company. Adam, Tonic's head of engineering, connected with CEO Ian from Tableau Software. Andrew, from Palantir, explored synthetic data projects before joining Tonic. Their backgrounds influenced the development of Tonic's core products.
Ephemeral: Tonic's Test Data Infrastructure Product
Ephemeral is Tonic's latest product focusing on test data infrastructure. Unlike production data infrastructure services, such as RDS, ephemeral provides lightweight and cost-effective database solutions for testing. Ephemeral databases are designed to be quick to spin up, shut down, and easily manageable with APIs, catering to CI/CD automation and individual developer needs.
Benefits of Tonic Ephemeral for CI/CD and Developer Ad Hoc Needs
Tonic Ephemeral offers benefits for CI/CD use cases by providing quick database setup for testing without unnecessary cash burn. It allows developers to quickly set up and shut down databases for ad hoc testing, ensuring efficiency and cost-effectiveness. The automation feature of ephemeral ensures that databases are only active when needed, avoiding unnecessary expenses and alignment with CI processes.
Tonic Validate: Evaluation Framework for LLM-backed Applications
Tonic Validate serves as a testing tool for LLM-based applications, evaluating response quality and providing scoring metrics for application performance. It integrates into CI/CD pipelines to monitor answer quality changes over time and allows for regression analysis based on response evaluations. This tool is crucial for maintaining and enhancing the performance of LLM-backed applications by offering insights into application quality and improvements.
All robust technology platforms require testing to ensure that features work as intended. In many cases, tests require data, but getting access to valid and high quality test data is a common challenge, especially when the technology runs on sensitive data. Realistically mimicking data that would normally contain sensitive financial or personal information is not easy.
Tonic.ai was started in 2018 to provide developer tools to transform production data into safe testing data. Andrew Colombi is the CTO and Adam Kamor is the Head of Engineering at Tonic. They join the show to talk about creating realistic synthetic data, data de-identification, validating LLM RAG output, Tonic’s subsetting engine, and much more.
Full Disclosure: This episode is sponsored by Tonic.
Gregor Vand is a security-focused technologist, and is the founder and CTO of Mailpass. Previously, Gregor was a CTO across cybersecurity, cyber insurance and general software engineering companies. He has been based in Asia Pacific for almost a decade and can be found via his profile at vand.hk.