SurveyMonkey’s Jing Huang on the Hidden Flaw in Synthetic Data for Enterprise AI Training

As enterprises race to integrate generative AI, SurveyMonkey is taking a uniquely methodical approach: applying 20 years of survey methodology to enhance LLM capabilities beyond generic implementations. In this episode, Jing Huang, VP of Engineering & AI/ML/Personalization at SurveyMonkey, breaks down how her team evaluates AI opportunities through the lens of domain expertise, sharing a framework for distinguishing between market hype and genuine transformation potential. 

Drawing from her experience witnessing the rise of deep learning since AlexNet's breakthrough in 2012, Jing provides a strategic framework for evaluating AI initiatives and emphasizes the critical role of human participation in shaping AI's evolution. The conversation offers unique insights into how enterprise leaders can thoughtfully approach AI adoption while maintaining competitive advantage through domain expertise.

Topics discussed:

• How SurveyMonkey evaluated generative AI opportunities, choosing to focus on survey generation over content creation by applying their domain expertise to enhance LLM capabilities beyond what generic models could provide.
• The distinction between internal and product-focused AI implementations in enterprise, with internal operations benefiting from plug-and-play solutions while product integration requires deeper infrastructure investment.
• A strategic framework for modernizing technical infrastructure before AI adoption, including specific prerequisites for scalable data systems, MLOps capabilities, and real-time processing requirements.
• The transformation of survey creation from a months-long process to minutes through AI, while maintaining methodological rigor by embedding 20+ years of survey expertise into the generation process.
• The critical importance of quality human input data over quantity in AI development, with insights on why synthetic data and machine-generated content may not be the solution to current data limitations.
• How to evaluate new AI technologies through the lens of domain fit and implementation readiness rather than market hype, illustrated through SurveyMonkey's systematic assessment process.
• The role of human participation in shaping AI evolution, with specific recommendations for how organizations can contribute meaningful data to improve AI systems rather than just consuming them.