Agile Data N’ Info

Raw and unedited Top 3 things: Get ahead of quarterly planning. Anna talked about the data team doing discovery before proposals reach the train meetings, so they arrive prepared rather than as surprise data requests. Prototype quickly, then govern what persists. Anna sees value in scrappy, source-connected prototypes, but not in leaving them as lasting, ungoverned dashboards. Keep AI agents behind explicit gates. The discussion highlighted agents missing small but critical instructions—including a child session bypassing Shane’s commit gate—and the need for tests, human checks, and coordination as parallel work scales.   https://agiledata.info/p/watch-now-recording-of-aura-anna-e3b?utm_source=youtube   <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you asked for.” Shane runs a one-day workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with his stakeholders, not a textbook version that falls apart on contact with messy data and vague stakeholder requirements. Latest dates: Gather data requirements in 30 minutes using a shared language.   Simply Magical Data Ways of Working

In this episode Shane Gibson sits down with data modeling veteran’s John Giles and Remco Broekmans The conversation explores the premise of democratising data modeling through AI, centered around a paper published by John regarding where AI can and cannot add real value. The group highlights the critical importance of conceptual and context data modeling, stressing that business terms and definitions must be established upfront before looking at physical source tables. They strongly critique the anti-pattern of placing humans “at the end of the line” merely to rubber-stamp automated outputs, arguing instead for an “AI-in-the-loop” approach where technology serves strictly as an assistant under human direction. The discussion also dives deep into complex real-world scenarios where automated tools fail—specifically when physical data or source systems do not yet exist. John shares extreme edge cases, such as establishing a brand-new government department from scratch based on a vague Act of Parliament, unifying over 70 distinct healthcare organisations into a single model, and modeling evolving visions like the Australian wool industry. Remco breaks down his multi-agent ELM framework, explaining how introducing deliberate manual friction points between LLM tasks keeps human modelers in control and why model-storming workshops, human trust, and organisational culture are far more vital than push-button automation. Whether you are curious about conceptual modeling techniques, how to effectively integrate “AI in the loop” while avoiding push-button modeling pitfalls, or strategies for aligning business definitions across complex organisations, this episode is packed with valuable insights for data professionals.   You can read the paper John published that spurred the conversation here: https://www.linkedin.com/feed/update/urn:li:activity:7511570544782356480/   If you want want to read a summary generated with GenAI, head over to: https://agiledata.substack.com/i/219573257/google-notebooklm-briefing   If you want to download the transcript for the podcast, head over to: https://agiledata.substack.com/i/219573257/transcript   Listen to more podcasts on applying Agile Data patterns over at https://podcast.agiledata.io/ Read more on the Agile Data Way of Working over at https://AgileDataGuides.com/   If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.   Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict |    <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you asked for.” Shane runs a one-day workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with his stakeholders, not a textbook version that falls apart on contact with messy data and vague stakeholder requirements. Latest dates: Gather data requirements in 30 minutes using a shared language.   Simply Magical Data Ways of Working  

Raw and unedited Top 3 things: How to evaluate new tools: Chris described a three-part approach: check whether a tool solves a real pain point, whether it has a community, and what a time-boxed trial reveals. He used Databricks serverless micro apps as an example of something worth revisiting later, rather than adopting now. How to reduce vendor and migration risk: We discussed keeping business logic and context outside execution platforms, retaining access to data, and testing a fallback plan—not just assuming a migration will work. AI-generated dashboards and their consequences: Faster creation could multiply duplicate reports. We discussed finding existing reports before making new ones, as well as the adoption, governance, and security challenges of switching BI tools.   https://agiledata.info/p/watch-now-recording-of-aura-chris-0f0?utm_source=youtube <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you asked for.” Shane runs a one-day workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with his stakeholders, not a textbook version that falls apart on contact with messy data and vague stakeholder requirements. Latest dates: Gather data requirements in 30 minutes using a shared language.   Simply Magical Data Ways of Working

Raw and unedited Top 3 things: Evaluating new technology: Shane and Tim discussed starting with a real use case, making small bets, checking risk tolerance, and planning how to replace a tool if its vendor changes course. Classification with Jive/Jev: Tim described using Gemini to extract image metadata, then Jive/Jev to make structured classification decisions at scale. Where data and BI are heading: They covered the OSI metrics standard, DuckDB and MotherDuck, and whether simpler, “disposable” BI apps will take off.   https://agiledata.info/p/watch-now-recording-of-aura-tim-frazer-384?utm_source=youtube   <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you asked for.” Shane runs a one-day workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with his stakeholders, not a textbook version that falls apart on contact with messy data and vague stakeholder requirements. Latest dates: Gather data requirements in 30 minutes using a shared language.   Simply Magical Data Ways of Working

Raw and unedited Top 3 things: Conference takeaways: Helsinki Data Week was nearly sold out. Juha said Big Data London felt less crowded than two years ago, with context, AI trust and governance, and data products prominent in the discussions. The missing enterprise context layer: Catalogs focused on tables and metrics are too narrow. He envisaged a shared reference layer that helps agents find relevant data, documents, processes and APIs—but said he had not seen it working in production yet. Making business meaning explicit: Definitions for the book and companion app, the distinction between a business glossary and a data dictionary, and why relationships such as “customer places order” and “customer cancels order” need their meaning captured.   https://agiledata.info/p/watch-now-recording-of-aura-juha-03f?utm_source=youtube   <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you asked for.” Shane runs a one-day workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with his stakeholders, not a textbook version that falls apart on contact with messy data and vague stakeholder requirements. Latest dates: Gather data requirements in 30 minutes using a shared language.   Simply Magical Data Ways of Working

Raw and unedited   Top 3 things: Personal OS / “Watson OS” a privacy-conscious household management system that filters emails, checks shared calendars, routes responsibilities, and sends daily/weekly digests rather than becoming another task manager. Context layers and shared organisational memory AI summaries may be more useful for copilots to retrieve on demand than for people to read; the broader challenge is consent-based creation, sharing, retrieval, and curation of context. Scaling innovation from individuals to the organisation power users need a path to turn useful experiments into trusted, governed products, while data teams may increasingly own mini-apps, write-back workflows, and source-system-like capabilities.   https://agiledata.info/p/watch-now-recording-of-aura-anna-2c8?utm_source=youtube   <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you asked for.” Shane runs a one-day workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with his stakeholders, not a textbook version that falls apart on contact with messy data and vague stakeholder requirements. Latest dates: Gather data requirements in 30 minutes using a shared language.   Simply Magical Data Ways of Working

Raw and unedited Top 3 things: How to migrate legacy data platforms safely audit first, choose greenfield vs. lift-and-shift early, define the target architecture, and validate with a thin-slice POC rather than a big-bang migration. Using AI in migration work, with human oversight LLMs can help translate code, create plans, surface outliers, and review alternatives, but edge cases, over-engineering, and validation mean people remain essential. Preserving business context and governance retain raw migration conversations, document why decisions were made, separate logic/context from the execution platform, and establish governance, cost monitoring, and FinOps from day one. https://agiledata.info/p/watch-now-recording-of-aura-chris-b41?utm_source=youtube   <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you asked for.” Shane runs a one-day workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with his stakeholders, not a textbook version that falls apart on contact with messy data and vague stakeholder requirements. Latest dates: Gather data requirements in 30 minutes using a shared language.   Simply Magical Data Ways of Working

Raw and unedited Top 3 things: Data platform architecture and orchestration whether agent meshes can replace traditional DAG-based orchestration, and when platform-native tools are sufficient. Choosing platforms vs. best-of-breed tools integration gaps, feature parity, and using explicit principles to decide whether to adopt, lease, or build. AI tooling, privacy, and context ownership custom AI harnesses, model cost/visibility problems, multiplayer workflows, and the risks of organisational and personal data being exposed or sold.   https://agiledata.info/p/watch-now-recording-of-aura-tim-frazer-98c?utm_source=youtube   <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you asked for.” Shane runs a one-day workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with his stakeholders, not a textbook version that falls apart on contact with messy data and vague stakeholder requirements. Latest dates: Gather data requirements in 30 minutes using a shared language.   Simply Magical Data Ways of Working

Raw and unedited   https://agiledata.info/p/watch-now-recording-of-aura-juha-a97?utm_source=youtube   <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you asked for.” Shane runs a one-day workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with his stakeholders, not a textbook version that falls apart on contact with messy data and vague stakeholder requirements. Latest dates: Gather data requirements in 30 minutes using a shared language.   Simply Magical Data Ways of Working

Raw and unedited   https://agiledata.info/p/watch-now-recording-of-aura-anna-ad5?utm_source=youtube   <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you asked for.” Shane runs a one-day workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with his stakeholders, not a textbook version that falls apart on contact with messy data and vague stakeholder requirements. Latest dates: Gather data requirements in 30 minutes using a shared language.   Simply Magical Data Ways of Working