IoT has created a tidal wave that data savvy organizations can turn into profitable business solutions. Most IoT data comes from sensors, which are now attached to almost every device imaginable, from factory floor machines and agricultural fields to your cell phone and toothbrush. But IoT is forcing companies to rethink their data architectures to ingest, process, and analyze streaming data in real-time. To help us understand the impact of IoT on data architectures, we invited Dan Graham to our show for a second time. Dan is a former product marketing manager at both IBM and Teradata, renowned for combining deep technical knowledge with industry marketing savvy. During his tenure at those companies, he was responsible for MPP data management systems, data warehouses, and data lakes, and most recently, the Internet of Things.

Just-in-time design is the practice of designing working software in small increments that support a business-defined need or story. Just-in-time design, as well as just-in-time testing, is an integral part of the agile software methodology. In fact, you can’t really do agile without just-in-time design. To help us understand the nuances of just-in-time design, we invited Aaron Fuller, a long-time data architect and member of Eckerson Group’s consulting network. Across an 11-year career as the enterprise data architect for an insurance company, he modeled data, created technical designs for a broad range of systems, established governance and stewardship, and led the establishment of their enterprise data warehousing, business intelligence, and enterprise architecture programs. As principal consultant and owner of Superior Data Strategies since 2010, he leads a team of highly skilled data professionals who are uniquely capable of planning and executing agile data projects.

Last month, I attended Domo’s annual user conference for the first time. I came a skeptic, but left a believer. Domo has invested large sums of money to create a comprehensive data and analytics platform that scales to run small and medium-size businesses, and possibly large ones. Most importantly, it has a cadre of highly satisfied brand-name customers who want to extend the platform to support all business users and their analytic applications. Originally published at: https://www.eckerson.com/articles/catching-the-domo-spirit

Processing continuous data streams is becoming increasingly important. However, traditional analytics architectures were often not built for real-time scenarios. This article will illustrate challenges and discuss how streaming-first approaches can change the way we think about analytics architectures. Originally published at: https://www.eckerson.com/articles/streams-everywhere-towards-streaming-first-architectures

This second article in a series on modern data architectures. It focuses on what drives customers to want a modern data architecture (i.e., fear and opportunity) in the first place. It then lists ten requirements that customers desire for a modern data architecture, ranging from “cloud-first” and “streaming-first” to “best of breed” and “predictable pricing”. Originally published at: https://www.eckerson.com/articles/ten-things-companies-want-from-a-modern-data-architecture

Data virtualization has been around for decades and has always been controversial. In the 1990s, it was called virtual data warehousing or VDW-- or as some skeptics liked to say, "voodoo and witchcraft”. It’s also been known as query federation and more recently, data services. The idea is that business users don't need to know the location of the data; they merely need to log into the data service and all data appears as if it’s local to their server, modeled in a fashion that makes sense to them. Andrew Sohn is the Global Head of Data and Analytics at Crawford & Company, a $1B+ service provider to the insurance and risk management industry, where he designed and leads its data and digital transformation strategy and program. With more than 25 years in the industry, Andrew has managed a broad range of infrastructure and application technologies. He’s a strong advocate of data virtualization technology and believes it is an integral part of a modern, agile data ecosystem.

Why is Data Quality still an issue after all these years? To get an answer to the prevalent question, Wayne Eckerson and Jason Beard engage in a dynamic exchange of questions which lead us to the root cause of data quality and data governance problems. Using examples from his past projects, Jason shows the value of business process mapping and how it exposes the hidden problems which go undetected under the standard IT lens. In his most recent role as Vice President of Process & Data Management at Wiley, a book publisher, he was responsible for master data setup and governance, process optimization, business continuity planning, and change management for new and emerging business models. Jason has led business intelligence, data governance, master data management, Process Improvement, Business Transformation, and ERP projects in a variety of industries, including Scientific and Trade publishing, Educational Technology, Consumer Goods, Banking, Investments, and Insurance.

Being a change agent is hard. It's tough to inspire people and get them motivated to work on a shared vision. To understand the mechanics of digitalization and tactics required to implement them, Wayne Eckerson invited Andrea Ballinger so that she could share her hard-won lessons from her illustrious career as a technology leader.   Andrea is currently leading a transformation program at LSU, revamping the university’s information technology resources across multiple campuses. Prior to that, she served as Interim CEO and President for the University of Illinois  Alumni Association and CTO of Illinois State University. She began her data career at the University of Illinois where she earned a reputation as the foremost data warehousing expert in higher education.

The road to AI adoption is far more complex than one can imagine. Building data science models and testing them is only one piece of the puzzle. To understand the roadblocks and best practices, Wayne Eckerson invited Nir Kaldero in our latest episode to learn why organizations need to start paying more attention to people, culture and processes to make data science projects a success and how democratization skills pays off in the long run. Nir Kaldero is the Head of Data Science, Vice President at Galvanize Inc. and the creator of the GalvanizeU Master’s of Science in Data Science program. A tireless advocate for transforming education and reshaping the field of data science, his vision and mission is to make an impact on a wide variety of communities through education, science, and technology. In addition to his work at some of the world’s largest international corporations, Kaldero serves as a Google expert/mentor and has been named an IBM Analytics Champion 2017 & 2018, a prestigious honor given to leaders in the field of science, technology, engineering, and math (STEM).

In this episode, Daniel Graham dissects the capabilities of data lakes and compares it to data warehouses. He talks about the primary use cases of data lakes and how they are vital for big data ecosystems. He then goes on to explain the role of data warehouses which are still responsible for timely and accurate data but don't have a central role anymore. In the end, both Wayne Eckerson and Dan Graham settle on a common definition for modern data architectures. Daniel Graham has more than 30 years in IT, consulting, research, and product marketing, with almost 30 years at leading database management companies. Dan was a Strategy Director in IBM’s Global BI Solutions division and General Manager of Teradata’s high-end server divisions. During his tenure as a product marketer, Dan has been responsible for MPP data management systems, data warehouses, and data lakes, and most recently, the Internet of Things and streaming systems.