Transition from Data Warehouses to Data Lakes: An Analytical Evolution

From the silver age of on-prem software companies like SAP and Siebel Systems to the golden age of enterprise software-as-a-service, we're now seeing an explosion of data. All types, all sizes, and all over the place. And much of it is a sort of industrial "data exhaust", where companies aren't quite sure what question to ask of the data but are being bombarded with data due to the variety of data sources available today -- from websites to sensors (and therefore data capture) everywhere. Before there is even a signal in the noise.

So how do you solve a problem like this-Data? Beyond requiring new types of plumbing and integrations, enterprises now expect -- given the age of mobile, web, cloud, and heck, let's add millennials to the mix too -- self service. To be able to ask, get, fit (curve-fit), predict. To take back the enterprise from the patchwork of integration and number of vendors we all have to deal with -- the scope of which most companies in fact are not truly aware of.

It's about the lifecycle of data in the enterprise, argues Snaplogic founder and CEO Gaurav Dhillon in this episode of the a16z Podcast, in conversation with Scott Kupor. It's in fact about the evolution of data overall -- from data warehouses to "data lakes": in stages, from purification (like wrangling data) to bottling (prepping for consumption by data scientists) to making sense of streams and streams of data!

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments and certain publicly traded cryptocurrencies/ digital assets for which the issuer has not provided permission for a16z to disclose publicly) is available at https://a16z.com/investments/.

Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.