Data Archives - Software Engineering Daily

A developer’s core deliverables are individual commits and the pull requests they aggregate into. While the number of lines of code written alone may not be very informative, in total, the code and metadata about the code found in tracking systems present a rich dataset with great promise for analysis and productivity optimization insights. LinearB is a systematic approach to engineering improvement. Their WorkerB Slack Bot connects with teams on an individual level to help with productivity and collaboration. In this episode, I speak with Dan Lines, Co-Founder, and COO. Sponsorship inquiries: sponsor@softwareengineeringdaily.com Show Notes: LinearB.io Learn more about WorkerB Check out the Dev Interrupted Podcast Save your spot for the INTERACT conference The post LinearB with Dan Lines appeared first on Software Engineering Daily.

Modern companies leverage dozens or even hundreds of software solutions to solve specific needs of the business.  Organizations need to collect all these disparate data sources into a data warehouse in order to add value.  The raw data typically needs transformation before it can be analyzed.  In many cases, companies develop homegrown solutions, thus reinventing the wheel and possibly planting deep rooted seeds of technical debt. Mozart Data helps you collect all of your data sources in under an hour.  They provide managed data pipelines, data warehousing, and transformation automation.  In this episode, I interview CEO Peter Fishman and CTO Dan Silberman about the modern data stack. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Modern Data Stacks Optimized by Mozart Data with Peter Fishman and Dan Silberman appeared first on Software Engineering Daily.

Instabase is a technology platform for building automation solutions. Users deploy it onto their own infrastructure and can leverage the tools offered by the platform to build complex workflows for handling tasks like income verification and claims processing. In this episode we interview Anant Bhardwaj, founder of Instabase. He describes Instabase as an operating system.  We explore what he means by that and discuss the types of use cases Instabase powers. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Instabase with Anant Bhardwaj appeared first on Software Engineering Daily.

Time series data are simply measurements or events that are tracked, monitored, downsampled, and aggregated over time. This could be server metrics, application performance monitoring, network data, sensor data, events, clicks, trades in a market, and many other types of analytics data (influxdata.com). The platform InfluxData is designed for building and operating time series applications. InfluxData is engineered for growth with enterprise-grade security, ingests metrics, events and logs in a high-performing time series database, and platform analytics for detecting and resolving problems. In this episode we talk to Russ Savage, Director of Product Management at InfluxData. Full disclosure: InfluxData is a sponsor of Software Engineering Daily. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post InfluxData: Time-Series Data with Russ Savage appeared first on Software Engineering Daily.

Whether sending messages, shopping in an app, or watching videos, modern consumers expect information and responsiveness to be near-instant in their apps and devices. From a developer’s perspective, this means clean code and a fast database.  Apache Druid is a database built to power real-time analytic workloads for event-driven data, like user-facing applications, streaming, and anything else that requires instant data visibility. Druid offers lower latency for OLAP-style queries, time-based partitioning, fast search and filter, and out-of-the-box integration with Apache Kafka, AWS Kinesis, HDFS, AWS S3, and more (Druid.apache.org). In this episode we talk with Eric Tschetter, Field CTO at Imply, Fellow at Splunk, and experience developing large swaths of backend infrastructure largely focusing on Druid. We discuss the use cases and power of Apache Druid. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Druid: Event-Driven Data with Eric Tschetter appeared first on Software Engineering Daily.

Auren Hoffman is the CEO of SafeGraph. In this episode we discuss data as a service and more. This interview was also recorded as a video podcast. Check out the video on the Software Daily YouTube channel. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post DaaS with Auren Hoffman appeared first on Software Engineering Daily.

Enterprise data warehouses store all company data in a single place to be accessed, queried, and analyzed. They’re essential for business operations because they support managing data from multiple sources, providing context, and have built-in analytics tools. While keeping a single source of truth is important, easily moving data from the warehouse to other applications is invaluable. The company Hightouch provides tools that easily move data from your warehouse to important business tools like Salesforce, Apache Airflow, Tableau and more. Hightouch uses SQL to move and transform your data and tracks every row to avoid moving data that hasn’t changed. Failed rows are retried, and all changes to rows are logged.  In this episode, we talk to Tejas Manohar, Founder of Hightouch. We talk about reverse ETL, managing data across multiple systems, and how Hightouch helps companies operationalize their data warehouse. Sponsorship inquiries: sponsor@softwareengineeringdaily.com   The post Reverse ETL: Operationalizing Data Warehouses with Tejas Manohar appeared first on Software Engineering Daily.

Prophecy is a complete Low-Code Data Engineering Platform for the Enterprise. Prophecy enables all your teams on Apache Spark with a unique low-code designer. While you visually build your Dataflows – Prophecy generates high-quality Spark code on Git. Then, you can schedule Spark workflows with Prophecy’s low-code Airflow. Not only that, Prophecy provides end-to-end visibility into your dataflows with Metadata Search and Column Level Lineage.  For Enterprises, in addition to developing new workflows, data teams also need to migrate thousands of old proprietary ETL workflows to the cloud. For that, Prophecy has built a Transpiler that automatically converts AbInitio, Informatica, SSIS and Alteryx workflows to high-quality Spark code. Learn more at www.prophecy.io. In this episode, we speak with Raj Bains, who is the founder & CEO of Prophecy. Previously, Raj was the product manager of Apache Hive at Hortonworks through the IPO. He also headed product management and marketing for a NewSQL database startup.  Raj continues to actively code in compiler and database technologies. His engineering roles include developing a NewSQL database, building CUDA at NVIDIA as a founding engineer, and working as a compiler engineer for Microsoft Visual Studio. Full disclosure: Prophecy is a sponsor of Software Engineering Daily. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Prophecy: Apple of Data Engineering with Raj Bains appeared first on Software Engineering Daily.

In the previous episode, Pulsar Revisited, we discussed how the company DataStax has added to their product stack Astra Streaming, their cloud-native messaging and event streaming service that’s built on top of Apache Pulsar. We discussed Apache Pulsar and the added features DataStax offers like injecting machine learning into your data streams and viewing real-time analytics. In today’s episode we’re going to continue this conversation with Enrico Olivelli, a Senior software engineer at DataStax and an ASF Member with The Apache Software Foundation. Apache Pulsar has released a number of exciting upgrades and enhancements in their recent 2.8.0 release. How will these changes affect Astra Streaming and what can users look forward to in future Astra Streaming releases? Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Pulsar Rerevisted with Enrico Olivelli appeared first on Software Engineering Daily.

In 2003, Google developed a robust cluster management system called Borg. This enabled them to manage clusters with tens of thousands of machines, moving them away from virtual machines and firmly into container management. Then, in 2014, they open sourced a version of Borg called Kubernetes, or K8s.  Now, in 2021, CockroachDB is a distributed database designed with Kubernetes architecture in mind. CockroachDB uses regular SQL and scales by automatically distributing data and workload demands. Their databases survive machine, datacenter, and region failures, and provide guaranteed ACID compliant transactions.   In this episode, we talk to Spencer Kimball, CEO at Cockroach Labs, about distributed databases and containerization. Full disclosure: Cockroach Labs is a sponsor of Software Engineering Daily. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post CockroachDB: Distributed Databases and Containerization with Spencer Kimball appeared first on Software Engineering Daily.