Data Archives - Software Engineering Daily

The internet is a layer cake of technologies and protocols.  At a fundamental level, the internet runs on the TCP/IP protocol.  It’s a packet based system.  When your browser requests a file from a web server, that server chops up the file into tiny pieces known as packets and puts them on the network labeled with your machine’s address as its destination. That system works incredibly well for receiving a file from a web server.  If some of the packets arrive out of order, that’s not a problem.  If one is lost, it can be sent again. There are no guarantees in a packet based system.  If a flood of new packets show up, the system can slow down and you may experience a lag in the response time.  This can be annoying when visiting a blog that is slow to load, but it’s not a ruined experience.  Streaming video, on the other hand, does not degrade elegantly in this situation.  No consumer wants to have their experience interrupted by a spinning wheel.  Traffic can be spiky and unpredictable, especially around live events.  The stakes are high for building a scalable, effective streaming video solution. Amit Mishra is a member team at FOX which is responsible for building platform-to-live stream content across all the FOX properties.  In this episode we discuss some of the technical milestones on delivering this platform and why golang was the right choice. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Scalable Streaming Video with Amit Mishra appeared first on Software Engineering Daily.

It does not matter if it runs on your machine.  Your code must run in the production environment and it must do so performantly.  For that, you need tooling to better understand your application’s behavior under different circumstances.  In the earliest days of software development, all we had were logs, which are still around and incredibly useful.  You’re likely to also consider Application Performance Monitoring tools or APM.  Observability is an evolving and important feature of any software system. In this episode, we interview Christine Yen, co-founder, and CEO of Honeycomb.io.  We talk about getting her start in software, meeting her co-founder while working at Parse, and how some of the experiences shared there shaped their vision for an idealized tool for figuring out what’s happening with your system. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Observability Using Honeycomb.io with Christine Yen appeared first on Software Engineering Daily.

The manner in which users interact with technology has rapidly switched to mobile consumption.  The devices almost all of us carry with us at all times open endless opportunities for developers to create location-based experiences. Foursquare became a household name when the introduced social check-ins.  Today they’re a location data platform.  Ankit Patel is the SVP of Engineering at Foursquare.  In this episode, we discuss Foursquare’s services, the way customers leverage location intelligence, and opportunities for areas like augmented reality. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Location-Based Experiences Using Foursquare with Ankit Patel appeared first on Software Engineering Daily.

Modern business applications are complex.  It’s not enough to have raw logs or some basic telemetry.  Today’s enterprise organizations require an application performance monitoring solution or APM.  Today’s applications are complex distributed systems whose performance depends on a wide variety of factors.  Every single line of code can affect production and teams need insights into the health of the system and how to improve them. In this episode, I interview Omri Sass and Hugo Kaczmarek from Datadog, a provider of cloud monitoring as a service.  We discuss the APM space and the challenges faced by modern enterprise teams.  We also get into some detail about their new live search option and why that solution was an important addition to their product suite. Full disclosure: Datadog is a sponsor of Software Engineering Daily. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Datadog with Omri Sass and Hugo Kaczmarek appeared first on Software Engineering Daily.

Infrastructure as Code is an approach to machine provisioning and setup in which a programmer describes the underlying services they need for their projects.  However, this infrastructure code doesn’t compile a binary artifact like traditional source code.  The successful completion of running the code signals that the servers and other components described in the configuration file have been created automatically by the tools being used. In this episode, I speak with Christian Tragesser, DevOps Consultant with World Wide Technology.  We discuss the tools and processes that are helping modern development groups solve technical problems. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Infrastructure as Code with Christian Tragesser appeared first on Software Engineering Daily.

The first industrial deployments of machine learning and artificial intelligence solutions were bespoke by definition and often had brittle operating characteristics.  Almost no one builds custom databases, web servers, or email clients.  Yet technology groups today often consider developing homegrown ML and data solutions in order to solve their unique use cases.  Today’s modern data stack is a patchwork of interconnecting tools built to suit a variety of personas that need to interact with the data in notably different ways. In this episode, I speak with Leigh Marie Braswell, an Investor with Founders Fund.  We have a wide-ranging discussion about the technology landscape of data and machine learning solutions and the modern enterprise data stack. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Modern Data Infrastructure and Tools with Leigh Marie Braswell appeared first on Software Engineering Daily.

In a version control system, a Monorepo is a version control management strategy in which all your code is contained in one potentially large but complete repository.  The monorepo is in stark contrast to an alternative approach in which software teams independently manage microservices or deliver software as libraries to be imported in other projects.  The monorepo strategy has been followed by noteworthy companies such as Google, Facebook, and Microsoft. Derrick Stolee is a software engineer working at GitHub.  He joins us today to talk about strategies for monorepos and innovations to the way git works in order to better support this style of the repository. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Git Scales for Monorepos with Derrick Stolee appeared first on Software Engineering Daily.

Companies that gather data about their users have an ethical obligation and legal responsibility to protect the personally identifiable information in their dataset.  Ideally, developers working on a software application wouldn’t need access to production data. Yet without high-quality example data, many technology groups stumble on avoidable problems.  Organizations need a solution to protect privacy while simultaneously preserving aspects of the data which are important. Tonic is automating data synthesis to advance data privacy.  Their solution gives your production-like data for development and analytical purposes without compromising on data quality or privacy.  In this episode, I interview Tonic’s CEO Ian Coe, and Head of Engineering Adam Kamor. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Faking Data Using Tonic.ai with Ian Coe and Adam Kamor appeared first on Software Engineering Daily.

Applications write data to persistent storage like a database.  The most popular database query language is SQL which has many similar dialects.  SQL is expressive and powerful for describing what data you want.  What you do with that data requires a solution in the form of a data pipeline.  Ideally, these analytical workflows can follow similar best practices to those handled in application code. DBT is a transformation workflow that lets teams deploy analytics code following software engineering best practices like modularity, portability, CI/CD, and documentation. Users who know SQL can build production-grade data pipelines.  In this episode, I interview Tristan Handy, CEO, and founder of DBT Labs. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post DBT: Data Build Tool with Tristan Handy appeared first on Software Engineering Daily.

Tedious, repetitive tasks are better handled by machines.  Unless these tasks truly require human intelligence, repetitive tasks are often good candidates for automation.  Implementing process automation can be challenging and technical.  Increasingly, engineers are seeking out tools and platforms to facilitate faster, more reliable automation. In this episode I talk to Yaseer Sheriff, Co-Founder and CEO at Axiom about no-code solutions, process automation, and some of the challenges in developing the software powering those services. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post No Code Process Automation at Axiom with Yaseer Sheriff appeared first on Software Engineering Daily.