Cloud Engineering Archives - Software Engineering Daily

Running applications in containerized environments involves regularly organizing, adding and replacing containers. This complex job may involve managing clusters of containers in different geographic locations with different configuration requirements. Platforms like Kubernetes are great for managing this complexity, but include steep learning curves to efficiently get anything off the ground. The company Portainer provides a universal container management tool that works with Kubernetes, Docker, Docker Swarm, and Azure ACI. It enables managing containers without knowing platform-specific code and best practices. Instead, deploying containerized applications is done through a simple Graphical User Interface. Once deployed, you can observe and monitor the apps and govern security settings all through Portainer.  In this episode, we talk to Neil Cresswell, CEO and Co-Founder of Portainer.io, based in Auckland New Zealand. Neil is a career consultant, having spent 25 years in the roles of systems engineering, IT consulting, and more recently IT management. Neil started his career with 12 years at IBM as his foundation, where he had a leading role in Server, Storage, and Virtualisation systems engineering. Portainer has now grown into an entity of its own right, and using Neil’s knowledge of the CIO’s requirements, knowledge of the technology, and knowledge of infrastructure management, Neil has crafted Portainer into a strong contender in the container management space. Neil now runs the Portainer.io business and is responsible for the overall product strategy and the team entrusted to deliver on that strategy. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Portainer: Container Management with Neil Cresswell appeared first on Software Engineering Daily.

Apache Spark is a popular open source analytics engine for large-scale data processing. Applications can be written in Java, Scala, Python, R, and SQL. These applications have flexible options to run on like Kubernetes or in the cloud.  The company Data Mechanics is a cloud-native Spark platform for data engineers. It runs continuously optimized Apache Spark workloads on a managed Kubernetes cluster within the user’s cloud account. They boast a 50%-75% cost reduction from cloud providers by dynamically scaling applications based on load and automatically tuning app configurations based on the historical Spark pipeline runs. Their Kubernetes clusters are deployed within user accounts so user data never leaves the environment and they handle the cluster management.  In this episode we talk to Jean-Yves Stephan, Co-Founder and CEO at Data Mechanics. Jean-Yves previously worked as a Software Engineer then a Tech Lead Manager at Databricks. We discuss big data engineering in Spark and the unique advantages of using Data Mechanics to make Spark development easier and more cost effective. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Data Mechanics: Data Engineering with Jean-Yves Stephan appeared first on Software Engineering Daily.

Microservice architecture has become very common over the past few years because of the availability of containers and container orchestrators like Kubernetes. While containers are overall positive for scaling apps and making them more available, they’ve also introduced hurdles like persisting data and state, and container restarts or pod failures. Development teams put significant work into designing applications that take these hurdles into account because without precautions you can lose valuable data or crash your app.  The company Temporal provides tools for both building complex microservices as well as for apps that use microservices. They use 2 primary function types: workflow and activity. Workflow functions persist all local variables and threads so that if the server the app runs on crashes, it’s picked up on a different server where it left off, down to the line. Activity functions automatically initiate retry logic if the service the function invokes fails for something like its server being down. Temporal provides visibility into end-to-end workflows that can span multiple services.  In this episode we talk to Ryland Goldstein, Head of Product at Temporal. Previously, Ryland was the Lead Product Manager at Reshuffle and a Software Engineer Lead at Parallel Machines. We discuss the challenges of managing state in micro services, orchestrating microservices and how Temporal simplifies this process for development teams. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Temporal Product: Managing State with Ryland Goldstein appeared first on Software Engineering Daily.

Cloud computing provides tools, storage, servers, and software products through the internet. Securing these resources is a constant process for companies deploying new code to their cloud environments. It’s easy to overlook security flaws because company applications are very complex and many people work together to develop them. Wyze Labs, for example, had millions of users’ data stolen due to a mistake by a single employee. The company Bridgecrew is a cloud security platform helping to prevent mistakes like that from happening. Bridgecrew integrates into developer workloads to automatically find infrastructure errors in cloud accounts, workloads, and infrastructure as code. Their platform also monitors code reviews and build pipelines to prevent errors from being deployed into production. If an error is found then Bridgecrew’s software reverts that code back to its last known correct state.  In today’s episode we talk with Barak Schoster, CTO and co-founder at Bridgecrew. Barak previously worked as a senior software architect at RSA Security and as a software architect at Fortscale before that. We discuss cloud security, Infrastructure as Code, and big data architecture. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Bridgecrew: Cloud Security with Barak Schoster appeared first on Software Engineering Daily.

Large portions of software development budgets are dedicated for testing code. A new component may take weeks to thoroughly test, and even then mistakes happen. If you consider software defects as security issues then the concern goes well beyond an application temporarily crashing. Although even minor bugs can cost companies a lot of time to locate the bug, resolve it, retest it in lower environments, then deploy it back to production.  The company Speedscale provides an intelligent, Kubernetes-friendly testing toolkit that runs at build time. Their virtual SRE-bot works inside automated release pipelines to forecast and test real-world conditions the new code will encounter. This process requires no manual scripting because Speedscale uses existing traffic to generate tests and mocks. The feedback is immediate after every build and covers regression, performance, fuzzing, and chaos tests automatically.  In this episode we talk with Ken Ahrens and Matt LeRay. Ken is a founder and the CEO at Speedscale. Previously Ken worked at New Relic as a senior director, solutions architects. Matt is a founder and CTO at Speedscale. He previously was the VP of Product at Observe, inc. We discuss testing in distributed environments, how Speedscale intelligently tests and mocks during builds, Kubernetes, and their future goals with Speedscale. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Speedscale: Automated Testing with Ken Ahrens and Matt LeRay appeared first on Software Engineering Daily.

IT infrastructure are the components required to operate IT environments, like networks, virtual machines or containers, an operating system, hardware, data storage, etc…. As companies build out different deployment environments with infrastructure configurations, they must maintain the different environments, replicate them, and update them. The management of infrastructure, often automated to some extent, is referred to as Infrastructure as Code (IaC).   The company Gold Fig Labs helps growing companies better understand their deployed infrastructure beyond the basic IaC principles. Gold Fig Labs developed 2 main tools to deliver the clearest view of infrastructure security and compliance. Their tool Checkup provides periodic security and best practices reports for AWS accounts. The report details specific, actionable, and relevant advice to improve security posture. Their other tool, Introspector, is a unique security and auditing tool that provides in-depth analysis of larger cloud deployments with complex regulatory requirements and custom internal policies.  In this episode we talk with Vikrum Nijjar and Greg Soltis. Vikrum is co-founder and CEO at Gold Fig Labs. He was previously an angel investor with Angel and special advisor for Google’s onboarding acquisitions, compliance, and security.  Greg is co-founder and CTO at Gold Fig Labs. Previously he was a senior software engineer at Google and a software engineer at Firebase. We discuss the principles of IaC, how Gold Fig Labs helps customers go beyond what IaC intends, and the complexity of cloud infrastructure security and regulatory compliance. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Gold Fig Labs: Cloud Infrastructure Security with Vikrum Nijjar and Greg Soltis appeared first on Software Engineering Daily.

Software-Defined Networking describes a category of technologies that separate the networking control plane from the forwarding plane. This enables more automated provisioning and policy-based management of network resources. Implementing software-defined networking is often the task of Site Reliability Engineers, or SREs. Site reliability engineers work at the intersection of development and operations by bringing software development practices to system administration.  Equinix manages co-location data centers and provides networking, security, and cloud-related services to their clients. Equinix is leveraging its status as a market leader in on-prem networking capabilities to expand into cloud and IaaS offerings such as Equinix Metal, which has been referred to as “bare-metal-as-a-service,” and offers integrations with 3rd party cloud technologies with a goal of creating a seamless alternative to modern public clouds for organizations seeking the benefits of colocation. Tim Banks is a Principal Solutions Architect at Equinix and he joins the show to talk about what Equinix offers and how it differs from other cloud providers. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Equinix Infrastructure with Tim Banks appeared first on Software Engineering Daily.

Google Cloud, AWS, and Azure are the dominant cloud providers on the market today. But the market is still highly competitive, and there is significant overlap in the services offered by all three large providers. Since all three offer a broad range of services, developers looking to choose a platform for their application must focus on providers’ domains of relative excellence and how those align with their needs. One domain where Google Cloud Platform excels is with its database offerings. Google has data management baked into its organizational DNA, and has been the source of several innovative technologies in the data space such as Spanner, BigTable, and BigQuery.  Andi Gutmans is a general manager and VP of engineering for databases at Google. He joins the show today to talk about how Google came to excel at databases and data management, how machine learning and Big Data users in particular can benefit from Google Cloud’s offerings, and how new features such as Database Migration Service are helping Google stay ahead of the curve in a competitive cloud landscape. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Google Cloud Databases with Andi Gutmans appeared first on Software Engineering Daily.

AWS offers over 200 services as part of its IaaS platform, and that number continues to grow. Organizing all of these services, and tracking the costs they incur, can be a significant challenge, often requiring teams of AWS-certified sysadmins working together to get a handle on an enterprise-scale system.  Vantage provides an alternative, streamlined AWS console that makes it easier to manage AWS services and track associated costs. Users link their AWS account to Vantage, and it automatically profiles all their services and aggregates the information into a dashboard. Users can customize how their Vantage console appears and allows users to break down service usage by region. Ben Schaechter is the co-founder of Vantage. Before founding Vantage he was a Senior Product Manager at AWS and DigitalOcean. Ben joins the show today to talk about how Vantage helps streamline the AWS experience, and why teams of all sizes can benefit from a better user experience on the AWS platform. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Vantage: AWS Console Alternative with Ben Schaechter appeared first on Software Engineering Daily.

Build automation tools automate the process of building code, including steps such as compiling, packaging binary code, and running automated tests. Because of this, build automation tools are considered a key part of a continuous delivery pipeline. Build automation tools read build scripts to define how they should perform a build. Common build scripts include Makefile, Dockerfile, and bash.  Earthly is a build automation tool that allows you to execute all your builds in containers. Earthly uses Earthfiles, which draws from the best features of Makefile and Dockerfile and provides a common layer between language-specific tooling and the CI build spec. Earthly builds are repeatable, isolated, and self-contained, and will run the same way across different environments such as a CI system or a developer’s laptop.  Vlad Ionescu is the Founder and CEO of Earthly Technologies. He was formerly the founder and chief architect at ShiftLeft.io. Vlad joins the show today to talk about why reproducible builds are important, how Earthly simplifies build scripts, and what the long-term vision for Earthly looks like. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Earthly: Build Automation with Vlad Ionescu appeared first on Software Engineering Daily.