Brandon Jacobs, Infrastructure architect at Coreweave, discusses how Coreweave uses Kubernetes to build an AI hyperscaler. They cover managing Day 0 & 2 operations for AI labs, lessons learned, and best practices for a Kubernetes based cloud. Topics include leveraging bare metal Kubernetes for GPU workloads, storage options for AI labs, observability, monitoring, handling CVEs, and customer cluster support.

Andy Grimes discusses OpenShift's AI Landscape with insights on MLOps, AI model development, and accelerating model deployment using OpenShift AI. The discussion covers local experimentation, governance tools for AI, and the collaboration between IBM and Red Hat in the AI space.

Bernie Wu from Memverge discusses training ML models on Kubernetes, including cost-saving tips with spot instances, efficient model checkpoints, hot restarts, and reclaiming GPU resources. They delve into topics like DAG phases, transparent checkpointing, and GPU snapshotting for AI workloads.

Christian Posta, VP and Global Field CTO at Solo.io, discusses the evolution of service mesh technologies from Linkerd to istio implementations, connecting application components outside Kubernetes. They explore shared responsibilities between developers and platform engineers, using internal developer platforms for service mesh. Topics include on-prem and cloud flexibility, challenges in cloud-based development, and importance of automation and observability in infrastructure architectures.

In this episode of the Kubernetes Bytes podcast, Ryan and Bhavin talk to Torsten Steinbach - VP, Chief Architect for Analytics & AI at EDB about all things Vector Databases, Postgres, and why Data is important for building AI platforms. The discussion dives into how vector databases are different than relational databases and why using Postgres extensions helps organizations use their existing data for AI applications.  Check out our website at https://kubernetesbytes.com/  Kubernetes Community Days (KCD) in New York City on May 22nd, use the promo code “KUBERNETESBYTES” to get a 10% discount on your registration fees!   Episode Sponsor: Nethopper  Learn more about KAOPS:  @nethopper.io  For a supported-demo:  info@nethopper.io  Try the free version of KAOPS now!    https://mynethopper.com/auth Cloud Native News:   https://www.reuters.com/markets/deals/ibm-nearing-buyout-deal-hashicorp-wsj-reports-2024-04-23/ https://www.wiz.io/blog/wiz-acquires-gem-security-to-reinvent-threat-detection-in-the-cloud https://techcrunch.com/2024/04/18/wiz-is-in-talks-to-buy-lacework-for-150-200m-security-firm-was-last-valued-at-8-3b/ https://www.prnewswire.com/news-releases/coreweave-secures-1-1-billion-in-series-c-funding-to-drive-the-next-generation-of-cloud-computing-for-the-future-of-ai-302133328.html https://kubernetes.io/blog/2024/04/17/kubernetes-v1-30-release  https://dok.community/blog/become-a-data-on-kubernetes-in-2024-ambassador/   Show Links:  https://www.enterprisedb.com/news/edb-acquires-splitgraph  https://www.enterprisedb.com/resources/events https://www.enterprisedb.com/  Timestamps:  00:03:22 Cloud Native News  00:17:45 Interview with Torsten  00:57:00 Key takeaways

Join Bhavin Shah and Ryan Wallner for a recap of announcments and news from KubeCon Paris 2024.Kubernetes Community Days (KCD) in New York City on May 22nd, use the promo code “KUBERNETESBYTES” to get a 10% discount on your registration fees!Nethopper Learn more about KAOPS:  @nethopper.io  For a supported-demo:  info@nethopper.io Try the free version of KAOPS now!   https://mynethopper.com/authNews https://about.gitlab.com/blog/2024/03/20/oxeye-joins-gitlab-to-advance-application-security-capabilities/ https://www.redhat.com/en/blog/unveiling-red-hat-openshift-415 https://developer.nvidia.com/blog/nvidia-nim-offers-optimized-inference-microservices-for-deploying-ai-models-at-scale/ https://www.acorn.io/resources/blog/our-new-focus-developing-an-llm-app-platform-based-on-gpt-script-technology?fromOther=true https://loft.sh/blog/deliver-secure-kubernetes-multi-tenancy-with-new-vcluster-in-rancher-integration/ https://www.observeinc.com/blog/stepping-on-the-gas/ https://thenewstack.io/kubecost-2-2-covers-carbon-cost-monitoring-and-more/ https://thenewstack.io/ovhcloud-unveils-roadmap-to-take-on-hyperscalers-from-europe/ https://www.suse.com/c/meet-rancher-prime-3-0/ https://www.suse.com/c/suse-releases-edge-3-0-highly-validated-edge-optimized-stack/ https://www.fermyon.com/blog/introducing-spinkube-fermyon-platform-for-k8s  https://www.cncf.io/blog/2024/03/19/announcing-the-ai-working-groups-new-cloud-native-artificial-intelligence-whitepaper/  https://github.com/Azure/kaito  https://azure.microsoft.com/en-us/updates/public-preview-kubernetes-ai-toolchain-operator-kaito-addon-for-aks/ https://cloudnativenow.com/features/solo-io-delivers-on-cilium-support-promise-for-gloo-networks/  https://docs.solo.io/gloo-network/latest/about/overview/  https://github.com/kosmos-io/kosmos  https://gateway.envoyproxy.io/blog/2024/03/14/announcing-envoy-gateways-1.0-release/  https://newrelic.com/press-release/20240319  https://siliconangle.com/2024/03/29/aviatrix-revolutionizes-networking-security-distributed-cloud-firewall-kubernetes-kubeconeu/ 

Guest Charlie Egan, Sr. Developer Advocate at Styra, talks about Open Policy Agent (OPA) and its benefits for Kubernetes security. They discuss use cases, Kubernetes admission control, auditing, and OPA's role in improving security posture. The podcast sheds light on OPA's integration with service mesh ecosystems, policy enforcement at scale, and its versatile applications beyond Kubernetes.

Join hosts as they discuss the challenges of constructing Internal Developer Platforms (IDPs), balancing flexibility and prescriptive golden paths, and gaining insights on effective implementation. Learn about Nethopper's vision for simplifying Kubernetes operations and the evolution of cloud native architecture. Explore the necessity of planning for smooth upgrades in IDPs, balancing flexibility in operational environments, and finding the right tool balance for Kubernetes cluster management.

Explore the integration of Generative AI models on Kubernetes with expert Janakiram MSV. Dive into the challenges faced in running LLM models on NVIDIA GPUs, lessons learned, and the evolution of managing AI models. Also, discover insights on market trends, acquisitions in cloud native solutions, and optimizing inference engines on Kubernetes for efficient model deployment.

The podcast delves into Internal Development Platforms (IDP), discussing their definition, pros, and cons. They explore speeding up deployment processes to prevent delays for developers. The hosts emphasize optimizing IDP for Kubernetes deployment through self-service capabilities and collaboration. Additionally, they discuss building consensus for product development and exploring tools and platforms for IDPs in development teams.