Understanding Fault Tolerance in Distributed Computing

Today we’re joined by Jonathan Hung, Sr. Software Engineer at LinkedIn. Jonathan presented at TensorFlow world last week, titled Scaling TensorFlow at LinkedIn. In our conversation, we discuss their motivation for using TensorFlow on their pre-existing Hadoop clusters infrastructure, TonY, or TensorFlow on Yard, LinkedIn’s framework that natively runs deep learning jobs on Hadoop, and its relationship with Pro-ML, LinkedIn’s internal AI Platform, and their foray into using Kubernetes for research.