Machine Learning Browsers: How to Make Models Smaller

Victor Dibia is a Research Engineer with Cloudera’s Fast Forward Labs. On today’s podcast, Wes and Victor talk about the realities of building machine learning in the browser. The two discuss the capabilities, limitations, process, and realities around using TensorFlow.js. The two wrap discussing techniques like Model distillation that may enable machine learning models to be deployed in smaller footprints like serverless. - While there are limitations in running machine learning processes in a resource constrained environment like the browser, there are tools like TensorFlow.js that make it worthwhile. One powerful use case is the ability to protect the privacy of a user base while still making recommendations. TensorFlow.js takes advantage of the WebGL library for its more computational intense operations. - TensorFlow.js enables workflows for training and scoring models (doing inference) purely online, by importing a model built offline with more tradition Python tools, and a hybrid approach that builds offline and finetunes online. To build an offline model, you can build a model with TensorFlow Python (perhaps using a GPU cluster). The model can be exported into the TensorFlow SaveModel Format (or the Keras Model Format) and then converted with TensorFlow.js into the TensorFlow Web Model Format. At that point, the can be directly imported into your JavaScript. - TensorFlow Hub is a library for the publication, discovery, and consumption of reusable parts of machine learning models and was made available by the Google AI team. It can give developers a quick jumpstart into using trained models. - Model compression promises to make models small enough to run in places we couldn’t run models before. Model distillation is a process where a smaller model is trained to replicate the behavior of a larger one. In one case, BERT (a library almost 500MB in size) was distilled to about 7MB (almost 60x compression). More on this: Quick scan our curated show notes on InfoQ https://bit.ly/32rWnab You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/32rWnab