Fully Connected – a series where Chris and Daniel keep you up to date with everything that’s happening in the AI community.

This week we discuss all things inference, which involves utilizing an already trained AI model and integrating it into the software stack. First, we focus on some new hardware from Amazon for inference and NVIDIA’s open sourcing of TensorRT for GPU-optimized inference. Then we talk about performing inference at the edge and in the browser with things like the recently announced ONNX JS.

Join the discussion

Changelog++ members support our work, get closer to the metal, and make the ads disappear. Join today!

Sponsors:

• DigitalOcean – DigitalOcean is simplicity at scale. Whether your business is running one virtual machine or ten thousand, DigitalOcean gets out of your way so your team can build, deploy, and scale faster and more efficiently. New accounts get $100 in credit to use in your first 60 days.
• Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com.
• Rollbar – We catch our errors before our users do because of Rollbar. Resolve errors in minutes, and deploy your code with confidence. Learn more at rollbar.com/changelog.
• Linode – Our cloud server of choice. Deploy a fast, efficient, native SSD cloud server for only $5/month. Get 4 months free using the code changelog2018. Start your server - head to linode.com/changelog

Featuring:

• Chris Benson – Twitter, GitHub, LinkedIn, Website
• Daniel Whitenack – Twitter, GitHub, Website

Show Notes:

News:

• NVIDIA’s open sourcing of TensorRT
• Amazon launches a machine learning chip
• The recently announced ONNX JS project
• Snapdragon Neural Processing Engine SDK

Learning resources:

• Rise of the model servers
• TensorRT server tutorial
• ONNX JS on GitHub
• TensorFlow JS tutorials

Something missing or broken? PRs welcome!