The landscape of AI infrastructure (Practical AI #37)

Being that this is “practical” AI, we decided that it would be good to take time to discuss various aspects of AI infrastructure. In this full-connected episode, we discuss our personal/local infrastructure along with trends in AI, including infra for training, serving, and data management.

Join the discussion

Changelog++ members support our work, get closer to the metal, and make the ads disappear. Join today!

Sponsors:

• DigitalOcean – Check out DigitalOcean’s dedicated vCPU Droplets with dedicated vCPU threads. Get started for free with a $100 credit. Learn more at do.co/changelog.
• DataEngPodcast – A podcast about data engineering and modern data infrastructure.
• Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com.
• Rollbar – We move fast and fix things because of Rollbar. Resolve errors in minutes. Deploy with confidence. Learn more at rollbar.com/changelog.

Featuring:

• Chris Benson – Website, GitHub, LinkedIn, X
• Daniel Whitenack – Website, GitHub, X

Show Notes:

Our locally installed stuff:

• Jupyter
• Docker
• Python
• Go
• Postman

Where we see AI workflows running:

• AWS
• GCP
• Azure
• Kubernetes and KubeFlow
• On-prem workstations:
  • NVIDIA
  • Lambda Labs
  • System76

Experimentation / model development:

• JupyterLab
• Google Colaboratory
• AWS SageMaker
• Data Science platforms:
  • Domino
  • DataBricks
  • DataRobot
  • H2O.ai

Pipelining and automation:

• Pachyderm
• Airflow
• Luigi
• Model optimization:
  • OpenVino
  • TensorRT
  • TensorFlow Lite

Serving:

• MXNet
• TensorFlow serving
• Seldon

Monitoring/visibility:

• TensorBoard
• Netron
• Knock knock
• Prometheus
• ElasticSearch

Something missing or broken? PRs welcome!