til.duyet.net
blog
Search…
til.duyet.net
Today I Learned
Data Engineer
AWS
Tools
Shell
EMR
Kubernetes
Helm Charts
Apache Airflow
Docker
Database
Google BigQuery
Apache Hive
AWS Redshift / Postgres
Presto
Python
date_range_generator
get_all_s3_keys
Pipenv
YAML config file with environment variables
Unix
Git - Pretty git branch graphs
Checking files in Docker build context
Bash get the directory of the current script
Vim
Find and replace
Articles
Miscellaneous
[Fig] Single machine and distributed system structure
Deploying Machine Learning Models at Scale
Bypass a Chrome certificate/HSTS error
Go
Functions and Methods in Go?
Convert JSON to Go struct
Frontend / React
Flexbox
Create hook to inject JS script
Javascript / Typescript
Intl.ListFormat
Powered By GitBook
Deploying Machine Learning Models at Scale
https://algorithmia.com/blog/deploying-machine-learning-at-scale
Deploying machine learning models at scale is one of the most pressing challenges faced by the community of data scientists today, and as ML models get more complex, it’s only getting harder. The most common way machine learning gets deployed today is on powerpoint slides.
We estimate that fewer than 5 percent of commercial data science projects make it to production. If you want to be part of that share, you need to understand how deployment works, why machine learning is a unique deployment problem, and how to navigate this messy ecosystem.
---

Machine learning has a few unique features that makes deploying it at scale harder

Deploying regular software applications is hard—but when that software is a machine learning pipeline, it’s worse.
    1.
    Multiple Data Science Languages
      1.
      R, Python, Scalar ...
    2.
    Data Science Languages Can Be Slow
    3.
    Machine Learning Can Be Extremely Compute Heavy, and Relies on GPUs
    4.
    Machine Learning Compute Works In Spikes
      1.
      Once your algorithms are trained, they’re not used consistently––your customers will only call them when they need them. That can mean that you’re only supporting 100 API calls at 8:00 AM, but 1 Million at 8:30 AM. Scaling up and down like that while making sure not to pay for servers you don’t need is a nightmare.
After taking months to write out your (awesome) models, you’re going to need to hand them over to engineering to deploy at scale. That process can take months, and the models you end up with may not at all resemble what you handed them originally. And if you want to make small changes after, or continually improve your models with new data? Forget about it.
​
Miscellaneous - Previous
[Fig] Single machine and distributed system structure
Next - Miscellaneous
Bypass a Chrome certificate/HSTS error
Last modified 1yr ago
Copy link
Contents
Machine learning has a few unique features that makes deploying it at scale harder