Slides of a talk given by Ted Dunning at the NYC ML Meetup on September 21th 2017: Rendezvous Server to the Rescue: Dealing with Machine Learning Logistics - NYC Machine Learning (New York, NY) - Meetup
"If you have put machine learning models into production, you’ve lived the truth of the maxim that 90% of what makes machine learning work is the logistics, not the learning. That 90% comes from many things, including the need to stage and deploy multiple versions of each model, to carefully collect and curate updated training data and to monitor model performance. Lately we have added scale, speed and the need to handle multiple machine learning frameworks at the same time to make the problem more difficult.
There is a way to make this easier and more effective – the rendezvous architecture. It makes use of recent advances in streaming micro-services, containerization, and orchestration. It solves many of the problems involved in continuous deployment of machine learning models. In presenting the rendezvous architecture, I’ll cover techniques for model deployment, management, monitoring and comparison. After the talk, we will have an open discussion about where this effort should go from here.