Data Science in the Cloud: Amazon SageMaker
Amazon Web Services (AWS) offers a vast catalog of services and tools which ought to make even the most veteran Data Scientists or Engineer’s minds bubble with ideas of how to leverage the incredible potential available to them in Amazon’s cloud platform. One service has always stood out to me in terms of its potential to be harnessed to solve the challenge of deploying machine learning models into production – Amazon SageMaker. On AWS, the SageMaker service provides a platform for data scientists to deploy pre-packaged amazon models or their own custom models as an easily accessible and scalable API endpoint. One of my favorite examples of an organization utilizing SageMaker is the NFL utilizing the platform to deploy machine learning models which are capable of quickly breaking down gameplay to improve the fan experience with NFL Next-Gen Stats (NGS).
Here at Softcrylic we’ve explored SageMaker internally using a the Kaggle Telco Customer Churn dataset to explore the ins and outs of the machine learning service. We ultimately wanted to learn as much as we could about the service and see if we could easily deploy our own custom Sci-Kit Learn Random Forest customer churn classifier model. We were shocked at how easy it was to get up and running; it was literally one line of code after training the model to deploy it.
To perform all the coding for this project we used a SageMaker Notebook, which is a Jupyter notebook or JupyterLab frontend to one of AWS’s machine learning ec2s. This really enabled our team at Softcrylic to get rolling with the Scikit Learn SageMaker SDK in a familiar Jupyter coding environment. Once inside your Notebook environment you can import the SageMaker module along with the other python packages you will be using.
After establishing your SageMaker session you can pre-process your data just like business as usual. Below we do some simple data cleaning and transformation prior to training our basic random forest classifier.
After processing your data, we need to create a s3 subdirectory for all our modeling data. SageMaker uses this subdirectory location during the training process to read all the data into the training environment.
Then we need to create the python script which will be executed by SageMaker to train the model. This requires us to define how SageMaker should parse the input arguments which represent the hyper parameters we plan to train the model with. We also need to tell SageMaker where to read the training data from and how to fit the model. Within the python script, we can also have SageMaker dump whatever model artifacts we want to another directory; in our testing we decided to have SageMaker drop the model coefficients.
After the creation of the training script we then define the size of the machine, we wish to train our model in. params_rf is a python dictionary we created, which contains the values of the hyper parameters we are choosing to train our model with. Further, it is important to note that we are using the sklearn.estimator.SKLearn() method to train our model, if we wanted to use another modeling library we would need to use a separate estimator belonging to that respective library instead.
Finally, with one line of code, we deploy our model as a SageMaker hosted API Endpoint.
Through our experience with SageMaker we found the service has a ton of potential. Importantly, we discovered SageMaker offers us a managed platform where any machine learning use case can be worked on and deployed in, which means SageMaker has the flexibility to be used within any business vertical. Further, SageMaker streamlines the process of deploying models. Due to this ease, other team members, stakeholders, or clients can quickly leverage deployed models and immediately extract value from the models in their new production setting.
Hopefully this post encourages you to explore the capabilities of Amazon SageMaker and please feel free to reach out to myself (@SamStiyer on twitter) if you have any questions or feedback regarding the post!
