|
|
From Machine Learning Bookcamp by Alexey Grigorev In this series, we cover model deployment: the process of putting models to use. In particular, we’ll see how to package a model inside a web service, allowing other services to use it. We also show how to deploy the web service to a production-ready environment.
|
Take 40% off Machine Learning Bookcamp by entering fccgrigorev into the discount code box at checkout at manning.com.
See part 1, part 2, and part 3 if you missed them.
We have learned how to manage Python dependencies with Pipenv. Some of the dependencies live outside of Python. Most importantly, these dependencies include the operating system (OS) as well as the system libraries.
For example, we might use Ubuntu 16.04 for developing our service, but if some of our colleagues use Ubuntu 20.04, they may run into troubles when trying to execute the service on their laptop.
Docker solves this “but it works on my machine” problem by also packaging the OS and the system libraries into a Docker container — a self-contained environment that works anywhere where Docker is installed (figure 1).
Figure 1. In case of no isolation (a), the service runs with system Python. In virtual environments (b), we isolate the dependencies of our service inside the environment. In Docker containers (c), we isolate the entire environment of the service, including the OS and system libraries.
Once the service is packaged into a Docker container, we can run it on the host machine — our laptop (regardless of the OS) or any public cloud provider.
Let’s see how to use it for our project. For the purposes of this article, it is assumed that you already have Docker installed.
First, we need to create a Docker image — the description of our service that includes all the settings and dependencies. Docker uses the image to create a container. To do it, we need a Dockerfile — a file with instructions on how the image should be created (figure 2).
Figure 2. We build an image using instructions from Dockerfile. Then we can run this image on a host machine.
Let’s create a file with name Dockerfile and the following content: (note that the file shouldn’t include comments like #A, #B)
FROM python:3.7.5-slim #A ENV PYTHONUNBUFFERED=TRUE #B RUN pip --no-cache-dir install pipenv #C WORKDIR /app #D COPY ["Pipfile", "Pipfile.lock", "./"] #E RUN pipenv install --deploy --system && \ #F rm -rf /root/.cache #F COPY ["*.py", "churn-model.bin", "./"] #G EXPOSE 9696 #H ENTRYPOINT ["gunicorn", "--bind", "0.0.0.0:9696", "churn_serving:app"] #I
#A Specify the base image
#B Set a special Python settings for being able to see logs
#C Install Pipenv
#D Set the working directory to /app
#E Copy the Pipenv files
#F Install the dependencies from the Pipenv files
#G Copy our code as well as the model
#H Open the port that our web service uses
#I Specify how the service should be started
This is a lot of information to unpack, like if you have never seen Dockerfiles previously.
Let’s go line-by-line.
First, in A, we specify the base Docker image:
FROM python:3.7.5-slim #A
This is the image we use as the starting point and build our own image on top of that. Typically, the base image already contains the OS and the system libraries like Python itself, and we only need to install the dependencies of our project. In our case, we use python:3.7.5-slim, which is based on Debian 10.2 and contains Python 3.7.5 and pip. You can read more about the Python base image in Docker hub — the service for sharing Docker images.
All Dockerfiles should start with the FROM statement.
In B, we set the PYTHONUNBUFFERED environmental variable to TRUE:
ENV PYTHONUNBUFFERED=TRUE #B
Without this setting, we won’t be able to see the logs when running Python scripts inside Docker.
In C, we use pip to install Pipenv:
RUN pip --no-cache-dir install pipenv #C
The RUN instruction in Docker runs a shell command. By default, pip saves the libraries to a cache, and later they can be installed faster. We don’t need that in a Docker container; we use the --no-cache-dir setting.
In D, we specify the working directory:
WORKDIR /app #D
This is roughly equivalent to the cd command in Linux (change directory), and everything we run after this is executed in the /app folder.
In E, we copy the Pipenv files to the current working directory (i.e. /app):
COPY ["Pipfile", "Pipfile.lock", "./"] #E
We use these files in D for installing the dependencies with Pipenv:
RUN pipenv install --deploy --system && \ #F rm -rf /root/.cache #F
Previously we used “pipenv install” for doing it. Here we include two extra parameters: --deploy and --system. Inside Docker, we don’t need to create a virtual environment — our Docker container is already isolated from the rest of the system. Setting these parameters allows us to skip creating a virtual environment and use the system Python for installing all the dependencies.
After installing the libraries, we clean the cache to make sure our Docker image doesn’t grow too big.
In G, we copy our project files as well as the pickled model:
COPY ["*.py", "churn-model.bin", "./"] #G
In H, we specify which port our application uses, In our case, it’s 9696:
EXPOSE 9696 #H
Finally, in I, we tell Docker how our application should be started:
ENTRYPOINT ["gunicorn", "--bind", "0.0.0.0:9696", "churn_serving:app"] #I
This is the same command we used previously when running gunicorn locally.
Let’s build the image. We do it by running the build command in Docker:
docker build -t churn-prediction .
The “-t” flag lets us set the tag name for the image, and the final parameter — the dot — specifies the directory with the Dockerfile. In our case, it means that we use the current directory.
When we run it, the first thing Docker does is download the base image:
Sending build context to Docker daemon 51.71kB Step 1/11 : FROM python:3.7.5-slim 3.7.5-slim: Pulling from library/python 000eee12ec04: Downloading 24.84MB/27.09MB ddc2d83f8229: Download complete 735b0bee82a3: Downloading 19.56MB/28.02MB 8c69dcedfc84: Download complete 495e1cccc7f9: Download complete
Then it executes each line of the Dockerfile one by one:
Step 2/9 : ENV PYTHONUNBUFFERED=TRUE ---> Running in d263b412618b Removing intermediate container d263b412618b ---> 7987e3cf611f Step 3/9 : RUN pip --no-cache-dir install pipenv ---> Running in e8e9d329ed07 Collecting pipenv ...
At the end, Docker tells us that it successfully built an image and it tagged it as churn-prediction:latest:
Successfully built d9c50e4619a1 Successfully tagged churn-prediction:latest
We’re ready to use this image to start a Docker container. Use the run command for that:
docker run -it -p 9696:9696 churn-prediction:latest
A few parameters that we specify here are:
- The “-it” flag tells Docker that we run it from our terminal and we need to see the results
- The “-p” parameter specifies the port mapping. “9696:9696” means to map the port 9696 on the container to the port 9696 on the host machine.
- Finally, we need the image name and tag, which in our case is churn-prediction:latest
Now our service is running inside a Docker container and we can connect to it using port 9696 (figure 3). This is the same port we used for our application previously.
Figure 3. The 9696 port on the host machine is mapped to the 9696 port of the container, and when we send a request to localhost:9696, it’s handled by our service in Docker
Let’s test it using the same code. When we run it, we’ll see the same response:
{'churn': False, 'churn_probability': 0.061875678218396776}
Docker makes it easy to run services in a reproducible way. With Docker, the environment inside the container always stays the same. This means that if we can run our service on a laptop, it works anywhere else.
We already tested our application on our laptop; in part 5 we’ll see how to run it on a public cloud and deploy it to AWS.
That’s all for this article.
If you want to learn more about the book, check it out on our browser-based liveBook platform here.