|
|
From Machine Learning Bookcamp by Alexey Grigorev In this series, we cover model deployment: the process of putting models to use. In particular, we’ll see how to package a model inside a web service, allowing other services to use it. We also show how to deploy the web service to a production-ready environment. |
Take 40% off Machine Learning Bookcamp by entering fccgrigorev into the discount code box at checkout at manning.com.
Check out part 1 and part 2 if you missed them.
For local development, Anaconda is a perfect tool: it has almost all the libraries we may ever need. This has also a downside: it takes 4 GB when unpacked, which is too large. For production, we prefer to have only the libraries we need.
Additionally, different services have different requirements. Often, these requirements conflict with each other, and we can’t use the same environment for running multiple services at the same time.
In this article, we’ll see how to manage dependencies of our application in an isolated way that doesn’t interfere with other services. We’ll cover two tools for this: Pipenv, for managing Python libraries and Docker, for managing the system dependencies such as the operating system and the system libraries.
Pipenv
For serving our churn model, we only need a few libraries: NumPy, Scikit-Learn, and Flask. Instead of bringing in the entire Anaconda distribution with all its libraries, we can get a fresh Python installation and install only the libraries we need with pip:
pip install numpy scikit-learn flask
Before we do that, let’s think for a moment what happens when we use pip to install a library:
- We run
pip install libraryto install a Python package called “Library” (let’s suppose it exists). - Pip goes to PyPI.org (the Python package index — a repository with python packages), gets and installs the latest version of this library: let’s say it’s 1.0.0.
After installing it, we develop and test our service using this particular version. Everything works great. Later, our colleagues want to help us with the project, and they run pip install to set up everything on their machine — except this time, the latest version turns out to be 1.3.1.
If we’re unlucky, 1.0.0 and 1.3.1 might be not compatible with each other, meaning that the code we wrote for 1.0.0 won’t work for 1.3.1.
It’s possible to solve this problem by specifying the exact version of the library when installing it with pip:
pip install library==1.0.0
Unfortunately, a different problem may appear: what if some of our colleagues already have 1.3.1 installed and they already used it for some other services? In this case, they can’t go back to using 1.0.0: it may mean that their code may stop working.
We can solve these problems by creating a virtual environment for each project: a separate Python distribution with nothing else, but libraries required for this particular project.
Pipenv is a tool that makes managing virtual environments easier. We can install it with pip:
pip install pipenv
After that, we use Pipenv instead of pip for installing dependencies:
pipenv install numpy pandas sklearn
When running it, we’ll see that first, it configures the virtual environment, and then it installs the libraries:
Running virtualenv with interpreter .../bin/python3 ✔ Successfully created virtual environment! Virtualenv location: ... Creating a Pipfile for this project… Installing numpy… Adding numpy to Pipfile's [packages]… ✔ Installation Succeeded Installing scikit-learn… Adding scikit-learn to Pipfile's [packages]… ✔ Installation Succeeded Installing flask… Adding flask to Pipfile's [packages]… ✔ Installation Succeeded Pipfile.lock not found, creating… Locking [dev-packages] dependencies… Locking [packages] dependencies… ⠙ Locking...
After finishing the installation, it creates two files: Pipenv and Pipenv.lock.
The Pipenv file looks pretty simple:
[[source]] name = "pypi" url = "https://pypi.org/simple" verify_ssl = true [dev-packages] [packages] numpy = "*" scikit-learn = "*" flask = "*" [requires] python_version = "3.7"
We see that this file contains a list of libraries as well as the version of Python we use.
The other file — Pipenv.lock — contains the specific versions of the libraries that we used for the project. The file is quite large to show here, but let’s take a look at one of the entries in the file:
"flask": {
"hashes": [
"sha256:4efa1ae2d7c9865af48986de8aeb8504...",
"sha256:8a4fdd8936eba2512e9c85df320a37e6..."
],
"index": "pypi",
"version": "==1.1.2"
}
As we see, it records the exact version of the library that was used during installation. To make sure the library doesn’t change, it also saves the hashes — the checksums that can be used to validate that in the future we download the exact same version of the library. This way, we “lock” the dependencies to specific versions. By doing it, we make sure that in the future we don’t have surprises with two incompatible versions of the same library.
If somebody needs to work on our project, they need to run the install command:
pipenv install
This creates a virtual environment and then installs all the required libraries from Pipenv.lock.
IMPORTANT: Locking the version of a library is important for reproducibility in the future and helps us avoid having unpleasant surprises with code incompatibility.
After all the libraries are installed, we need to activate the virtual environment — this way, our application uses the correct versions of the libraries. We do it by running the shell command:
pipenv shell
It tells us that it’s running in a virtual environment:
Launching subshell in virtual environment…
Now we can run our script for serving:
python churn_serving.py
Alternatively, instead of first explicitly entering the virtual environment and then running the script, we can do these two steps with one command:
pipenv run python churn_serving.py
The run command in Pipenv runs the specified program in the virtual environment.
Regardless of the way we run it, we should see exactly the same output as previously:
* Serving Flask app "churn" (lazy loading) * Environment: production WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Debug mode: on * Running on http://0.0.0.0:9696/ (Press CTRL+C to quit)
When we test it with requests, we see the same output:
{'churn': False, 'churn_probability': 0.061875678218396776}
You most likely also noticed the following warning in the console:
* Environment: production WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
The built-in Flask web-server is indeed for development only: it’s easy to use for testing our application, but it won’t be able to work reliably under load. We should use a proper WSGI server instead, as the warning suggests.
WSGI stands for “Web Server Gateway Interface”, which is a specification describing how Python applications should handle HTTP requests. The details of WSGI aren’t important for the purposes of this article, and we won’t cover it in detail.
We do address the warning by installing a “production WSGI server”. Multiple possible options can be found in Python. We use gunicorn.
Let’s install it with Pipenv:
pipenv install gunicorn
This command installs the library and includes it as a dependency in the project by adding it to Pipenv and Pipenv.lock files.
Let’s run our application with gunicorn:
pipenv run gunicorn --bind 0.0.0.0:9696 churn_serving:app
If everything goes well, we should see the following messages in the terminal:
[2020-04-13 22:58:44 +0200] [15705] [INFO] Starting gunicorn 20.0.4 [2020-04-13 22:58:44 +0200] [15705] [INFO] Listening at: http://0.0.0.0:9696 (15705) [2020-04-13 22:58:44 +0200] [15705] [INFO] Using worker: sync [2020-04-13 22:58:44 +0200] [16541] [INFO] Booting worker with pid: 16541
Unlike the Flask built-in web-server, gunicorn is ready for production, and it doesn’t have any problems under load when we start using it.
If we test it with the same code as previously, we’ll see the same answer:
{'churn': False, 'churn_probability': 0.061875678218396776}
Pipenv is a great tool for managing dependencies: it isolates the required libraries into a separate environment, helping us avoid conflicts between different versions of the same package.
In the part 4, we’ll look at Docker, which allows us to isolate our application even further and ensure it runs smoothly anywhere.
That’s all for this article.
If you want to learn more about the book, check it out on our browser-based liveBook platform here.