|From Spring Microservices in Action by John Carnell
This article discusses controlling service configuration using Spring Cloud Config.
Separating service configuration from service code is critical to the deployment of a cloud-based microservice. At one point or another a developer will be forced to separate configuration information from code. After all, it has been drilled into their heads since school that they shouldn’t hard-code values into the application code. Some developers use a constants class file in their application to help centralize their configuration in one place. Application configuration data written directly into the code is often problematic because every time a change to the configuration is made, the application must be recompiled and/or re-deployed. To avoid this, developers separate the configuration information from the application code. This makes it easy to make changes to configuration without going through a recompile process, but also introduces complexity, as you now have another artifact to manage and deploy with the application.
Many developers turn to the lowly property file (or YAML, JSON, or XML) to store their configuration information. This property file sits on a server, often containing database and middleware connection information and meta-data about the application used to drive the application’s behavior. Segregating your application into a property file is easy, and most developers never do any more operationalization of their application configuration than placing their configuration file under source control (if that) and deploying it as part of their application.
This approach might work with a small number of applications, but it quickly falls apart when dealing with cloud-based applications that might contain hundreds of microservices, where each microservice might have multiple service instances running.
Suddenly configuration management becomes a big deal, as the application and operations team in a cloud-based environment must wrestle with a rat’s nest of figuring out which configuration files go where.
Cloud-based development emphasizes:
- Completely separating the configuration of an application from the code being deployed.
- Building the server and the application, and an immutable image thatnever changes as it’s promoted through your environments.
- Injecting any application configuration information at startup time of the server through either environment variables or through a centralized repository read by the application’s microservices on startup.
On managing configuration (and complexity)
Managing application configuration is critical for a microservice, cloud-based application, because microservices running in the cloud need to be able to launch quickly, with minimal human intervention. Every time a human needs to manually configure or touch a service to get it deployed is an opportunity for configuration drift and outage.
Let’s begin our discussion about application configuration for cloud-based microservices by establishing four principles we want to follow.
- We want to separate the services configuration information from the physical deployment of a service. Application configuration shouldn’t be deployed with the service instance. Instead configuration information should either be passed to the starting service as environment variables or read from a centralized repository when the service starts.
- Abstract. Abstract access to the configuration data behind a service interface. Rather than writing code that directly accesses the service repository (e.g. read the data out of a file or a database using JDBC), have the application use a REST-based JSON service to retrieve information.
- Because a cloud-based application might literally have hundreds of services, it’s critical to minimize the number of different repositories used to hold configuration information. Centralize your application configuration into as few repositories as possible.
- Because your application configuration information is going to be completely segregated from your deployed service and centralized, it’s critical that your solution is highly available and redundant.
One of the key things to remember is that when you separate configuration information from code, you’re creating an external dependency that needs to be managed and version controlled. I can’t emphasize enough that the application configuration data needs to be tracked and version-controlled, as mismanaged application configuration is fertile ground for difficult to detect bugs and unplanned outages.
On accidental complexity
I’ve experienced the dangers of not having a strategy for managing my application configuration data. I worked at a Fortune 500 financial services company, and I was asked to help bring a large WebSphere upgrade project back on track. The company in question had over 120 applications on WebSphere and needed to upgrade their infrastructure from WebSphere 6 to WebSphere 7 before the entire application environment went end-of-life.
The project had been going on for a year and only one of the 120 applications was deployed. The project cost a million dollars in people and hardware, and their trajectory put them on track to finish the upgrade in two more years.
When I started working with the application team, one of the major problems I uncovered was that the application team managed all of their configuration for their databases, and the endpoints for their services, inside of property files. These property files were managed by hand and weren’t under source control. With 120 applications spread across four environments and multiple WebSphere nodes for each application, this turned into the team trying to migrate 12,000 configuration files that were spread across all of the hundreds of servers and applications running on the server. (You’re reading this number right: 12,000). These files were only for application configuration, and didn’t include application server configuration.
I convinced the project sponsor to take two months to consolidate the application information down to a centralized, version-controlled configuration repository with 20 configuration files. When I asked the framework team how it got to the point where they had 12,000 configuration files, the lead engineer on the team said originally, they designed their configuration strategy around a small group of applications. Unfortunately the number of web applications built and deployed exploded over five years, and though they begged for money and time to rework their configuration management approach, their business partners and IT leaders never considered it a priority.
Not spending the time up front to figure out how you’re going to do configuration management can have real (and costly) downstream impacts.
Our configuration management architecture
Loading configuration management for a microservice occurs during the bootstrapping phase of the microservice.
Figure 1 The application configuration data is read during the service bootstrapping phase
Let’s look at four principles of configuration management, and go through them in detail. The diagram below shows how the configuration management part of the bootstrapping process works.
Figure 2 Configuration management conceptual architecture
In figure 2, we see several activities taking place.
- When a microservice instance comes up it’s going to call a service endpoint to read its configuration information that’s specific to the environment it operates in. The connection information for the configuration management (connection credentials, service endpoint, etc.) are passed into the microservice when it starts up.
- The actual configuration resides in a repository. Based on the implementation of configuration repository, you can choose to use a number of different implementations to hold your configuration data. The implementation choices can include files under source control, a relational database, or a key-value data store.
- The actual management of the application configuration data occurs independently of how the application is deployed. Changes to configuration management are typically handled through the build and deployment pipeline, where changes of the configuration can be tagged with version information and deployed through the different environments.
- When a configuration management change is made, the services that use that application configuration data must be notified of the change and refresh their copy of the application data.
At this point we’ve worked through the conceptual architecture that illustrates the different pieces of a configuration management pattern and how these pieces fit together.