An excerpt from Infrastructure as Code, Patterns and Practices

By Rosemary Wang

This article discusses how to apply dependency injection to infrastructure to isolate and minimize resource dependencies.

Read it if you’re a sysadmin or software engineer familiar with Python, the basics of provisioning tools like Terraform, and public cloud providers such as Google Cloud Platform that wants to get a better grasp on DI for infrastructure.


Take 25% off Infrastructure as Code, Patterns and Practices by entering fccwang into the discount code box at checkout at manning.com.


Dependency injection

You must express dependencies to minimize the impact of low-level module changes to high-level modules. For example, network changes shouldn’t disrupt any high-level resources like queues, applications, or databases.

Dependency injection involves two principles: inversion of control and dependency inversion.

Inversion of Control

Unidirectional dependencies mean a high-level resource depends on a low-level one; the high-level resource calls the low-level one for attributes. When you enforce unidirectional relationships in your infrastructure dependencies, you naturally apply inversion of control. The high-level resource calls the low-level resource for the attributes it needs (figure 1). You use inversion of control when you call to schedule a doctor’s appointment instead of the doctor’s office calling to remind you.


Figure 1. With inversion of control, the high-level resource or module calls the low-level module for information and parses its metadata for any dependencies.


I’ll apply inversion of control to show how a server depends on a network. Rather than hard-code or parameterize the subnet name, the server calls the network module’s outputs for the subnet name. The network module returns the subnet name in a JSON file named “terraform.tfstate” for any high-level resource to use.

Listing 1 Network module creates a JSON file with a subnet name.

 
 {
  "version": 4,
  "terraform_version": "0.14.8",
  "serial": 8,
  "lineage": "7feea4f3-631e-24f9-c5e2-0b3aa95d9517",
  "outputs": {     #A
    "name": {     #B
      "value": "hello-world-subnet",    #B
      "type": "string"
    }
  }    #C
 }
  

#A Network module creates a JSON file with a list of outputs.

#B Network module includes an output for subnet name.

#C Remainder of JSON file omitted for clarity.

The server uses inversion of control to get outputs from the network’s “terraform.tfstate” file. Then, the server module parses it for the subnet name.

Listing 2 Applying inversion of control to create a server on a network.

 
 import json
  
  
 class NetworkModuleOutput:    #A
    def __init__(self):
        with open('network/terraform.tfstate', 'r') as network_state:
            network_attributes = json.load(network_state)
        self.name = network_attributes['outputs']['name']['value']
  
  
 class ServerFactoryModule:
    def __init__(self, name, zone='us-central1-a'):
        self._name = name
        self._network = NetworkModuleOutput()    #B
        self._zone = zone
        self.resources = self._build()
  
    def _build(self):
        return {
            'resource': [{
                'google_compute_instance': [{
                    self._name: [{
                        'allow_stopping_for_update': True,
                        'boot_disk': [{
                            'initialize_params': [{
                                'image': 'ubuntu-1804-lts'
                            }]
                        }],
                        'machine_type': 'f1-micro',
                        'name': self._name,
                        'zone': self._zone,
                        'network_interface': [{
                            'subnetwork': self._network.name    #C
                        }]
                    }]
                }]
            }]
        }
  
  
 if __name__ == "__main__":
    server = ServerFactoryModule(name='hello-world')
    with open('main.tf.json', 'w') as outfile:
        json.dump(server.resources, outfile, sort_keys=True, indent=4)
  

#A Network module output creates an object with the subnet name.

#B Server calls the network output and retrieves all of the module’s attributes.

#C Retrieve the name of the subnet to create the server.

The server only needs to get the subnet name from the network module. It eliminates a direct reference in my server module. I also control and limit the information the network returns for high-level resources to use. When I have new high-level resources depending on the network, I can add more network attributes they’ll need. An implementation that uses inversion of control removes the explicit dependencies and empowers the high-level module or resource to maintain the dependency. Inversion of control improves evolvability and composability.

What happens if I need to change the network IP address range? Thanks to the inversion of control, the server module recognizes the new range and reallocates a new IP address, but a change to the low-level module still disrupts the high-level one.

Dependency Inversion

Although inversion of control enables the evolution of high-level modules, dependency inversion isolates changes to low-level modules and mitigates disruption to their dependencies. Dependency inversion dictates that high-level and low-level resources should have dependencies expressed through abstractions. You can think of the abstraction as a translator that communicates the required attributes. The abstraction allows you to change the low-level module without affecting the high-level one. You use dependency inversion when you use a translation application. The translator becomes the interface for you to retrieve information about the text. With my server and network example, I need to access the network’s attributes that update when I change the network (figure 2).


Figure 2. Dependency inversion returns an abstraction of the low-level resource metadata to the resource that depends on it.


You can choose from three types of abstraction:

  1. Interpolation of resource attributes (within modules)
  2. Module outputs (between modules)
  3. Infrastructure state (between modules)

Interpolation handles the passing of attributes between resources or tasks within a module or configuration. The tool retrieves the information for you from the infrastructure API. For example, I could use Terraform’s native interpolation to get the network name for subnet creation.

Listing 3 Use interpolation to pass the network to a subnet

 
 {
  "resource": [
    {
      "google_compute_network": [
        {
          "hello-world-network": [
            {
              "auto_create_subnetworks": false,
              "name": "hello-world-network"    #B
            }
          ]
        }
      ]
    },
    {
      "google_compute_subnetwork": [
        {
          "hello-world-network": [
            {
              "ip_cidr_range": "10.0.0.0/16",
              "name": "hello-world-subnet",
              "network": "${google_compute_network.hello-world-network.name}",    #A
              "region": "us-central1"
            }
          ]
        }
      ]
    }
  ]
 }
  

#A Use Terraform resource interpolation to retrieve network name for subnet creation.

#B Terraform replaces the interpolation with the name of the network.

Some tools use outputs to pass resource attributes between modules. With a configuration management tool such as Ansible, the tool passes variables by standard output between automation tasks. For a provisioning tool like AWS CloudFormation or HashiCorp Terraform, you generate outputs for modules or stacks which higher-level ones can consume. You can customize outputs to any schema or parameters you need. For example, I can create an output for the network module with the subnet name.

Listing 4 Setting the subnet name as the output for a module.

 
 {
  "resource": [
    {
      "google_compute_subnetwork": [    #A
        {
          "hello-world-network": [
            {
              "ip_cidr_range": "10.0.0.0/16",
              "name": "hello-world-subnet",
              "network": "hello-world-network",
              "region": "us-central1"
            }
          ]
        }
      ]
    }
  ],
  "output": [    #B
    {    #B
      "name": [    #B
        {    #B
          "value": "${google_compute_subnetwork.hello-world-network.name}"    #B
        }    #B
      ]    #B
    }    #B
  ]    #B
 }
  

#A Definition of the subnet for the network.

#B Terraform outputs the subnet name as part of the network module.

You can also use infrastructure state as a state file or infrastructure provider’s API metadata. Recall the previous section on the inversion of control. I got the subnet name for the server from a file called “terraform.tfstate”. This file is the network state file, an abstraction offered by my tool. Not all tools offer a state file, and I prefer to use the infrastructure provider’s API. Infrastructure APIs rarely change, provide detailed information, and account for out-of-band changes that a state file may not include. You can find the different abstractions you can use for inversion of control in figure 3.


Figure 3. Abstractions for dependency inversion can use attribute interpolation, module outputs, or infrastructure state depending on the tool and dependencies.


Applying Dependency Injection

When you combine inversion of control and dependency inversion, you get dependency injection. Inversion of control isolates changes to the high-level module, ands dependency inversion isolates changes to the low-level module. Dependency injection further isolates changes, mitigating the potential blast radius of both high-level and low-level module changes (figure 4).


Figure 4. Dependency injection combines the principle of inversion of control and dependency inversion to loosen infrastructure dependencies and allow isolated evolution of low-level and high-level resources.


I implement dependency injection for the server and network example with Apache Libcloud. Apache Libcloud’s a library for Google Cloud Platform (GCP) API. I use it to search for the network. The server calls the GCP API for the subnet name, parses the GCP API metadata, and assigns itself the fifth IP address in the network’s range.

Listing 5 Using dependency injection to create a server on a network.

 
 import credentials
 import ipaddress
 import json
 from libcloud.compute.types import Provider
 from libcloud.compute.providers import get_driver
  
  
 def get_network(name):  #A
    ComputeEngine = get_driver(Provider.GCE)  #A
    driver = ComputeEngine(  # A
        credentials.GOOGLE_SERVICE_ACCOUNT,  #A
        credentials.GOOGLE_SERVICE_ACCOUNT_FILE,  #A
        project=credentials.GOOGLE_PROJECT,  #A
        datacenter=credentials.GOOGLE_REGION)  #A
    return driver.ex_get_subnetwork(  #A
        name, credentials.GOOGLE_REGION)  #A
  
  
 class ServerFactoryModule:
    def __init__(self, name, network, zone='us-central1-a'):
        self._name = name
        gcp_network_object = get_network(network)  #B
        self._network = gcp_network_object.name  #C
        self._network_ip = self._allocate_fifth_ip_address_in_range(   #C
            gcp_network_object.cidr)  #C
        self._zone = zone
        self.resources = self._build()
  
    def _allocate_fifth_ip_address_in_range(self, ip_range):    #D
        ip = ipaddress.IPv4Network(ip_range)    #D
        return format(ip[-2])    #D
  
    def _build(self):
        return {
            'resource': [{
                'google_compute_instance': [{
                    self._name: [{
                        'allow_stopping_for_update': True,
                        'boot_disk': [{
                            'initialize_params': [{
                                'image': 'ubuntu-1804-lts'
                            }]
                        }],
                        'machine_type': 'f1-micro',
                        'name': self._name,
                        'zone': self._zone,
                        'network_interface': [{
                            'subnetwork': self._network,
                            'network_ip': self._network_ip
                        }]
                    }]
                }]
            }]
        }
  
  
 if __name__ == "__main__":
    server = ServerFactoryModule(name='hello-world', network='default')    #A
    with open('main.tf.json', 'w') as outfile:
        json.dump(server.resources, outfile, sort_keys=True, indent=4)
  

#A Get all attributes for the default network by calling the Google Cloud Platform API via Apache Libcloud.

#B Get a network object with its schema defined by Apache Libcloud.

#C Select the name and CIDR block parameters to create a server on the fifth IP address of the network.

#D This method calculates the fifth IP address of the CIDR block range to the server.

The example shown in figure 5 implements inversion of control by allowing the server to call the network. It uses GCP API as an abstraction to retrieve network attributes, applying dependency inversion. When I change the IP address range for the network, my server gets the updated address range and reallocates the IP address if needed.


Figure 5. Dependency injection allows me to change the low-level module, the network, and automatically propagate the change to the server, the high-level module.


AWS equivalent

You can use the AWS Python SDK to get AWS VPC information. The SDK interacts with the AWS API and returns similar information to the GCP example.

Dependency injection applies as a general principle for infrastructure dependency management. If you apply dependency injection when you write your infrastructure configuration, you sufficiently decouple dependencies so that you can change them independently without affecting other infrastructure. As your module grows, you can continue to refactor to more specific patterns and further decouple infrastructure based on the type of resources and modules.

That’s all for this excerpt.

If you want to learn more about the book, you can check it out on Manning’s liveBook platform here.