Testing Heat Templates

In , I talked about what Heat is and the basics of creating instances by defining Heat Templates. Those templates were simple and deliciously declarative, however, let’s push the boundaries towards production environments.

Complex Heat Templates

In my day job, a tenant project names follow a pattern of combining various component labels separated by dashes. Our team felt the name for the tenant network and the domain associated with the tenant, should match this project (seems reasonable). However, our SDN implementation doesn’t like dots in network names.

While we could simply require specifying both values: one with component labels separated with dots, and the other separated with dashes. However, wouldn’t it be more reliable to simply accept the project name and automatically convert this name into a network name with dot separators?

When I first started playing with Heat, I was expecting that I could take the project parameter, and convert it to the value I need with some sort of plugin/function thingie, like: 

properties:
  name:
    get_param: project
    convert: dots_to_dashes  # Warning: This isn't real

I would hope I could write this dots_to_dashes in some coding language like Python. However, this is whimsical speculation, as Heat current doesn’t allow this, at least not easily.

Instead, Heat has some crude abilities in manipulating and modifying data values into new values, so to spare the user from entering slightly redundant parameters, we could convert a project name parameter to a network name with the following: 

resources:
  virtual_dns:
    type: OS::ContrailV2::VirtualDns   # <- TODO: Change to OVS
    properties:
     name:
       str_replace:
         template: vdns-project
         params:
           project:
             list_join:
               - '-'
               - { str_split: ['.', { get_param: project }]}

Really.

Embedding logic in a declarative template seems problematic to me. Especially if the goal of Heat Templates is to be declarative.

Most editors can analyze code to detect some semantic errors along with syntax problems in real programming languages. But while my editor can analyze this YAML for syntax correctness, it can’t do anything about validating, or even parsing the code-like logic. How could we verify this code is correct?

However, I’m my case, this situation gets worse. Since each of my tenants can have multiple networks, and each virtual network requires multiple properties.

So now my template needs to repeatably create and associate multiple networks. How quickly can you read the following code? 

resources:
  virtual_network:
    type: OS::ContrailV2::VirtualNetwork
    properties:
      name: { get_param: [networks, { get_param: index }, name] }
      fq_name:
        list_join:
          - ':'
          - - get_param: domain
            - get_param: project
            - get_param: [networks, { get_param: index }, name]
      route_target_list:
        route_target_list_route_target:
          - list_join: [':',['target', { get_param: route_target }]]
      project:
        list_join:
          - ':'
          - - { get_param: domain }
            - { get_param: project }
      network_ipam_refs: [{ get_param: network_ipam_ref }]
      network_ipam_refs_data:
        - network_ipam_refs_data_ipam_subnets:
            - network_ipam_refs_data_ipam_subnets_subnet:
                network_ipam_refs_data_ipam_subnets_subnet_ip_prefix:
                  get_param: [networks, { get_param: index }, prefix]
                network_ipam_refs_data_ipam_subnets_subnet_ip_prefix_len:
                  get_param: [networks, { get_param: index }, size]

TODO: Replace all references of Contrail in the example above.

Validating Templates

The Heat system is both expansive, flexible and complex. It allows us to keep our environment files simple for our users, at the expense of embedding sophisticated logic inside our templates for massaging data values.

You can specify the --dry-run option, and then visually parse the resulting table: 

+-----------------------+-------------------------------------------+
| Field                 | Value                                     |
+-----------------------+-------------------------------------------+
| description           | Template to create a Nova custom flavor.  |
| parameters            | {                                         |
|                       |   "OS::project_id": "046050ab3c1248c3b6b",|
|                       |   "OS::stack_name": "my-flavor-1",        |
|                       |   "OS::stack_id": "None"                  |
|                       | }                                         |
| resources             | [                                         |
|                       |   {                                       |
|                       |     "resource_name": "nova_flavor",       |
|                       |     "resource_identity": {                |
|                       |       "stack_name": "my-flavor-1",        |
|                       |       "stack_id": "None",                 |
|                       |       "path": "/resources/nova_flavor"    |
|                       |     },                                    |
|                       |     "description": "",                    |
|                       |     "stack_identity": {                   |
|                       |       "stack_name": "my-flavor-1",        |
|                       |       "stack_id": "None",                 |
|                       |       "tenant": "046050ab3c1248c3b6b35a9",|
|                       |       "path": ""                          |
|                       |     },                                    |
|                       |     "stack_name": "my-flavor-1",          |
|                       |     "resource_action": "INIT",            |
|                       |     "resource_status": "COMPLETE",        |
|                       |     "properties": {                       |
|                       |       "disk": 0,                          |
|                       |       "name": "custom-tiny.m1",           |
|                       |       "ram": 512,                         |
|                       |       "ephemeral": 1,                     |
|                       |       "vcpus": 1,                         |
|                       |       "extra_specs": null,                |
|                       |       "swap": 0,                          |
|                       |       "rxtx_factor": 1.0,                 |
|                       |       "is_public": true,                  |
|                       |       "flavorid": null,                   |
|                       |       "tenants": []                       |
|                       |     },                                    |
|                       |     "resource_type": "OS::Nova::Flavor",  |
|                       |     "metadata": {}                        |
|                       |   }                                       |
|                       | ]                                         |
| stack_name            | my-flavor-1                               |
| template_description  | Template to create a Nova custom flavor.  |
+-----------------------+-------------------------------------------+

An initial approach to validating templates is compare the output of a dry-run to the results of a known, good state. This means that every template change requires a test change, and since engineers are quite lazy, the visual validation gets sloppy or ignored.

Heat Components

The --dry-run option has merit, as we really can’t analyze a template without rendering it with some OpenStack state. To do this, we take a template that defines the stuff we want and smoosh it with parameters, and pass it along the Heat components as shown in this diagram:

testing-heat-templates-1.png

Heat has four main components:

heat
a CLI communicates with heat-api
heat-api
provides a ReST API to heat-engine
heat-api-cfn
AWS-style Query API
  • compatible with AWS CloudFormation
  • also sends processed requests to heat-engine
heat-engine
main work of orchestrating, by:
  • compiling and performing templates
  • provides events back

Rendering Heat Templates

A better validation strategy would be to create a collection of Python Nose tests that validated specific code logic embedded in the template.

While this level of validation could easily be called “unit testing”, as shown in the section above, rendering a template requires a running OpenStack cluster with Heat support. Tests that require infrastructure are categorically not unit tests.

That said, let’s see what level of benefit we get from this approach. This next section is more like a workshop, so get ready to fire up Python, and follow along at home….

Begin by starting your virtual environment and installing the Python client library: 

pip install python-heatclient

Now, let’s start the Python REPL interpreter, or better yet, bpython, and copy and paste the code in the following sections, substituting where appropriate.

Connect to Keystone

Before we can connect to the Heat server, we need to get a token from Keystone: 

from keystoneauth1.identity import v3
from keystoneauth1 import session

hostname = "your.openstack.controller"
auth_info = {
    "auth_url": "https://%s:5000/v3" % hostname,
    "username": "admin",
    "password": "mypassword",
    "project_name": "admin",
    "user_domain_name": "Default",
    "project_domain_name": "Default"
}
auth = v3.Password(**auth_info)
sess = session.Session(auth=auth, verify=False)

This code creates a Python dictionary structure containing the required credentials and passes that to Keystone. The sess variable contains the token we need.

Connect to the Heat Server

Let’s import and instantiate a Heat client. Of course, in a source file, the import line would go at the top of the file, but in a REPL, this line needs to be entered before it is used: 

from heatclient.client import Client

heat = Client('1', service_type='orchestration', session=sess)

At this point, we can use this heat connection to get a list of the stacks using this comprehension: 

[stack for stack in heat.stacks.list()]

Chances are good that this returns an empty array, unless you’ve already been creating Heat stacks.

Parsing the Environment and Template Files

We now want to call the Heat client’s preview function with the environment parameters and the template file, which for our prototyping, we’ll just place into variables. 

templateFile = "virtual_dns.yaml"
envFiles = ["my-environment.yaml"]

To acquire the contents of the template file(s), lets use a function from template_utils: 

from heatclient.common import template_utils

tpl_files, template = template_utils.process_template_path(templateFile)

The tpl_files is supposed to be references to the files themselves, but it always seems to be empty, so we’ll ignore it. The template variable contains the contents, which is what we need.

Next, parse all the environments, and get the combined contents in the env variable. Again, we’ll use a function from the template_utils: 

env_files, env = template_utils.process_multiple_environments_and_files(env_paths=envFiles)

Similarly, the env_files variable is supposed to return a list of the “good” environment files, but this too, always seems to be an empty list. The env variable is what we need.

Previewing the Results

Create a fields hash to contain all the arguments to the preview function. This takes the contents of the environment, as well as the contents of the template files. 

fields = {
    'stack_name': 'test-stack',
    'template': template,
    'environment': env
}

And ship the entire payload over to the Heat API’s preview endpoint using the preview function: 

results = heat.stacks.preview(**fields)

This is pretty much the same as this curl call: 

curl -g -i -X POST https://your.openstack.controller:8004/v1/046050ab3c1248c3b6b35a9/stacks/preview -H "User-Agent: python-heatclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}3ad83cda7da55770f1a027e080301a2b" -d $PAYLOAD

Where the $PAYLOAD variable is a JSON hash that contains the following keys: 

{
    "files": {},
    "disable_rollback": true,
    "parameters": {},
    "stack_name": "test-stack",
    "environment": {
        "parameters": {
            "project": "foobar"
        }
    },
    "template": {
        // ... JSON structure representing the Template
    }
}

Since both the template and the environment are really just hashmaps, er… dictionary in Python parlance, we can read and send them in any way we’d like. We don’t have to use those utility functions. You just need to have a data structure representing the constituent components of the template file, like: 

fields = {
    'stack_name': 'test-stack',
    'environment': {'parameters': {'name': 'foobar'}},
    'template': {
        'heat_template_version': '2015-10-15',
        'description': 'HOT template to create a tiny flavor',
        'parameters': {
            'name': {
                'type': 'string',
                'description': 'The name of the flavor'
            }
        },
        'resources': {
            'nova_flavor': {
                'type': 'OS::Nova::Flavor',
                'properties': {
                    'ephemeral': 1,
                    'is_public': True,
                    'name': 'xp.m1.tiny',
                    'ram': 512,
                    'vcpus': 1
                }
            }
        }
    }
}

Regardless of how we create the template and the environment, what is returned from the preview function is a Stack object: 

from heatclient.v1.stacks import Stack

We see that with code like: 

results.__class__.__name__

Returns: 

'Stack'

The Stack class has a to_dict method that can be used to easily walk down the results or making standard hashmap, er, dictionary comparisons. For instance: 

results.to_dict().get('description')

Returns: 

'HOT template to create a tiny flavor'

Testing Heat Templates

We should now have enough information to create a testing framework that renders a template with some parameters, and then asserts on the results of the returned Stack instance.

I envision being able to writing a UnitTest class (which would extend from a helper class) and having some class methods available, for instance: 

class VirtualNetworkTest(HeatTestBase):
    file_to_test = HeatTestBase.get_template("virtual_network")

    def test_default_template(self):
        params = {
            'name': 'foobar',
            # ... etc. ...
        }
        results = self.render('net-stack', self.file_to_test, params)

        # After rendering the template into a Stack, we really just
        # want to validate the first (and only) resource:
        resource = results.resources[0]
        properties = resource['properties']

        self.assertEquals(params['networks'][0]['name'], properties['name'])
        # ... etc. ...

Well, I’ve got that written, and just need to talk my company into sharing the source code… watch this space.