Using Heat Templates

The following are some notes I use to introduce myself to OpenStack’s Heat project, and how to get things done with its Templates.

What is Heat?

According to their web page, Heat is the main project in the OpenStack Orchestration program, which attempts to create an interface, for managing the entire life-cycle of infrastructure resources within OpenStack clouds.

Heat is a way to automate (and orchestrate) the creation of cloud components. In other words, instead of calling the actions associated with OpenStack’s CLI, you would define the things you want to be, and Heat makes it so.

Like a typical four-letter technology, Heat has lot of esoteric terms, and a reader may appreciate a mini-glossary before the deeper dives on this page:

Resource

A resource is any cloud object that Heat either creates, modifies or removes, for instance:

Simple things like flavors and volumes
Virtual machine instances
Complex networks, routers, subnets, security groups, etc.

Stack

collection of resources

Template

defines a Stack, e.g. a bunch of resources

Parameters

input to the template during deployment

Output

provides information back to user

Heat Templates

Heat Orchestration templates (HOT) are declarative configurations for the Heat orchestration tool, used to automate the creation of many OpenStack resources. These are described on the Heat Wiki page as:

templates in the form of text files that can be treated like code.

I don’t know about you, but statements of instructional code promiscuously covorting with declarative definitions, makes me nervous, but let’s timidly press forward and demonstrate templates with a simple example.

Create a Flavor Resource

A HOT is defined in YAML. A basic example to define a Nova flavor might look like this:

heat_template_version: 2016-04-08

description: Template to create a Nova custom flavor.

resources:
  nova_flavor:
    type: OS::Nova::Flavor
    properties:
      ephemeral: 1
      is_public: true
      name: xp.m1.tiny
      ram: 512
      vcpus: 1

Fairly obvious when you compare this to the parameters you’d give the openstack flavor create command line call.

Note that each HOT template must include the heat_template_version key with a valid version, e.g. 2015-10-15 (see Heat template version for a list of all versions).

Try it out:

openstack stack create -t xp-flavor.yaml my-flavor

The -t specifies the template, and you need to name the collection of resources you create with this call. This returns:

+---------------------+------------------------------------------+
| Field               | Value                                    |
+---------------------+------------------------------------------+
| id                  | 010a2f59-6064-4d59-86c1-8564adc348e6     |
| stack_name          | my-flavor                                |
| description         | Template to create a Nova custom flavor. |
| creation_time       | 2017-06-27T20:39:16                      |
| updated_time        | None                                     |
| stack_status        | CREATE_IN_PROGRESS                       |
| stack_status_reason | Stack CREATE started                     |
+---------------------+------------------------------------------+

Calling:

openstack stack show my-flavor

(or even just a openstack stack list) will probably show the status_status in a complete (or failed) state.

The nature of declarative definitions means that we are specifying what should be, not what should be done. If we were to re-do our request, nothing should be done if the flavor already exists, but if you request another create, we’ll get this error:

ERROR: The Stack (my-flavor) already exists.

However, in this case, you can re-issue all the update commands you want:

openstack stack update -t xp-flavor.yaml my-flavor

Which goes through any changes in the YAML file and updates the resource if anything changed, or just leaves it alone if it is the same:

+---------------------+-----------------------------------------------------+
| Field               | Value                                               |
+---------------------+-----------------------------------------------------+
| id                  | c2a2c1f4-9107-4c6f-b6b9-1d3a6a1d6a11                |
| stack_name          | my-flavor                                           |
| description         | Simple template to deploy a single compute instance |
| creation_time       | 2017-07-17T10:51:30                                 |
| updated_time        | 2017-07-17T10:52:18                                 |
| stack_status        | UPDATE_IN_PROGRESS                                  |
| stack_status_reason | Stack UPDATE started                                |
+---------------------+-----------------------------------------------------+

After running this update command a few times, I still only have a single flavor.

Create an Instance

Heat allows you to create an instance, and with the knowledge gained in the previous section, we should be able to not only define the instance, but also define each of the parts it may depend on:

heat_template_version: 2016-04-08

description: Template to create an tiny Instance

resources:
  my_flavor:
    type: OS::Nova::Flavor
    properties:
      ephemeral: 1
      is_public: true
      name: my-tiny.m1
      ram: 512
      vcpus: 1
  my_image:
    type: OS::Glance::Image
    properties:
      name: cirros
      container_format: bare
      disk_format: qcow2
      min_disk: 0
      min_ram: 0
      location: https://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img
  my_key:
    type: OS::Nova::KeyPair
    properties:
      name: heat_key
      save_private_key: true
  my_instance:
    type: OS::Nova::Server
    properties:
      image: cirros
      flavor: xp.m1.tiny
      key_name: heat_key

How do I know things like OS::Glance::Image is what we need to specify the image? Well, you can query your system like:

openstack orchestration resource type list | grep Image

Next, how do you know what properties are available for a particular type? Let’s query the type with:

openstack orchestration resource type show OS::Glance::Image

As expected, the available properties are similar to if you were to run the image create command:

glance --os-image-api-version 2 image-create --architecture x86_64 \
 --protected False --min-disk 0 --disk-format qcow2 --min-ram 0 \
 --file ./cirros-0.3.5-x86_64-disk.img

Note: I still haven’t figured out the trick to get Heat to generate the key, and needed to create the key manually on the command line:

nova keypair-add heat_key > heat_key.priv

When you kick off this Heat template:

openstack stack create -t xp-instance.yaml my-instance

We get a table giving us the current status:

+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| id                  | 3d209955-e2d5-4df9-95ac-a34971f129fb |
| stack_name          | my-instance                          |
| description         | Template to create an tiny Instance  |
| creation_time       | 2017-07-19T09:53:47                  |
| updated_time        | None                                 |
| stack_status        | CREATE_IN_PROGRESS                   |
| stack_status_reason |                                      |
+---------------------+--------------------------------------+

Check on the resulting build status with:

openstack stack list

Oh noes, something went wrong:

+--------------------------------------+-------------+-----------------+---------------------+---------------------+
| ID                                   | Stack Name  | Stack Status    | Creation Time       | Updated Time        |
+--------------------------------------+-------------+-----------------+---------------------+---------------------+
| 3d209955-e2d5-4df9-95ac-a34971f129fb | my-instance | CREATE_FAILED   | 2017-07-19T09:53:47 | None                |
| c2a2c1f4-9107-4c6f-b6b9-1d3a6a1d6a11 | my-flavor-1 | UPDATE_COMPLETE | 2017-07-17T10:51:30 | 2017-07-17T10:52:18 |
+--------------------------------------+-------------+-----------------+---------------------+---------------------+

Let’s investigate:

openstack stack show my-instance

Begins with these results:

+-----------------------+-------------------------------------------------------------------------------------------------+
| Field                 | Value                                                                                           |
+-----------------------+-------------------------------------------------------------------------------------------------+
| id                    | 5ba62fde-1b75-477e-ac47-59591abcdf39                                                            |
| stack_name            | my-instance                                                                                     |
| description           | Template to create an tiny Instance                                                             |
| creation_time         | 2017-07-19T10:05:59                                                                             |
| updated_time          | None                                                                                            |
| stack_status          | CREATE_FAILED                                                                                   |
| stack_status_reason   | Resource CREATE failed: HTTPBadRequest: resources.my_image: 400 Bad Request: External sources   |
|                       | are not supported: 'cirros-0.3.5-x86_64-disk.img' (HTTP 400)                                    |
| parameters            | OS::project_id: 5e3b4f4b513545dc80c0c63c0a85439e                                                |
|                       | OS::stack_id: 5ba62fde-1b75-477e-ac47-59591abcdf39                                              |
|                       | OS::stack_name: my-instance                                                                     |
...

Updates versus Create

One feature touched on above that I don’t like is that Heat isn’t completely declarative.

If it is the first time you are create all resources, things work fine, but if you are creating some of the resources in the template (like an image), but reusing other resources (like a flavor), you might get errors like:

Resource CREATE failed: Conflict: resources.my_flavor: Flavor with name my-tiny.m1 already
exists. (HTTP 409) (Request-ID: req-79250c9b-bb41-48d0-8eb4-65eb223b81b0)

How to work around this? One approach is to break up your template into parts for initially creating your reusable resources, like flavors and images, and then others for regular requestions, like instances and new users with their tenant projects.

Still more to come in this space.

Add a Name Parameter

Writing everything in the template limits its re-usability. Let’s suppose we wanted to create many similar instances, each with a different name.

Heat allows some values to be replaced by a parameter:

heat_template_version: 2016-04-08

description: Template to create an tiny Instance

parameters:
  name:
    type: string
    description: Name for the tiny instance

resources:
  my_instance:
    type: OS::Nova::Server
    properties:
      image: cirros
      flavor: xp.m1.tiny
      key_name: heat_key
      name:
        get_param: name

Try it out and let’s start two instances with the same template:

openstack stack create --parameter name=foobar -t xp-instance2.yaml my-instance-2
openstack stack create --parameter name=flingbaz -t xp-instance2.yaml my-instance-2

We can see the stacks we’ve created:

openstack stack list

Returns a table of information:

+--------------------------------------+---------------+-----------------+---------------------+---------------------+
| ID                                   | Stack Name    | Stack Status    | Creation Time       | Updated Time        |
+--------------------------------------+---------------+-----------------+---------------------+---------------------+
| c33baf4a-c5ac-4fd3-8219-f45c53fc833a | my-instance-3 | CREATE_COMPLETE | 2017-07-19T12:50:08 | None                |
| 98f15ca3-ac22-415e-931f-996282023b59 | my-instance-2 | CREATE_COMPLETE | 2017-07-19T10:43:16 | None                |
+--------------------------------------+---------------+-----------------+---------------------+---------------------+

Let’s see if they are actually created:

openstack server-list

Returns:

+--------------------------------------+----------+--------+----------+------------+
| ID                                   | Name     | Status | Networks | Image Name |
+--------------------------------------+----------+--------+----------+------------+
| c231fa58-4623-46a4-90bb-0fda05c06a2d | flingbaz | ACTIVE |          | cirros     |
| 059a54a2-8914-406b-bdb1-a77fd37f7e96 | foobar   | ACTIVE |          | cirros     |
+--------------------------------------+----------+--------+----------+------------+

Parameter Files

You can imagine a template having many substituted parameters. For instance, to create a virtual DNS, we may need to call our template with the following parameters:

openstack stack create --parameter domain=default-domain --parameter virtual_DNS_data_domain_name=cedev13.d001.pdx.eng --parameter virtual_DNS_data_record_order=round-robin --parameter fq_name=default-domain:vdns-cedev1-d001-pdx-eng --parameter virtual_DNS_data_default_ttl_seconds=5 --parameter name=vdns-demo -t virtual-vdns.yaml vdns-13

A bit wordy, eh?

Parameters like these could also be seen as a part of the definition, and instead of entering multiple --parameter entries, we could specify a single file that contains all the needed parameters in an environment file:

parameters:
  name: vdns-13
  domain: default-domain
  virtual_DNS_data_domain_name: cedev13.d001.pdx.eng
  virtual_DNS_data_record_order: round-robin
  fq_name: default-domain:vdns-cedev1-d001-pdx-eng
  virtual_DNS_data_default_ttl_seconds: 5

We would then combine that file with our template this way:

openstack stack create -e vdns-13.env -t virtual_dns.yaml vdns-13

So far, our example really demonstrates the purpose in wanting to use Heat, however, in Part two, I’ll demonstrate ways to get burned by Heat and minimizing the pain by testing the templates.