Using YAML and Jinja to Create Network Configurations

In a previous post, I went over the basics of using Jinja to create templates for network configurations. This time I will be showing some examples of using YAML to store information about our network devices and using it to feed information to jinja.

What is YAML?

YAML is a data serialization language that is also very easy for humans to read. Often used to store data or information, such as settings for a program. Of course, it also happens to be quite good for storing network information, specifically configuration items for a device. You can think of it like a folder structure in an OS like windows or macOS. You may have a folder for documents, which contains another folder for photos which may contain another folder for vacation photos specifically. For us as network engineers this would translate to a container labeled “devices”, which has another container labeled “R1”, “R2”, “R3” and so on. Even further each of those containers may have yet another container for specific information about “interfaces”, “snmp”,”routing protocol” etc.

A visual aid using folders on my macbook might look like this:

So you can see how we start very broad at devices, then move into specific devices by name such as R1, then further into interfaces belonging to R1, then once again into a specific interface Ethernet1/1, in which we finally find the properties, attributes, configuration, or whatever you want to call it for Ethernet 1/1.

Now here’s what this looks like in YAML:

devices:
  - name: "R1"
    interfaces:
      - name: "Ethernet1/1"
        state: "enabled"
        ip: "10.0.0.1"
        mask: "255.255.255.0"
        qos_policy: "SHAPE_200MB"
      - name: "Loopback0"
        state: "enabled"
        ip: "1.1.1.1"
        mask: "255.255.255.255"
    bgpasn: "65000"
    bgp_neighbors:
      - ip: "10.0.0.2"
        remote_as: "65001"
  - name: "R2"
    interfaces:
      - name: "Ethernet1/1"
        state: "enabled"
        ip: "10.0.0.2"
        mask: "255.255.255.0"
        qos_policy: "SHAPE_200MB"
    bgpasn: "65001"
    bgp_neighbors:
      - ip: "10.0.0.1"
        remote_as: "65000"

Still pretty easy to read isn’t it? Not only is it easy for us to read, but its also really easy for python to parse. If you’ve tinkered with automation, you have probably noticed it can be pretty difficult to parse CLI output from a show command in python, even though it’s really easy for us as humans to read. Having data structured in such a way that we can understand and use programmatically is quite useful.

In this example we have created an object to hold all of our devices by writing “devices:” with a colon. The next line leads with two spaces and a “-” to indicate we are beginning a new list which will represent a single device with the name “R1”. This list will then contain an object to hold this devices interfaces, which we indicate with “interfaces:”, the name of the object following by just a colon. Then, we are creating another list within this object to describe “Ethernet1/1” and all of its configuration items, as well as a second list for another interface “Loopback 0”. Its difficult to coherently describe nested objects with multiple layers, hence giving multiple examples in hopes one of them will stick. Formatting is pretty important here, as the indentation level indicates a sort of parent child relationship. So, the individual interfaces are children of object “interfaces”, which is part of a list describing a particular device, which is a child of the overall object “devices”.

Now, theres one more way I want to present YAML, and thats how it looks after python has consumed it. To use YAML in python you will need to install the PyYAML module using pip. Once thats done import the module, open the file containing your YAML, and use the yaml.safe_load() function to read it in like this:

import yaml
from pprint import pprint as p

with open("devices.yaml") as file:
    device_info = yaml.safe_load(file)

p(device_info)

I used prettyprint to get a better formatted output to display here, but its not required of course. Heres the output:

If you are pretty familiar with python, you will probably now realize how using YAML provides python with good data to parse. Its your basic python data structure: lists and dictionaries. You can easily loop through any lists and use specific values from the dictionary key value pairs to plug in variables into a config. Also remember that some data types in python are unordered, so its not necessarily going to come out in the same way it went in. But that doesn’t really matter, the data is still accessible all the same.

There are many ways to you could format your data with YAML, what I’ve written here is just one way. You can use it to describe a single device or many devices. There are even more tools within YAML available than what ive shown here, I encourage you to experiment with YAML on your own to find what works best for you.

Lets quickly go through an example of using YAML to feed a jinja template. Check out this jinja template below:

{{ device }}

{% for int in interfaces -%}
interface {{ int.name }}
 ip address {{ int.ip }} {{ int.mask }}
 service-policy output {{ int.qos_policy }}
 {% if int.state == "enabled" -%}
 no shutdown
 {% else -%}
 shutdown
 {% endif -%}
!
{% endfor -%}
router bgp {{ bgpasn }}
{% for neighbor in bgp_neighbors -%}
 neighbor {{ neighbor.ip }} remote-as {{ neighbor.remote_as }}
{% endfor -%}

This template uses a few new tricks I didn’t show in my previous post. In Jinja, you can actually use some common python procedures such as for loops and if statements. These are indicated by opening with {% and ending with %}. You may also notice a “-” at the end of the for loops and if statements. This is to prevent that line from showing a blank line in your output and messing up the formatting. Jinja does require you indicate the end of your for loops and if statements, which python its self does not require.

The other thing thats a little different here is our variables. For example we are using “int.name”. It will become a little more clear when I show the python code below, but this is because we are passing lists and dictionaries into jinja instead of plain string variables. In the first for loop, we are iterating over “interfaces” which is a list of dictionaries. We can then extract the values we need to complete our configuration by access the data stored in each key of the dictionary. So “int.ip” will access the ip address for the interface. In python code, this would look something like “print(int[‘ip’])”.

So now, we have our data in a YAML file named “devices.yaml” and we have our jinja template in a file named “template.j2”. Lets look at how to use these in python:

#import yaml and jinja
import yaml
from jinja2 import Template

#read your yaml file
with open("devices.yaml") as file:
    devices = yaml.safe_load(file)

#read your jinja template file
with open("template.j2") as file:
    template = Template(file.read())

#iterate over the devices described in your yaml file
#and use jinja to render your configuration
for device in devices["devices"]:
    print(template.render(device=device["name"],interfaces=device["interfaces"], 
          bgpasn=device["bgpasn"], bgp_neighbors=device["bgp_neighbors"]))

The YAML object “devices” begins as a dictionary. When you iterate over it, you get a list. That list contains dictionaries. So when we pass our variables into jinja using template.render(), we need to be mindful of the data type we are passing in and what our jinja template is expecting. In our first iteration, our list contains a dictionary with they key “name”. So we can pass the value of that key in directly. Same for bgpasn. For interfaces, our jinja template is expecting an iterable object, so passing in devices[“interfaces”] will give it a list of dictionaries to iterate over. The same is true for the bgp_neighbors variable. Heres the final output:

 R1

interface Ethernet1/1
 ip address 10.0.0.1 255.255.255.0
 service-policy output SHAPE_200MB
 no shutdown
 !
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
 service-policy output 
 no shutdown
 !
router bgp 65000
neighbor 10.0.0.2 remote-as 65001

R2

interface Ethernet1/1
 ip address 10.0.0.2 255.255.255.0
 service-policy output SHAPE_200MB
 no shutdown
 !
router bgp 65001
neighbor 10.0.0.1 remote-as 65000

YAML is one of the many ways we can store information about our network devices in such a way python can easily parse. You can write YAML directly yourself, or tie to a provisioning system or perhaps a web gui for users to input data while python takes the user input and creates a YAML file in the background. Here we used it to feed a jinja template, however you might even want to write YAML and convert it to JSON or XML for use directly with some devices.

As always, theres more to discover but this is a blog post and not a text book. So I hope this has been a useful demonstration to get you started with YAML.