Extract Config Context YAML information from NetBox

We have a NetBox instance which we use to store information about virtual machines. The Config Context tab for a virtual machine object is populated with a YAML file structured as such:

stuff:
- hostname: myserver
  os: Linux
  os_version: RHEL 8.1
  network:
  - ip_address: 192.168.2.3
    network: ABC
    gateway: 192.168.2.1
  server_type: XYZ
  other_data: Foobar

The same data is also available in JSON format.

I am doing extraction of the standard NetBox fields to a CSV file via the following Python script:

config = configparser.ConfigParser()
netbox_token = config.get('NetBox', 'token')
netbox_api_base_url = config.get('NetBox', 'api_base_url')
netbox_devices_endpoint = config.get('NetBox', 'devices_endpoint')
nb = pynetbox.api(netbox_api_base_url, token=netbox_token)
nb.http_session.verify = True

vms = nb.virtualization.virtual_machines.all()
csv_file_vms = 'netbox_vms.csv'

with open(csv_file_vms, mode='w', newline='') as csv_file:
    csv_writer = csv.writer(csv_file)
    csv_writer.writerow(['Name', 'Status', 'Site', 'VCPUs', 'Memory (MB)', 'Disk (GB)', 'IP Address'])
    for vm in vms:
        csv_writer.writerow([vm.name, vm.status, vm.site, vm.vcpus, vm.memory, vm.disk, vm.primary_ip])

How do I modify the writerow() function to add the data stored in the Config Context, e.g. the os_version field?

Solution

Preface

You will need to add a dedicated column in the first call to writerow, and access the relevant VM attribute in consecutive calls inside the for loop. As I assume your question revolves more around attribute access rather than how to write CSVs - I'll dive deeper into the former;

Config Context

The Config Context can be accessed via the vm.config_context attribute. The provided YAML example translates to a dictionary with one key stuff, which points to a list of items. Each item in that list is a dictionary having keys such as hostname, os_version, and network. The values of the first keys mentioned are strings, but the value of network is yet another list. Following is a practical demonstration for accessing such attributes for the given example, while assuming that all list types have one (and only one) item:

>>> vms = nb.virtualization.virtual_machines.all()
>>> for vm in vms:
...     print(f"{vm.name=}")
...     print(f"{vm.vcpus=}")
...     print(f"os_version: {vm.config_context['stuff'][0]['os_version']}")
...     print(f"network/gateway: {vm.config_context['stuff'][0]['network'][0]['gateway']}")
...     print(f"other_data: {vm.config_context['stuff'][0]['other_data']}")
...     print("======================")
...
vm.name='VM1'
vm.vcpus=1.0
os_version: RHEL 9.2
network/gateway: 192.168.2.1
other_data: BarFoo
======================
vm.name='vm2'
vm.vcpus=2.0
os_version: RHEL 8.1
network/gateway: 192.168.2.1
other_data: Foobar
======================
>>>

Solution

A complete answer to your question would include the adaptations required for writing the CSV, so here goes:

>>> vms = nb.virtualization.virtual_machines.all()
>>> csv_file_vms = 'netbox_vms.csv'
>>> with open(csv_file_vms, mode='w', newline='') as csv_file:
...     csv_writer = csv.writer(csv_file)
...     csv_writer.writerow(['Name', 'Status', 'Site', 'VCPUs', 'Memory (MB)', 'Disk (GB)', 'IP Address', 'OS Version'])
...     for vm in vms:
...         csv_writer.writerow([vm.name, vm.status, vm.site, vm.vcpus, vm.memory, vm.disk, vm.primary_ip, vm.config_context['stuff'][0]['os_version']])
...
68
37
37
>>>
>>> with open(csv_file_vms) as input_file:
...     print(input_file.read())
...
Name,Status,Site,VCPUs,Memory (MB),Disk (GB),IP Address,OS Version
VM1,Active,,1.0,1023,2049,,RHEL 9.2
vm2,Active,,2.0,1025,2051,,RHEL 8.1

>>>

Additional thoughts

If you can change the YAML structure, removing the first dash - (from in front of the hostname field in your example) can make attribute access simpler, and also represent the reality more accurately; consider the following YAML as a config context for a router:

stuff:
  hostname: MyRouter
  os: Linux
  os_version: RHEL 8.1
  network:
  - ip_address: 192.168.1.1
    name: ABC
  - ip_address: 172.16.1.1
    name: DEF

Handling such case in code can look like this:

>>> for vm in routers:
...     print(f"{vm.name=}")
...     print(f"{vm.vcpus=}")
...     print(f"{vm.config_context['stuff']['os_version']=}")
...     print("Routing:")
...     for network in vm.config_context['stuff']['network']:
...         print(f"{network['name']}: {network['ip_address']}")
...
vm.name='Router1'
vm.vcpus=None
vm.config_context['stuff']['os_version']='RHEL 8.1'
Routing:
ABC: 192.168.1.1
DEF: 172.16.1.1
>>>

You may not have to deal with multiple IP addresses in your situation, but you won't need to handle the possibility of a single VM configured (probably erroneously) with multiple OSes.

Note: This question involves interoperability between client and server functions. I've tested this on python 3.9.18, netbox 3.6.5, and pynetbox 7.2.0. Other versions may behave differently.