Pagination, next links, status filters & building device lists

Follow every page

Lesson 17 of 20 · 15 min · Module 5: APIs and the source of truth

APIs return long lists in pages. A script that reads only the first page works in a small lab and silently misses devices in production.

Start this lesson →

Free. No installation: you write and run the code in your browser, and it is checked automatically.

By the end, you’ll be able to…

  • Read every page of a paginated API
  • Filter on the status the source of truth gives you
  • Turn API records into Netmiko dictionaries

The idea

NetBox returns at most a page of results, plus next: the full URL of the following page, or None on the last page. Keep requesting next until it is None. The link already contains the query, so do not pass the parameters again. Devices also have a status: only active devices should be automated, so a device marked offline for maintenance is left alone. The platform slug in the practice inventory matches the Netmiko driver name, which many teams do on purpose.

url = f"{URL}/api/dcim/devices/"
while url:
    page = requests.get(url, headers=HEADERS).json()
    print(len(page["results"]), "devices on this page")
    url = page["next"]

01One page is not everything

The practice API, like NetBox, returns a page of results. count tells you how many exist in total.

import os
import requests

URL = os.environ["INVENTORY_URL"]
HEADERS = {"Authorization": f"Token {os.environ['INVENTORY_TOKEN']}",
           "Accept": "application/json"}

page = requests.get(f"{URL}/api/ipam/ip-addresses/", headers=HEADERS).json()
print(page["count"], len(page["results"]))
print(page["next"])
Output
43 5
https://inventory.lab/api/ipam/ip-addresses/?limit=5&offset=5

02Follow next until it is None

Each answer points to the next page. The loop ends on the page whose next is None.

url = f"{URL}/api/dcim/devices/"
while url:
    page = requests.get(url, headers=HEADERS).json()
    print([d["name"] for d in page["results"]])
    url = page["next"]
Output
['core1', 'sw1', 'sw2', 'sw3', 'sw4']
['sw5']

03Status is a small dictionary

value is meant for code, label for people. Compare the value.

response = requests.get(f"{URL}/api/dcim/devices/", headers=HEADERS,
                        params={"name": "sw4"})
sw4 = response.json()["results"][0]
print(sw4["status"])
print(sw4["status"]["value"] == "active")
Output
{'value': 'offline', 'label': 'Offline'}
False

Your turn

Write get_all(path, params=None) that returns the results of every page for an API path such as /api/dcim/devices/. Use it to build netmiko_devices: a dictionary mapping the name of each active device to its Netmiko dictionary (device_type from the platform slug, host from the primary IPv4 address, username and password from the environment). Then connect to each one and print its prompt.

What the checks look for

  • get_all returns every page
  • netmiko_devices has every active device
  • sw5 is built from the API data
  • Connected to each active device
Try it in the editor →

Take this with you

Follow next until it is None, and filter on the status value from the source of truth.

Go a little further

Filter on the server with params={"status": "active"} instead of in Python, and compare how many requests each version needs.

Learn Network Automation for free

20 short lessons with real code and instant checks.

See the whole course →