1 Architecture
Louis Raymond edited this page 2026-08-19 13:48:46 -04:00

Architecture

The problem

OPNsense has no official cloud-init support. Running stock cloud-init inside the guest doesn't help either: its FreeBSD network renderer writes addressing into /etc/rc.conf, which OPNsense's own configuration store (/conf/config.xml) and GUI never read. The network technically comes up, but OPNsense itself has no idea — the box is misconfigured from its own point of view.

/conf/config.xml is the single source of truth for everything OPNsense does: interfaces, addressing, VLANs, users, SSH/API access, whether the setup wizard should run. Anything that doesn't end up there doesn't count.

The approach

This project builds an OPNsense base image with a custom first-boot hook (opnseed) that parses a standard Proxmox NoCloud cloud-init seed — user-data / meta-data / network-config, exactly what the Proxmox GUI's Cloud-Init tab, qm set --ipconfig0/--sshkeys/--ciuser, or OpenTofu's initialization block produce — and renders it directly into /conf/config.xml. No cloud-init package ever runs inside the guest.

An optional opnsense: key inside the same user-data document carries whatever standard cloud-init schema can't express (VLANs, explicit WAN/LAN/OPTn role overrides, API keys — see Extension-Reference). Real cloud-init ignores unknown top-level keys, so a user-data document carrying this extension is still a fully valid #cloud-config. The common case — WAN + LAN, DHCP or static, an SSH key — needs none of it.

Pipeline

flowchart LR
    subgraph build [Image build]
        direction LR
        A["OPNsense ISO<br/>(DVD amd64)"] --> B["Packer + QEMU<br/>(unattended install)"]
        B --> C["base/config.xml +<br/>build/opnsense-*.qcow2"]
    end

    C -- "imported as a<br/>Proxmox template" --> D

    subgraph deploy [Deployment]
        direction LR
        D["Proxmox Cloud-Init<br/>(GUI / qm / tofu)"] --> E["VM clone, native<br/>Cloud-Init drive (NoCloud)"]
        E --> F["OPNsense boots,<br/>opnseed merges the seed<br/>into config.xml, reboot-free"]
    end

Image build (packer/): boots the OPNsense installer under QEMU, scripts it through an unattended UFS install, installs the opnseed Python package and wires it into rc.syshook.d/early/, then extracts the resulting config.xml as base/config.xml — the known-good template every future merge starts from. See Development for the build process and its installer-automation gotchas.

Runtime (opnseed/, guest/): on every boot, the early rc.syshook stage — which runs before rc.bootup parses config.xml — mounts the cloud-init seed volume (if present), parses it, resolves each interface's declared MAC to a real device via ifconfig -a, and writes the result into /conf/config.xml. Never generates XML from scratch: it only ever edits the known-good base tree, so every OPNsense default the base image already carries stays untouched unless a transformer explicitly changes it.

Contracts that matter

  • Merge, not generate. opnseed/src/opnseed/merge.py loads base/config.xml and applies a pipeline of pure (tree, spec) -> tree transformers (transformers/system.py, users.py, interfaces.py, apikeys.py). Nothing downstream of spec.py's dataclasses knows about cloud-init's on-the-wire formats.
  • MAC-based device resolution, always. A seed's declared interface name (net0, eth0, ...) is meaningless to the running OS. opnseed only ever binds <if> to a real device name resolved from that interface's MAC address against the guest's actual hardware (ifconfig -a) — never trusts a name from the seed directly. This is what prevents the interactive console assignment wizard from ever triggering on a seeded boot.
  • Never fail the boot. No seed present → log one line, leave the template's baked-in default config untouched (DHCP everywhere, SSH by key only, no wizard). Any error after a seed is found → log loudly and leave the existing config.xml untouched, rather than risk writing something that could trip the console wizard.
  • Idempotent by instance-id. meta-data's instance-id is compared against /conf/.opnseed-instance-id before doing any work; a reboot with the same seed attached is a no-op.
  • Atomic write, with backup. The rendered config is written to a temp file and renamed into place; the previous config.xml is copied to /conf/backup/ first.

Division of responsibility

opnseed owns bring-up: interfaces, addressing, identity, initial SSH/API access — everything the setup wizard would otherwise ask for by hand. It deliberately does not own firewall rules, VPN configuration, or package management; that's policy, and belongs in a config-management tool (e.g. Ansible via OPNsense's REST API) driven by the API key opnseed can seed on first boot. See Usage.