diff --git a/docs/caveats.md b/docs/caveats.md index f3b1d9afae..7b0b358208 100644 --- a/docs/caveats.md +++ b/docs/caveats.md @@ -62,6 +62,39 @@ GRE tunnel caveats: * IS-IS does not work over GRE tunnels * Arista cEOS might forward multiple copies of the packets received over GRE tunnels +(caveats-arcos)= +## Arrcus ArcOS + +ArcOS is a commercial NOS; there is no public container image. See the [ArcOS installation +notes](build-arcos) for the image, the first-boot bootstrap, the supported configuration modules, +and validation. Configuration is deployed with a bash script executed within the ArcOS container. + +* Initial device configuration sets the ArcOS interface MTU to the lab topology layer-3 MTU + (**interface.mtu**) plus 26. +* _netlab_ configures static routes and they are used for traffic forwarding, but the tested ArcOS + image does not report them in the OpenConfig RIB tree. +* While _netlab_ configures SR-MPLS, MPLS/LDP, and SRv6, the dataplane does not work in the + container image. +* BGP sessions come up roughly 70 seconds after the rest of the configuration; the BGP daemon + rejects session state changes until it has finished reading its boot-time configuration. The + integration tests declare that wait. OSPF and IS-IS are not affected. +* An OSPF ABR summarizes loopback host routes with the wrong prefix length, so a loopback placed in + a non-backbone area is not reachable from other areas. Loopbacks in area 0 and all inter-area + transit prefixes are unaffected. +* IS-IS IPv6 is single-topology; the tested image has no multi-topology (RFC 5120) knob. Devices + running multi-topology IS-IS -- which is how _netlab_ configures every other device -- will not + install ArcOS IPv6 routes, so IPv6 IS-IS works only in an all-ArcOS IS-IS domain. IPv4 IS-IS + interoperates normally. +* ArcOS computes the IPv4 VRRPv3 checksum over the IPv4 pseudo-header and has no knob to disable + it, while _netlab_ configures every other device the other way, so IPv4 VRRP does not interoperate + with other _netlab_ devices. A single ArcOS first-hop router works. IPv6 VRRP is unaffected. +* With static VXLAN flooding, the container image withdraws the head-end-replication entry shortly + after programming it, so an ArcOS VTEP cannot originate BUM traffic (receiving works). Use + `vxlan.flooding: evpn`. +* A discard (blackhole) static route pointing to a directly-connected host is displaced by the + connected route once that host's neighbor entry becomes reachable. Discard routes for prefixes + that are not directly connected work normally. + (caveats-aruba)= ## Aruba AOS-CX diff --git a/docs/labs/arcos.md b/docs/labs/arcos.md new file mode 100644 index 0000000000..80f7d91196 --- /dev/null +++ b/docs/labs/arcos.md @@ -0,0 +1,60 @@ +(build-arcos)= +# Installing Arrcus ArcOS + +netlab runs **Arrcus ArcOS** as a [containerlab](clab.md)-provisioned device. ArcOS is a +**native** containerlab kind (`arrcus_arcos`) -- no vrnetlab packaging -- and only the **clab** +provider is supported (no Vagrant box). + +## Container image + +ArcOS is a commercial NOS; there is no public image. Obtain an ArcOS container image and tag it as +the device expects (or point `clab.image` at your own tag): + +``` +defaults.devices.arcos.clab.image: arcos:8.2.1A.P2 +``` + +The device sets clab `kind: arrcus_arcos`. Verified against the **8.2.1A.P2** container image. + +## First-boot bootstrap + +The tested image boots with SSH, NETCONF, and gNMI disabled, and ArcOS refuses to enable +interfaces until the factory-default admin-user password is changed. netlab handles both +automatically with the native `netlab_start_exec` group variable (containerlab's post-start +`exec:`): it enables `ssh-server`, sets the admin-user password, and creates an AAA user before any +configuration is deployed. No manual steps are required. + +## Configuration deployment + +ArcOS uses netlab's native containerlab **"sh" config mode** +([Linux configuration scripts](../dev/config/deploy.md)): each configuration module is rendered into +`/config/netlab/NN-.sh` with a `#!/config/netlab/netlab-config.sh` shebang and deployed with +`docker exec`, which hands the rendered config to the mapped wrapper +`netsim/templates/provider/clab/arcos/netlab-config.j2` to load through `confd_cli` +(`load merge` / `commit`). No Ansible is used to deploy configuration -- this is the same +mapped-script pattern Juniper cRPD uses. `ansible_connection: docker` is retained only for the +validation/collect path. + +ArcOS also ships the official `arrcus.arcos` `network_cli` collection (the project-recommended +interactive model), but every published version hangs against this image; see the +[ArcOS caveats](caveats-arcos). No extra Ansible collection is required. + +## Supported configuration modules + +`initial`, `ospf` (v2/v3), `bgp`, `isis`, `vrf` (+ospf/isis/bgp), `vlan`, `lag`, `gateway` (VRRP), +`dhcp`/relay, `bfd`, `routing` (static/prefix-set/policy), `mpls` (LDP), `sr` (SR-MPLS via IS-IS), +`srv6`, `vxlan`, and `evpn` (L2VNI). See `netsim/devices/arcos.yml` `features:` for the +authoritative list; support level is **best-effort** (see [caveats](caveats-arcos)). + +VLANs use the native switched-VLAN model (`vlan ` plus `interface ... ethernet switched-vlan`), +and SVIs are named `vlan`. Routing-protocol instances inside a VRF are tagged with the VRF name, +because a protocol instance tag is a single global namespace across every network instance on this +build. + +## Validation + +`netlab validate` reads ArcOS device state over the same docker-exec path used to deploy config +(`show | display json | confd_cli`, parsed as OpenConfig JSON by +`netsim/validate/**/arcos.py`) -- netlab's standard device-side show-command validation, with no +SSH/NETCONF/gNMI and no `ansible` validation action. ArcOS is exercised with netlab's regular +[integration tests](../dev/integration-tests.md). diff --git a/docs/module/stp.md b/docs/module/stp.md index 4fb23db027..45b4ace843 100644 --- a/docs/module/stp.md +++ b/docs/module/stp.md @@ -12,6 +12,7 @@ The following table describes per-platform support of individual STP features: | Operating system | STP | MSTP | RSTP | Per-VLAN
RSTP | Enable
per port | | ------------------ |:---:|:---:|:---:|:---:|:---:| | Arista EOS[^EOS] | ✅ | ✅ | ✅ | ✅ | ✅ | +| Arrcus ArcOS[^ARCOS] | ❌ | ❌ | ❌ | ✅ | ❌ | | Cisco IOL L2[^IOLL2] | ✅ | ✅ | ✅ | ✅ | ✅ | | Cisco IOSv L2[^IOLL2] | ✅ | ✅ | ✅ | ✅ | ✅ | | Aruba AOS-CX[^AOSCX] | ❗ | ✅ | ❌ | ✅ | ✅ | @@ -22,6 +23,7 @@ The following table describes per-platform support of individual STP features: [^EOS]: MSTP is enabled by default +[^ARCOS]: Rapid PVST is the only flavor and is enabled by default; the only off switch is global -- there is no per-port and no per-VLAN disable [^AOSCX]: MSTP is enabled by default; STP is stated as not supported, but it is configured as MSTP (see tip below). [^CL]: STP is enabled by default [^OS10]: PVRST is enabled by default, STP does not work on virtual networks (which are used for VXLAN) diff --git a/docs/platforms.md b/docs/platforms.md index 17487b312c..d1e56fc3b6 100644 --- a/docs/platforms.md +++ b/docs/platforms.md @@ -15,6 +15,7 @@ | Virtual network device | netlab device type | support level | | -----------------------| ------------------ | ------------- | | Arista vEOS/cEOS [❗](caveats-eos) | eos | full | +| Arrcus ArcOS [❗](caveats-arcos) | arcos | best effort | | Aruba AOS-CX [❗](caveats-aruba) | arubacx | full | | Cisco 8000v [❗](caveats-cisco8000v) | cisco8000v | minimal | | Cisco ASAv [❗](caveats-asav) | asav | minimal | @@ -114,6 +115,7 @@ You cannot use all supported network devices with all virtualization providers. | Virtual network device | Vagrant
[Libvirt](labs/libvirt.md) | [Containerlab](labs/clab.md) | | ------------------ | :-: | :-: | | Arista vEOS | [✅](build-eos) | [✅](build-ceos) | +| Arrcus ArcOS | ❌ | [✅](build-arcos) | | Aruba AOS-CX | [✅](build-arubacx) | ✅[❗](clab-vrnetlab) | | Cisco 8000v | ❌ | ✅ | | Cisco ASAv | [✅](build-asav) | ✅ [❗](clab-vrnetlab) | @@ -194,6 +196,7 @@ Ansible playbooks included with **netlab** can deploy and collect device configu | Operating system | Deploy
configuration | Collect
configuration | Configurable
save to startup | | --------------------- |:--:|:--:|:--:| | Arista EOS | ✅ | ✅ | +| Arrcus ArcOS [❗](caveats-arcos) | ✅ | ✅ | | Aruba AOS-CX | ✅ | ✅ | | Cisco ASAv | ✅ | ✅ | | Cisco IOS/IOS XE[^18v]| ✅ [❗](cisco-ios-ssh) | ✅ | @@ -288,6 +291,7 @@ The following system-wide features are configured on supported network operating | Operating system | Hostname | IPv4/IPv6
hosts | LLDP | IPv4
Loopback | IPv6
Loopback | | --------------------- | :------: | :--------: | :-----------------------: | :------------------------: | :------------------------: | | Arista EOS | ✅ | ✅ | ✅ | ✅ | ✅ | +| Arrcus ArcOS | ✅ | ❌ | ❌ | ✅ | ✅ | | Aruba AOS-CX | ✅ | ❌ | ✅ | ✅ | ✅ | | Cisco ASAv | ✅ | ✅ | ❌ | ❌ | ❌ | | Cisco IOS/IOS XE[^18v] | ✅ | ✅ | ✅ | ✅ | ✅ | @@ -320,6 +324,7 @@ The following interface parameters are configured on supported network operating | Operating system | Interface
description | Interface
bandwidth | MTU | Additional
loopbacks | --------------------- |:---:|:---:|:---:|:---:| | Arista EOS | ✅ | ✅ | ✅ | ✅ | +| Arrcus ArcOS | ❌ | ❌ | ✅ | ✅ | | Aruba AOS-CX | ✅ | ❌ | ✅ | ✅ | | Cisco ASAv | ✅ | ❌ | ✅ | ❌ | | Cisco IOSv/IOSvL2 | ✅ | ✅ | ✅ | ✅ | @@ -350,6 +355,7 @@ The following interface addresses are supported on various platforms; most daemo | Operating system | IPv4
addresses | IPv6
addresses | Unnumbered
IPv4 interfaces | Configurable
IPv6 RA | | --------------------- | :-: | :-: | :-: | :-: | | Arista EOS | ✅ | ✅ | ✅ | ✅ | +| Arrcus ArcOS | ✅ | ✅ | ❌ | ❌ | | Aruba AOS-CX | ✅ | ✅ | ✅ | ❌ | | Cisco ASAv | ✅ | ✅ | ❌ | ❌ | | Cisco IOSv/IOSvL2 | ✅ | ✅ | ❌ | ✅ | @@ -400,6 +406,7 @@ Routing protocol [configuration modules](module-reference.md) are supported on t | Operating system | [OSPF](module/ospf.md) | [IS-IS](module/isis.md) | [EIGRP](module/eigrp.md) | [BGP](module/bgp.md) | [RIPv2/ng](module/ripv2.md) | --------------------- | :--: | :--: | :--: | :--: | :--: | | Arista EOS | ✅ | ✅ | ❌ | ✅ | ✅ | +| Arrcus ArcOS [❗](caveats-arcos) | ✅ | ✅ | ❌ | ✅ | ❌ | | Aruba AOS-CX | ✅ | ❌ | ❌ | ✅ | ❌ | | BIRD Internet Routing Daemon | ✅ [❗](caveats-bird) | ❌ | ❌ | ✅ [❗](caveats-bird) | ❌ | | Cisco ASAv | ✅ [❗](caveats-asav) | ✅ [❗](caveats-asav) | ❌ | ✅ | ❌ | @@ -427,6 +434,7 @@ These devices support additional control-plane protocols or BGP address families | Operating system | [BFD](module/bfd.md) | [EVPN](module/evpn.md) | [MPLS/VPN](module/mpls.md) | [FHRP](module/gateway.md) | | --------------------- | :-: | :-: | :-: | :-: | | Arista EOS | ✅ | ✅ | ✅ | ✅ | +| Arrcus ArcOS [❗](caveats-arcos) | ✅ | ✅ | ✅ | ✅ | | Aruba AOS-CX | ✅ | ✅ | ✅ | ✅ | | BIRD | ❌ | ✅ | ❌ | ✅ | | Cisco IOS XE[^XE] | ✅ | ✅ | ✅ | ✅ | @@ -459,6 +467,7 @@ The layer-2 control plane [configuration modules](module-reference.md) are suppo | Operating system | [Spanning
Tree Protocol](module/stp.md) | [Link Aggregation
Groups](module/lag.md) | | ------------------------- |:--:|:--:| | Arista EOS | ✅ | ✅ | +| Arrcus ArcOS [❗](caveats-arcos) | ✅[❗](caveats-arcos) | ✅ | | Aruba CX | ✅ | ✅ | | BIRD | ❌ | ✅ | | Cumulus Linux | ✅ | ✅ | @@ -475,6 +484,7 @@ The data plane [configuration modules](module-reference.md) are supported on the | Operating system | [VLAN](module/vlan.md) | [VRF](module/vrf.md) | [VXLAN](module/vxlan.md) | [MPLS](module/mpls.md) | [SR-MPLS](module/sr-mpls.md) | [SRv6](module/srv6.md) | | --------------------- |:--:|:--:|:--:|:--:|:--:|:--:| | Arista EOS | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | +| Arrcus ArcOS [❗](caveats-arcos) | ✅ | ✅ | ✅[❗](caveats-arcos) | ✅[❗](caveats-arcos) | ✅[❗](caveats-arcos) | ✅[❗](caveats-arcos) | | Aruba AOS-CX | ✅ | ✅ | ✅[❗](caveats-aruba) | [❗](caveats-aruba) | ❌ | ❌ | | BIRD | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | | Cisco 8000v (IOS XR) | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | @@ -525,6 +535,7 @@ Core *netlab* functionality and all multi-protocol routing protocol configuratio | Operating system | OSPFv3 | IS-IS MT | EIGRP
IPv6 AF | BGP
IPv6 AF | SR-MPLS | | --------------------- |:--:|:--:|:--:|:--:|:--:| | Arista EOS | ✅ | ✅ | ❌ | ✅ | ✅ | +| Arrcus ArcOS | ✅ | ❌[❗](caveats-arcos) | ❌ | ✅ | ❌ | | Aruba AOS-CX | ✅ | ❌ | ❌ | ✅ | ❌ | | BIRD | ✅ | ❌ | ❌ | ✅ | ❌ | | Cisco ASAv | ❌ | ✅ | ❌ | ✅ | ❌ | diff --git a/docs/release/26.08.md b/docs/release/26.08.md index bce22c74e6..d82e6fbccb 100644 --- a/docs/release/26.08.md +++ b/docs/release/26.08.md @@ -9,6 +9,8 @@ (release-26.08)= ## New Functionality +* [Arrcus ArcOS](https://arrcus.com/) is now supported (best-effort) as a native containerlab node ([details](build-arcos), [caveats](caveats-arcos)). + **Minor improvements** * Something new diff --git a/netsim/ansible/tasks/deploy-config/arcos.yml b/netsim/ansible/tasks/deploy-config/arcos.yml deleted file mode 100644 index b49785632b..0000000000 --- a/netsim/ansible/tasks/deploy-config/arcos.yml +++ /dev/null @@ -1,12 +0,0 @@ ---- -- name: Push Candidate Config - template: - src: '{{ config_template }}' - dest: '/tmp/.{{ inventory_hostname }}.txt' - -- name: Apply candidate config - arrcus.arcos.arcos_config: - src: "/tmp/.{{ inventory_hostname }}.txt" - load_operation: merge - comment: "initial config pushed by netsim" - delete_src_file: true diff --git a/netsim/ansible/tasks/fetch-config/arcos.yml b/netsim/ansible/tasks/fetch-config/arcos.yml index 4f6aa7516a..c2dda0e001 100644 --- a/netsim/ansible/tasks/fetch-config/arcos.yml +++ b/netsim/ansible/tasks/fetch-config/arcos.yml @@ -1,7 +1,16 @@ --- -- name: save config - shell: echo 'show run' | confd_cli +# Fetch the ArcOS running-config over ansible_connection: docker. The tested container build has +# SSH/NETCONF/gNMI disabled, so confd_cli over docker exec is the only reachable CLI (config +# DEPLOY uses netlab's native sh-mode wrapper -- see netsim/devices/arcos.yml; this collect path +# stays on the docker connection, the same pattern netsim/devices/frr.yml + cRPD keep). +- name: "confd_cli: show running-config" + # confd_cli has no "-c " one-shot flag (-c is --cwd, not "run this command"), so the + # command is fed non-interactively over stdin instead. + shell: printf 'show running-config\n' | confd_cli -C -u admin + args: + executable: /bin/bash register: arcos_run_config + changed_when: false - set_fact: ansible_net_config: "{{ arcos_run_config.stdout }}" diff --git a/netsim/ansible/templates/bfd/arcos.j2 b/netsim/ansible/templates/bfd/arcos.j2 new file mode 100644 index 0000000000..2033420887 --- /dev/null +++ b/netsim/ansible/templates/bfd/arcos.j2 @@ -0,0 +1,13 @@ +{# + ArcOS BFD template. Global `bfd profile ` with enable + timers; protocols reference the + profile (OSPF area-interface `bfd enabled true` is rendered by the ospf template). The GLOBAL + profile is the implicit default and "must have been explicitly configured" -- so we always + create it. +#} +bfd profile GLOBAL + enable + required-minimum-receive {{ bfd.min_rx|default(500) }} + desired-minimum-tx-interval {{ bfd.min_tx|default(500) }} + detection-multiplier {{ bfd.multiplier|default(3) }} + exit +! diff --git a/netsim/ansible/templates/bgp/arcos.j2 b/netsim/ansible/templates/bgp/arcos.j2 new file mode 100644 index 0000000000..fe35612260 --- /dev/null +++ b/netsim/ansible/templates/bgp/arcos.j2 @@ -0,0 +1,91 @@ +{# + ArcOS BGP. Dual-stack peers get TWO separate BGP sessions (one keyed on the IPv4 transport + address, one on IPv6) rather than one session carrying both AFs. + + A session can take ~70 seconds longer to reach ESTABLISHED than committing this implies: the + ArcOS BGP daemon rejects every FSM transition until it finishes reading its startup config. That + is a device startup barrier, not a protocol timer, and nothing is rendered for it -- the wait is + declared as arcos.ebgp_session / arcos.ibgp_session in tests/integration/wait_times.yml. + + The address-family, network and neighbor rendering lives in templates/bgp/arcos.macro.j2 so the + per-VRF BGP instances in templates/vrf/arcos.j2 render exactly the same way (global/VRF parity). + The AS and the router-id stay with the caller: a VRF BGP instance has no `global as` at all, and + it takes its router-id from a different place in the data model. + + Because a dual-stack peer is two single-AF sessions, `n.activate` (bgp.activate, gated by + features.bgp.activate_af) is applied per SESSION rather than per neighbor. That gating lives in + the neighbor macro, so the per-VRF instances honour it too -- see templates/bgp/arcos.macro.j2. + + The bgp.session plugin (per-neighbor password/gtsm/timers/passive/bfd) is a separate template + and is not yet supported. +#} +{% import "templates/bgp/arcos.macro.j2" as bgpcfg %} +{{ bgpcfg.nhs_policy(bgp.neighbors|default([]),bgp) -}} +network-instance default protocol BGP b1 + global as {{ bgp.as }} + global router-id {{ bgp.router_id }} +{{ bgpcfg.cluster_id(bgp) -}} +{{ bgpcfg.afi_safi(bgp) -}} +{{ bgpcfg.advertise(bgp.advertise|default([]),bgp) -}} +{#- bgp.import.connected in the global routing table: every connected subnet outside a VRF, + rendered as BGP network statements (see the connected() macro for why ArcOS cannot use a + DIRECTLY_CONNECTED table-connection). The node loopback is passed in with the interfaces + because netlab keeps it out of node.interfaces; advertise() may already have emitted it + (bgp.advertise_loopback), and the macro de-duplicates against bgp.advertise. -#} +{% if bgp.import.connected is defined %} +{{ bgpcfg.connected( + ([loopback] if loopback is defined else []) + + (interfaces|default([])|rejectattr('vrf','defined')|list), + bgp,bgp.import.connected,bgp.advertise|default([])) -}} +{% endif %} +{{ bgpcfg.neighbors(bgp.neighbors|default([]),bgp) -}} + +{# Redistribution into BGP via OpenConfig table-connections (bgp.import): + STATIC->BGP with src-dst-instance + import-policy, rendered for + EACH AF the BGP instance carries so IPv4 and IPv6 statics are both redistributed -- STATIC + BGP IPV6 commits identically to IPV4 (verified). Static protocol instance is "default" + (routing template), BGP instance is "b1". The per-AF policy name matches + templates/routing/arcos.j2's per-AF policy-definitions (-ipv4 / -ipv6). + The other declared sources are rendered differently: connected as BGP network statements + (ArcOS has no DIRECTLY_CONNECTED source instance) and ospf/isis as their own table-connections + -- both live in templates/bgp/arcos.macro.j2 and are shared with the per-VRF instances. + + Two ArcOS rules, identical to STATIC->ISIS in templates/isis/arcos.j2 (both verified live): + (1) the BGP instance "b1" must be COMMITTED before a table-connection can reference it -- + ArcOS rejects the table-connection if b1 is created in the SAME transaction -- so a #COMMIT + marker (see provider/clab/arcos/netlab-config.j2) splits the transaction; (2) the BGP dst + rejects `default-import-policy` and a bare table-connection redistributes nothing (implicit + reject), so an explicit accept import-policy is required. The default accept policy is emitted + per AF; a custom bgp.import.static.policy references templates/routing/arcos.j2's per-AF + policy-definitions (-ipv4 / -ipv6). #} +{#- The IGP sources (bgp.import.ospf / bgp.import.isis) are rendered by the shared + redistribute_igp macro; the global OSPF/OSPFv3 instance tag is "p1" and the IS-IS instance tag + is "i1" (templates/ospf/arcos.j2, templates/isis/arcos.j2). Both this block and the static + block below live after the same #COMMIT marker because both reference the "b1" instance + created earlier in this file. -#} +{% set _redist_pol = [] %} +{% if bgp.import.static is defined or bgp.import.ospf is defined or bgp.import.isis is defined %} +#COMMIT +{% endif %} +{% if bgp.import.ospf is defined or bgp.import.isis is defined %} +{{ bgpcfg.redistribute_igp('default',bgp.import,bgp,'b1', + {'ospf': 'p1' if 'ospf' in module else none, 'isis': 'i1' if 'isis' in module else none}, + _redist_pol) -}} +{% endif %} +{% if bgp.import.static is defined %} +{% set imp = bgp.import.static %} +{% set _base = imp.policy if (imp is mapping and imp.policy is defined) else "bgp-redist-accept" %} +{% for _af in ['ipv4','ipv6'] if bgp[_af] is defined %} +{% set _pol = _base ~ "-" ~ _af %} +{% if _base == "bgp-redist-accept" and _pol not in _redist_pol %} +{% set _ = _redist_pol.append(_pol) %} +routing-policy policy-definition {{ _pol }} + statement s10 + actions accept-route + exit +! +{% endif %} +network-instance default table-connection STATIC BGP {{ _af|upper }} src-dst-instance default b1 import-policy [ {{ _pol }} ] +! +{% endfor %} +{% endif %} diff --git a/netsim/ansible/templates/bgp/arcos.macro.j2 b/netsim/ansible/templates/bgp/arcos.macro.j2 new file mode 100644 index 0000000000..430d9b9766 --- /dev/null +++ b/netsim/ansible/templates/bgp/arcos.macro.j2 @@ -0,0 +1,216 @@ +{#- ArcOS BGP rendering macros, shared by the global-table BGP configuration + (templates/bgp/arcos.j2) and the per-VRF BGP instances rendered from + templates/vrf/arcos.j2 -- a VRF BGP instance takes the same router-id, + address-family, network and neighbor knobs as the default-instance one. + + Every macro emits the lines that belong UNDER a + "network-instance protocol BGP " header; the caller emits that + header because both the network-instance and the protocol instance tag differ + between the default routing table and a VRF. + + netlab AF names map onto the OpenConfig afi-safi names as _UNICAST. Every + macro takes the dictionary carrying the BGP instance's AF flags (node 'bgp' in + the default table, 'vrfs..af' in a VRF) instead of assuming IPv4, so + IPv4-only, dual-stack and IPv6-only topologies render only the families they + actually use. -#} +{% macro afi_safi(af_flags) -%} +{% for af in ['ipv4','ipv6'] if af_flags[af] is defined %} + global afi-safi {{ af|upper }}_UNICAST +{% endfor %} +{%- endmacro %} +{% macro advertise(pfx_list,af_flags) -%} +{% for pfx in pfx_list %} +{% for af in ['ipv4','ipv6'] if af_flags[af] is defined and pfx[af] is defined %} + global afi-safi {{ af|upper }}_UNICAST network {{ pfx[af] }} +{% endfor %} +{% endfor %} +{%- endmacro %} +{#- Redistribution of CONNECTED subnets into BGP (bgp.import.connected), used by the default + routing table and by the per-VRF BGP instances alike. + + ArcOS has no usable DIRECTLY_CONNECTED redistribution: the OpenConfig source identity is in + the model, but "table-connection DIRECTLY_CONNECTED BGP ..." is rejected with "Source protocol + must be configured" for every src-instance value, and the "protocol DIRECTLY_CONNECTED " + container the schema does accept never registers a runtime instance ("SRC instance does not + exist"). What this device does have is an explicit BGP "network" statement per connected + prefix: the same set of prefixes, originated exactly like bgp.advertise, with + "network export-policy " carrying an optional bgp.import.connected.policy + (all verified on the device, IPv4 and IPv6). + + Prefixes the advertise() macro already emitted (the node/VRF loopbacks netlab takes from + bgp.advertise) are skipped: "network" is a keyed list, so the same prefix must not be emitted + twice with different options. Interface addresses that are not prefix strings (unnumbered + IPv4, IPv6 LLA-only) have no subnet to advertise and are skipped as well. -#} +{% macro connected(intf_list,af_flags,imp,pfx_list=[]) -%} +{% set policy = imp.policy if imp is mapping and imp.policy is defined else '' %} +{% set done = [] %} +{% for pfx in pfx_list %} +{% for af in ['ipv4','ipv6'] if pfx[af] is defined %} +{% set _ = done.append(pfx[af]) %} +{% endfor %} +{% endfor %} +{% for l in intf_list %} +{% for af in ['ipv4','ipv6'] if af_flags[af] is defined and l[af] is string %} +{% set subnet = l[af]|ansible.utils.ipaddr('subnet') %} +{% if subnet not in done %} +{% set _ = done.append(subnet) %} + global afi-safi {{ af|upper }}_UNICAST network {{ subnet }}{{ " export-policy " ~ policy ~ "-" ~ af if policy else "" }} +{% endif %} +{% endfor %} +{% endfor %} +{%- endmacro %} +{#- IGP -> BGP redistribution (bgp.import.ospf / bgp.import.isis) rendered as OpenConfig + table-connections, for the default routing table and for a VRF BGP instance alike. + + netlab AUTO-GENERATES these imports: netsim/modules/_routing.py::process_imports() adds + bgp.import. = { auto: True } for every protocol in defaults.const.vrf_igp_protocols + that a VRF runs, and that branch never consults the device feature list -- so a VRF running + OSPF or IS-IS next to BGP asks for the redistribution whether the user wrote it or not. It has + to be rendered, or those routes silently never reach the CE. + + ArcOS specifics, all verified on the device: + * the source is keyed by PROTOCOL INSTANCE TAG, and OSPFv3 is a separate source identity + (OSPF3) from OSPFv2 (OSPF) -- hence the per-AF source protocol name; + * the same two rules as STATIC->BGP in templates/bgp/arcos.j2 and STATIC->ISIS in + templates/isis/arcos.j2 apply: the destination BGP instance must be COMMITTED before a + table-connection may reference it (the caller emits the #COMMIT split), and a bare + table-connection redistributes nothing (implicit reject), so an explicit accept + import-policy is required; + * a custom bgp.import..policy references templates/routing/arcos.j2's per-AF + policy-definitions (-ipv4 / -ipv6), exactly like the static import. + + `defined` accumulates the accept policies already emitted into this file, so the default + "bgp-redist-accept-" definition is written once even when several VRFs redistribute. -#} +{% macro redistribute_igp(ni,imports,af_flags,dst_inst,src_tags,defined=[]) -%} +{% for proto in ['ospf','isis'] if imports[proto] is defined and src_tags[proto] is defined %} +{% set imp = imports[proto] %} +{% set base = imp.policy if imp is mapping and imp.policy is defined else 'bgp-redist-accept' %} +{% for af in ['ipv4','ipv6'] if af_flags[af] is defined %} +{% set pol = base ~ '-' ~ af %} +{% set src = ('OSPF3' if af == 'ipv6' else 'OSPF') if proto == 'ospf' else 'ISIS' %} +{% if base == 'bgp-redist-accept' and pol not in defined %} +{% set _ = defined.append(pol) %} +routing-policy policy-definition {{ pol }} + statement s10 + actions accept-route + exit +! +{% endif %} +network-instance {{ ni }} table-connection {{ src }} BGP {{ af|upper }} src-dst-instance {{ src_tags[proto] }} {{ dst_inst }} import-policy [ {{ pol }} ] +! +{% endfor %} +{% endfor %} +{%- endmacro %} +{#- A dual-stack peer gets TWO sessions (one keyed on the IPv4 transport address, + one on IPv6), each carrying only its own AF -- ArcOS does not run both AFs + over a single session. `n.activate` (bgp.activate, gated by features.bgp.activate_af) + therefore applies per SESSION, exactly as eos/frr key their "neighbor activate" + on the same-AF address: a session whose only address family is not activated is not + configured at all, because ArcOS cannot bring up a neighbor with no address family. + Inside a VRF the flag is not user-controllable -- netsim/modules/bgp.py sets + activate[af] on every VRF neighbor that carries that AF -- so the gate is a no-op there. -#} +{% macro neighbors(nb_list,af_flags) -%} +{% for n in nb_list %} +{% for af in ['ipv4','ipv6'] if af_flags[af] is defined and n[af] is defined and n.activate[af]|default(false) %} + neighbor {{ n[af] }} peer-as {{ n.as }} +{# bgp.local_as: netlab's replace_global_as (default true) means the local AS REPLACES the + global one in the AS path rather than being prepended to it -- "no-prepend replace-as" on + FRR, and the identically named leaves here. Rendering only "local-as" left the global AS + in the path, so a peer whose own AS equals it dropped the update as an AS-path loop: in + evpn/12-vxlan-ibgp-ebgp the spine advertised its loopback to an FRR leaf that never + installed it ("0 accepted" with nothing inbound-filtered, because a loop is discarded + before policy), the leaf had no route to the spine loopback, and the EVPN session over + that loopback stayed Active forever. With both leaves set the leaf receives the prefix + with AS path "65100" and the session comes up. -#} +{% if n.local_as is defined %} + neighbor {{ n[af] }} ebgp-local-as local-as {{ n.local_as }} +{% if n.replace_global_as|default(True) %} + neighbor {{ n[af] }} ebgp-local-as no-prepend true + neighbor {{ n[af] }} ebgp-local-as replace-as true +{% endif %} +{% endif %} +{#- iBGP-over-loopback: without an explicit transport local-address a session + addressed at the peer loopback stays IDLE forever -- ArcOS does not + auto-source from the interface with the best route to the peer. #} +{% if n.type|default("") == "ibgp" and n._source_intf[af] is defined %} + neighbor {{ n[af] }} transport local-address {{ n._source_intf[af] | ansible.utils.ipaddr('address') }} +{% endif %} + neighbor {{ n[af] }} afi-safi {{ af|upper }}_UNICAST +{# Route reflection (bgp.rr): route-reflector-client is a per-neighbor knob that applies + to every AFI/SAFI configured on the session, so it is emitted once per session rather + than per address family. The matching cluster-id is global -- see the cluster_id macro. #} +{% if n.rr_client|default(False) %} + neighbor {{ n[af] }} route-reflector route-reflector-client true +{% endif %} +{# next-hop-self (bgp.next_hop_self). The per-AF "next-hop SELF" knob under the neighbor + is unconditional -- it rewrites the next hop of REFLECTED IBGP routes too (verified on + the device), so it is netlab's 'all' flavour, the equivalent of "next-hop-self force". + The default 'ebgp' flavour must not touch reflected routes and is rendered as an export + policy instead -- see the nhs_policy and export_policy macros below. + No integration test reaches this branch: every topology that sets next_hop_self to 'all' + (bgp/08-ibgp-localas, bgp/09-vrf-localas, bgp/40-confederation) is refused on arcos for an + unrelated reason. The force semantics were established by measuring the leaf directly. #} +{% if n.next_hop_self|default('') == 'all' %} + neighbor {{ n[af] }} afi-safi {{ af|upper }}_UNICAST next-hop SELF +{% endif %} +{{ export_policy(n,af) -}} +{% endfor %} +{% endfor %} +{%- endmacro %} +{#- RFC 4456 cluster-id: a global (per BGP instance) knob, emitted only on a route reflector + that has an explicit rr_cluster_id. Without it ArcOS prepends the router-id to the + CLUSTER_LIST, which is netlab's own default. Only templates/bgp/arcos.j2 calls this -- netlab + has no way to make a VRF BGP instance a route reflector, so templates/vrf/arcos.j2 does not. -#} +{% macro cluster_id(bgp_data) -%} +{% if bgp_data.rr|default(False) and bgp_data.rr_cluster_id|default(False) %} + global cluster-id {{ bgp_data.rr_cluster_id }} +{% endif %} +{%- endmacro %} + +{#- Export-policy chain that carries next-hop-self. An ArcOS policy chain stops at the first + accept-route/reject-route action and, if no policy in the chain executes one, the route is + REJECTED -- the chain's default disposition is reject-route and a BGP neighbor has no + default-export-policy knob to change that. The next-hop-self policy carries no disposition of + its own (it must not stop the chain), so the accept-everything policy that nhs_policy renders + alongside it closes the chain. + + This never collides with the export policy extra/bgp/policy/arcos.j2 writes into the same + leaf-list: bgp.policy attaches policies only to interface EBGP neighbors + (extra/bgp/policy/__init__.py -> intf_neighbors(select=['ebgp'])), while next_hop_self == + 'ebgp' is set only on IBGP loopback sessions (netsim/modules/bgp.py), so no session ever + carries both. -#} +{% macro export_policy(n,af) -%} +{% if n.next_hop_self|default('') == 'ebgp' %} + neighbor {{ n[af] }} afi-safi {{ af|upper }}_UNICAST apply-policy export-policy [ bgp-next-hop-self-{{ af }} bgp-export-accept-{{ af }} ] +{% endif %} +{%- endmacro %} +{#- next-hop-self for netlab's default 'ebgp' flavour: rewrite the next hop of EBGP-learned routes + only, leaving routes reflected to an RR client with the originator's next hop (RFC 4456). The + statement has NO accept-route action -- an ArcOS policy chain stops at the first accept-route, + so accepting here would leave the trailing accept-everything policy unreachable and make the + pair fragile to reorder. The trailing policy is what supplies the disposition; see the + export_policy macro. + + Rendered once per address family, in the global routing-policy tree (routing-policy is not + per-network-instance), so the global BGP instance owns it; netsim/modules/bgp.py never sets the + 'ebgp' flavour on a VRF neighbor. -#} +{% macro nhs_policy(nb_list,af_flags) -%} +{#- The selection mirrors the neighbors macro exactly, activate gate included, so an address + family that is present on a neighbor but not activated does not leave orphan + policy-definitions behind (visible in bgp/20-dual-stack-activate). -#} +{% for af in ['ipv4','ipv6'] if af_flags[af] is defined + and nb_list|selectattr('next_hop_self','defined')|selectattr('next_hop_self','eq','ebgp') + |selectattr(af,'defined')|map(attribute='activate.' ~ af,default=False)|select|list %} +routing-policy policy-definition bgp-next-hop-self-{{ af }} + statement s10 + conditions bgp-conditions route-type EXTERNAL + actions bgp-actions set-next-hop SELF + exit +! +routing-policy policy-definition bgp-export-accept-{{ af }} + statement s10 + actions accept-route + exit +! +{% endfor %} +{%- endmacro %} diff --git a/netsim/ansible/templates/dhcp/arcos.j2 b/netsim/ansible/templates/dhcp/arcos.j2 new file mode 100644 index 0000000000..f8b895602a --- /dev/null +++ b/netsim/ansible/templates/dhcp/arcos.j2 @@ -0,0 +1,23 @@ +{# + ArcOS DHCP relay-agent. Driven by the standard netlab dhcp interface model: + intf.dhcp.relay.ipv4 = list of server IPs, intf.dhcp.vrf = server-reach VRF. + Per-interface form: relay-agent dhcp interface {enable|helper-address|server-vrf}. +#} +{% for l in interfaces|default([]) if l.dhcp is defined and l.dhcp.relay is defined %} +{% if l.dhcp.relay.ipv4|default([]) %} +relay-agent dhcp interface {{ l.ifname }} enable true +relay-agent dhcp interface {{ l.ifname }} helper-address [ {% for s in l.dhcp.relay.ipv4 %}{{ s }} {% endfor %}] +{% if l.dhcp.vrf is defined and l.dhcp.vrf %} +relay-agent dhcp interface {{ l.ifname }} server-vrf {{ 'default' if l.dhcp.vrf == 'global' else l.dhcp.vrf }} +{% endif %} +{% endif %} +{#- DHCPv6 relay: SAME per-interface shape, "relay-agent dhcpv6" instead of "relay-agent dhcp" + -- enable + helper-address + optional server-vrf, one-for-one with the v4 block above. -#} +{% if l.dhcp.relay.ipv6|default([]) %} +relay-agent dhcpv6 interface {{ l.ifname }} enable true +relay-agent dhcpv6 interface {{ l.ifname }} helper-address [ {% for s in l.dhcp.relay.ipv6 %}{{ s }} {% endfor %}] +{% if l.dhcp.vrf is defined and l.dhcp.vrf %} +relay-agent dhcpv6 interface {{ l.ifname }} server-vrf {{ 'default' if l.dhcp.vrf == 'global' else l.dhcp.vrf }} +{% endif %} +{% endif %} +{% endfor %} diff --git a/netsim/ansible/templates/evpn/arcos.j2 b/netsim/ansible/templates/evpn/arcos.j2 new file mode 100644 index 0000000000..364061d92d --- /dev/null +++ b/netsim/ansible/templates/evpn/arcos.j2 @@ -0,0 +1,102 @@ +{# + ArcOS EVPN. + * default-NI BGP gets L2VPN_EVPN globally + per EVPN neighbor (one-liners OK). + * iBGP-over-loopback EVPN sessions REQUIRE `transport local-address ` or they stay + IDLE (same landmine documented in templates/bgp/arcos.j2). + * each EVPN VLAN's auto-created L2VLAN NI gets advertise-mac-routes + a MAC-VRF BGP instance + carrying RD/RT -- this block MUST be mode-descent (the flat one-liner form aborts: + "network-instance vlanN protocol BGP vlanN: element does not exist"). BGP instance name + matches the main bgp template's ("b1"). +#} +network-instance default protocol BGP b1 global afi-safi L2VPN_EVPN +{# peer-as has to be emitted here, not only in templates/bgp/arcos.j2. + + That template renders a neighbor only for an address family the neighbor actually activates + (bgp.activate / features.bgp.activate_af), because ArcOS cannot bring up a session with no + address family. An EVPN-only session has activate.ipv4 == activate.ipv6 == false -- its address + family is L2VPN_EVPN -- so the whole neighbor block is skipped there and this template was + creating the neighbor with no AS at all. The device took it and reported + "state peer-as 0 / session-state INCOMPLETE", and the peer sat in Active forever + (evpn/12-vxlan-ibgp-ebgp). Emitting it here is idempotent when the other template rendered it + too: same leaf, same value. #} +{% for n in bgp.neighbors|default([]) if n.evpn|default(false) and n.ipv4 is defined %} +network-instance default protocol BGP b1 neighbor {{ n.ipv4 }} peer-as {{ n.as }} +network-instance default protocol BGP b1 neighbor {{ n.ipv4 }} afi-safi L2VPN_EVPN +{% if n._source_intf is defined and n._source_intf.ipv4 is defined %} +network-instance default protocol BGP b1 neighbor {{ n.ipv4 }} transport local-address {{ n._source_intf.ipv4 | ansible.utils.ipaddr('address') }} +{% endif %} +{% endfor %} +{% for vname, vdata in (vlans|default({})).items() if vdata.evpn is defined %} +network-instance vlan{{ vdata.id }} + advertise-mac-routes true + protocol BGP vlan{{ vdata.id }} + global route-distinguisher {{ vdata.evpn.rd }} +{% for rt in vdata.evpn.import|default([]) if rt in vdata.evpn.export|default([]) %} + route-target {{ rt }} both +{% endfor %} +{% for rt in vdata.evpn.import|default([]) if rt not in vdata.evpn.export|default([]) %} + route-target {{ rt }} import +{% endfor %} +{% for rt in vdata.evpn.export|default([]) if rt not in vdata.evpn.import|default([]) %} + route-target {{ rt }} export +{% endfor %} + ! +! +{% endfor %} +{# + EVPN symmetric-IRB / L3VNI. netlab models this as `vrfs..evpn.transit_vni` (evpn.yml + module `vrf: {transit_vni: ...}`) -- an L3VNI shared by every VLAN/subnet bound to that VRF, + giving symmetric routing (both ingress AND egress PE route via the L3VNI) instead of the + per-VLAN distributed-anycast-gateway pattern above (which only proves intra-subnet bridging + + local routing, not real inter-subnet transport). + + PARTLY FUNCTIONAL on this image, and NOT for the reason recorded here before. Type-5 routes ARE + originated: with the VRF BGP instance carrying its connected subnets (bgp.import.connected -> + network statements, see templates/bgp/arcos.macro.j2) the default instance's L2VPN_EVPN table + holds one well-formed route-type 5 per subnet -- RD-qualified, with encap-type:VXLAN and a + per-VTEP router-mac -- and "show overlay vni-all " reports vni-type ROUTED. The + earlier claim that ZERO type-5 routes are ever originated was measured before those network + statements existed, i.e. with nothing for the VRF to originate. + + What does NOT work is advertising them: the local type-5 routes carry + path-types [ BEST_PATH EXPORTED_PATH ] and no rib-out entry, so the peer never receives them + (its EVPN table holds only its own). Setting rt-afi-safi L2VPN_EVPN evpn-route-options + suppress-rt5-routes false commits and changes nothing. The import direction is fine once the + route target below is explicit. Rendered here for completeness but symmetric IRB is still NOT + declared in features.evpn -- only the L2VNI/distributed-anycast-gateway path is. +#} +{#- The route distinguisher is netlab's vrfs..rd, the same value templates/vrf/arcos.j2 writes + on this very instance (evpn is deployed after vrf, so it would otherwise silently overwrite it + with a different RD). Both templates render the identical leaf, so whichever runs last the + committed RD still matches the data model -- the same thing templates/vrf/eos.bgp.j2 does. -#} +{#- The L3VNI route target must be written explicitly, per address family. + + Left to itself ArcOS derives the IP-VRF route target as : (65000:5042 for a transit + VNI of 5042), while netlab -- and every other netlab EVPN device, FRR included -- derives it as + : from vrfs..import/export (65000:1). The two never match, so neither side + imports the other's type-5 and symmetric IRB does nothing. Measured on the container image: with + the auto-derived value the DUT showed "evpn-v4-import-route-targets 65000:5042" against FRR's + "Import RT 65000:1" and its customer VRF held only its own prefixes; writing the netlab value + made the peer's subnet appear in that VRF immediately. + + The RT does NOT live where the MAC-VRF one does. For an L2VLAN instance it is + "protocol BGP route-target both" (above); an L3VRF has no such node -- it sits under + the unicast AFI in an rt-afi-safi L2VPN_EVPN container, and the direction keyword is mandatory. + It is per address family: setting it only under IPV4_UNICAST leaves the IPv6 type-5 routes + carrying the auto-derived value. -#} +{% for vname, vdata in (vrfs|default({})).items() if vdata.evpn is defined and vdata.evpn.transit_vni is defined %} +network-instance {{ vname }} protocol BGP {{ vname }} +#COMMIT +network-instance {{ vname }} protocol BGP {{ vname }} global route-distinguisher {{ vdata.rd }} +network-instance {{ vname }} protocol BGP {{ vname }} global afi-safi L3VPN_IPV4_UNICAST +{% for af in ['ipv4','ipv6'] if vdata.af[af]|default(false) %} +{% for rt in vdata.export|default([]) %} +network-instance {{ vname }} protocol BGP {{ vname }} global afi-safi {{ af|upper }}_UNICAST rt-afi-safi L2VPN_EVPN route-target {{ rt }} export +{% endfor %} +{% for rt in vdata.import|default([]) %} +network-instance {{ vname }} protocol BGP {{ vname }} global afi-safi {{ af|upper }}_UNICAST rt-afi-safi L2VPN_EVPN route-target {{ rt }} import +{% endfor %} +{% endfor %} +network-instance {{ vname }} vni {{ vdata.evpn.transit_vni }} local-tunnel-endpoint-id 0 +#COMMIT +{% endfor %} diff --git a/netsim/ansible/templates/gateway/arcos.j2 b/netsim/ansible/templates/gateway/arcos.j2 new file mode 100644 index 0000000000..e471aed226 --- /dev/null +++ b/netsim/ansible/templates/gateway/arcos.j2 @@ -0,0 +1,35 @@ +{# + ArcOS first-hop redundancy -- VRRP. The config path hangs off the interface's OWN primary + address in the address family the VRRP group serves: + interface subinterface 0 address vrrp vrrp-group + virtual-address | priority

| advertisement-interval | accept-mode + IPv4 and IPv6 expose the identical leaf set (verified on the device), so both are rendered from + one loop -- the interface may be IPv4-only, IPv6-only or dual-stack, and gateway.ipv4 / + gateway.ipv6 are independent. + + gateway.vrrp.preempt is NOT rendered: this image has no preempt leaf under vrrp-group in either + address family ("preempt", "preempt enabled" and "preempt-mode" are all rejected with "element + does not exist"), and VRRP preemption is on by default. Anycast gateway is not declared -- + untested on this image, and nothing here renders it. + + The IPv4 VRRPv3 checksum is NOT rendered either, because ArcOS has no leaf for it: ArcOS follows + RFC 5798 and always includes the IPv4 pseudo-header, while netlab tells every other VRRP device + to exclude it (gateway/frr/vrrp-cp.j2, gateway/eos.j2, gateway/exos.j2), so IPv4 advertisements + are discarded in both directions and both routers stay Master. IPv6 is unaffected. See the ArcOS + section of docs/caveats.md. + + The VRRP daemon itself only runs because netsim/devices/arcos.yml restores the process manager's + file-descriptor soft limit to the systemd default from netlab_start_exec -- without it the config + below commits and the protocol never starts. +#} +{% for l in interfaces|default([]) if l.gateway is defined and l.gateway.protocol|default('') == 'vrrp' %} +{% for af in ['ipv4','ipv6'] if l[af] is defined and l[af] is string and l.gateway[af] is defined %} +{% set _vrrp = 'interface ' ~ l.ifname ~ ' subinterface 0 ' ~ af ~ ' address ' + ~ (l[af] | ansible.utils.ipaddr('address')) ~ ' vrrp vrrp-group ' + ~ l.gateway.vrrp.group %} +{{ _vrrp }} virtual-address {{ l.gateway[af] | ansible.utils.ipaddr('address') }} +{% if l.gateway.vrrp.priority is defined %} +{{ _vrrp }} priority {{ l.gateway.vrrp.priority }} +{% endif %} +{% endfor %} +{% endfor %} diff --git a/netsim/ansible/templates/initial/arcos.j2 b/netsim/ansible/templates/initial/arcos.j2 index 3bd5892bac..21304c3674 100644 --- a/netsim/ansible/templates/initial/arcos.j2 +++ b/netsim/ansible/templates/initial/arcos.j2 @@ -1,29 +1,88 @@ -system hostname {{ inventory_hostname }} -! -! -interface loopback0 - enable true - subinterface 0 - {% if 'ipv4' in loopback %} - ipv4 address {{ loopback.ipv4 | ansible.netcommon.ipv4('address') }} prefix-length 32 - {% endif %} - {% if 'ipv6' in loopback %} - ipv6 address {{ loopback.ipv6 | ansible.netcommon.ipv6('address') }} prefix-length 64 - {% endif %} -! -! +{# + ArcOS initial configuration. confd_cli block-indent CLI (OpenConfig-modelled). + + ONE loop over `netlab_interfaces` (== the system loopback prepended to `interfaces`, see + netsim/outputs/common.py) configures every L3 interface -- system loopback, additional + loopbacks, LAG aggregates and routed physical ports -- so the IPv4 and IPv6 address config + always stay in sync (they are rendered side by side, from the same `l`, with the interface's + ACTUAL prefix length via ipaddr('prefix'); netlab hands loopbacks a /64 IPv6 and a /32 IPv4 by + default, but a topology may override either). Interfaces that are configured by another module + are skipped here: LAG members (templates/lag/arcos.j2, aggregate-id only -- ArcOS rejects + "Bond member Interface can't have it's own L3 configuration") and SVIs / tunnel / VXLAN virtual + interfaces. VLAN access/trunk ports are switched (no subinterface/IP -- templates/vlan/arcos.j2 + puts them in switched-vlan mode) so they still get an `interface`/`enable`/`mtu` stanza but no + `subinterface 0` addressing. + + MTU: the `interface mtu ` config leaf is NOT a pass-through -- ArcOS always programs + the real kernel netdev (and, critically, the MTU value it advertises in OSPF's DBD packets) + as (configured-leaf - 26), confirmed on a real ArcOS node across three data points (leaf 1500 + -> kernel/advertised 1474; leaf 2000 -> 1974; leaf 1526 -> exactly 1500) and matching the + platform's own default ("show interface swp1" reports the unset leaf's default as 9000 -> the + observed 8974 default veth/advertised MTU is that same -26 rule applied to the default). The + 26 bytes are reserved headroom (constant across the tested range, not a percentage). So: to + make ArcOS advertise/use netlab's intended interface MTU (l.mtu, already resolved by + netsim/augment/links.py for every device), the leaf must be set to l.mtu + 26, not l.mtu -- + setting the leaf to the bare intended value visibly "commits clean" but silently under-shoots + the real kernel/OSPF-advertised MTU by 26. This device-side compensation alone makes an + ArcOS<->FRR OSPF adjacency converge (both sides agree on 1500); no containerlab/core change is + required. Loopbacks carry no MTU (the leaf is skipped for them). +#} +{# ArcOS accepts only letters/digits/hyphen in a hostname, so the underscores netlab + allows in node names have to be replaced (same as eos/frr/sonic/...) -- otherwise the + whole initial commit aborts on any multi-DUT topology (dut_a, dut_spine, ...) #} +system hostname {{ inventory_hostname.replace("_","-") }} ! -{% for l in interfaces|default([]) %} -interface {{ l.ifname }} +{# Every interface this loop configures is remembered in _cfg.intf so the VRF-binding loop at + the end of this file can bind EXACTLY the interfaces this file creates -- see the comment + there. #} +{% set _cfg = namespace(intf=[]) %} +{% for l in netlab_interfaces|default([]) + if (l.type|default('') in ['loopback','lag','vlan_member'] or not l.virtual_interface|default(false)) + and l.lag._parentindex is not defined %} +{% set _cfg.intf = _cfg.intf + [ l.ifname ] %} +{# A routed VLAN subinterface (netlab type vlan_member, named . by + devices/arcos.yml) is not a standalone ArcOS interface -- it is `subinterface ` + under the PARENT port, and ArcOS derives the 802.1Q tag from that subinterface index. Every + other L3 interface carries its addressing on the untagged `subinterface 0`. #} +{% set _ifname = l.parent_ifname|default(l.ifname) %} +{% set _subif = l.vlan.access_id if l.type|default('') == 'vlan_member' else 0 %} +interface {{ _ifname }} enable true - subinterface 0 - {% if 'ipv4' in l %} - ipv4 address {{ l.ipv4 | ansible.netcommon.ipv4('address') }} prefix-length {{ l.ipv4 | ansible.netcommon.ipaddr('prefix') }} - {% endif %} - {% if 'ipv6' in l %} - ipv6 address {{ l.ipv6 | ansible.netcommon.ipv6('address') }} prefix-length {{ l.ipv6 | ansible.netcommon.ipaddr('prefix') }} - {% endif %} +{# MTU is a per-PORT property on ArcOS: there is no per-subinterface mtu leaf, and an `mtu` + written in subinterface context silently lands on the parent interface (verified). A routed + VLAN subinterface must therefore not render one -- it would overwrite the parent port's MTU, + and on a trunk carrying several routed VLANs the last one would win. The parent port gets + its own MTU from its own pass through this loop. #} +{% if l.mtu is defined and l.type|default('') not in ['loopback','vlan_member'] %} + mtu {{ l.mtu + 26 }} +{% endif %} +{% if not (l.vlan is defined and (l.vlan.access is defined or l.vlan.trunk is defined)) %} + subinterface {{ _subif }} +{% if l.ipv4 is defined and l.ipv4 is string %} + ipv4 address {{ l.ipv4 | ansible.utils.ipaddr('address') }} prefix-length {{ l.ipv4 | ansible.utils.ipaddr('prefix') }} +{% endif %} +{% if l.ipv6 is defined and l.ipv6 is string %} + ipv6 address {{ l.ipv6 | ansible.utils.ipaddr('address') }} prefix-length {{ l.ipv6 | ansible.utils.ipaddr('prefix') }} +{% endif %} +{% endif %} ! {% endfor %} +{# An interface has to belong to its VRF by the time its addresses are committed: two VRFs may + legitimately use the SAME address on their loopbacks (tests/integration/vrf/04-multi-vrf- + loopback-ipv6), and while both loopbacks are still in the default network-instance ArcOS + rejects the second one with "IPv6 subnet overlaps with 'interface loopback2 ...'". The L3VRF + instances and the interface bindings therefore have to be part of THIS transaction rather + than of the later vrf module. templates/vrf/arcos.j2 still owns the routing protocols inside + each VRF and re-binds the interfaces created by the modules in between (an SVI cannot be + bound before the VLAN template creates it). -no lldp interface ma1 + ArcOS rejects a binding for an interface that does not exist yet ("Aborted: 'network-instance + customer interface vlan1000': Interface vlan1000"), which aborts the whole initial commit, so + only the interfaces configured by the loop above may be bound here -- hence _cfg.intf rather + than a second copy of that loop's filter. #} +{% for vname in vrfs|default({}) %} +network-instance {{ vname }} type L3VRF +{% endfor %} +{% for l in netlab_interfaces|default([]) if l.vrf is defined and l.ifname in _cfg.intf %} +network-instance {{ l.vrf }} interface {{ l.ifname }} +{% endfor %} diff --git a/netsim/ansible/templates/isis/arcos.j2 b/netsim/ansible/templates/isis/arcos.j2 new file mode 100644 index 0000000000..f9c48b7e21 --- /dev/null +++ b/netsim/ansible/templates/isis/arcos.j2 @@ -0,0 +1,125 @@ +{# + ArcOS IS-IS. Default-instance IS-IS only when a default IS-IS net exists (VRF-only ISIS has + none, see templates/vrf/arcos.j2 for the per-VRF form). + + ADDRESS FAMILIES: the IPV4 and IPV6 unicast AFs are enabled side by side (globally, on the + loopback and on every IS-IS interface), each gated on isis.af. so an IPv4-only box never + emits IPv6 IS-IS config and -- crucially -- an IPv6-only box never emits IPv4 IS-IS config + (isis.af.ipv4 is false there). + + IPv6 is SINGLE-TOPOLOGY, NOT multi-topology (RFC 5120). This ArcOS build has no multi-topology + knob (no such leaf in the OpenConfig IS-IS model; every `... global/level multi-topology` + form is rejected on commit). It advertises IPv6 prefixes in the standard IPv6 Reachability TLV + (TLV 236), verified in a peer's LSDB: `IPv6 Reachability: ` with NO `MT Router Info: + ipv6-unicast` and NO `MT IPv6 Reachability` TLV. netlab configures every OTHER IS-IS device + with multi-topology for IPv6 (ios `multi-topology`, iosxr `no single-topology`, frr `topology + ipv6-unicast`, eos/srlinux/sros `multi-topology`), so a peer running MT will NOT install + ArcOS's IPv6 routes (confirmed: an FRR neighbour formed the adjacency and installed ArcOS's + IPv4 loopback but showed no ArcOS IPv6 route). IPv6 IS-IS therefore interoperates only in an + all-ArcOS IS-IS domain; cross-vendor IPv6 IS-IS is unsupported. Documented in docs/caveats.md; + platforms.md "IS-IS MT" for ArcOS is therefore false (marked as unsupported there). +#} +{% set typ = isis.type|default("level-2") %} +{% set caps = {"level-1":"LEVEL_1","level-2":"LEVEL_2","level-1-2":"LEVEL_1_2"} %} +{% set do_l1 = typ in ["level-1","level-1-2"] %} +{% set do_l2 = typ in ["level-2","level-1-2"] %} +{% if isis.net is defined %} +network-instance default protocol ISIS i1 + global net {{ isis.net }} + global level-capability {{ caps[typ]|default("LEVEL_2") }} +{% if isis.af.ipv4|default(false) %} + global af IPV4 UNICAST enabled true +{% endif %} +{% if isis.af.ipv6|default(false) %} + global af IPV6 UNICAST enabled true +{% endif %} +{% if do_l1 %} + level 1 enabled true +{% endif %} +{% if do_l2 %} + level 2 enabled true +{% endif %} + interface loopback0 enabled true + interface loopback0 passive true +{% if isis.af.ipv4|default(false) %} + interface loopback0 af IPV4 UNICAST enabled true +{% endif %} +{% if isis.af.ipv6|default(false) %} + interface loopback0 af IPV6 UNICAST enabled true +{% endif %} +{% if do_l1 %} + interface loopback0 level 1 enabled true +{% endif %} +{% if do_l2 %} + interface loopback0 level 2 enabled true +{% endif %} +{% for l in interfaces|default([]) if l.vrf is not defined %} +{% if l.isis is defined %} + interface {{ l.ifname }} enabled true +{% if isis.af.ipv4|default(false) %} + interface {{ l.ifname }} af IPV4 UNICAST enabled true +{% endif %} +{% if isis.af.ipv6|default(false) %} + interface {{ l.ifname }} af IPV6 UNICAST enabled true +{% endif %} +{% if l.isis.network_type|default("") == "point-to-point" %} + interface {{ l.ifname }} network-type POINT_TO_POINT +{% endif %} +{% if l.isis.passive|default(false) %} + interface {{ l.ifname }} passive true +{% endif %} +{# per-interface circuit-type: l.isis.type overrides the node level-capability #} +{% set ityp = l.isis.type|default(typ) %} +{% if ityp in ["level-1","level-1-2"] %} + interface {{ l.ifname }} level 1 enabled true +{% endif %} +{% if ityp in ["level-2","level-1-2"] %} + interface {{ l.ifname }} level 2 enabled true +{% endif %} +{#- Per-interface IS-IS metric. ArcOS hangs it off the interface's LEVEL container + ("interface level metric "), not off the interface itself, so it is + emitted once per level the interface participates in. Wide metrics need no knob: this + build originates Extended Reachability / Extended IP Reachability (TLV 22/135) + natively -- a metric of 80000 shows up verbatim in a peer's LSDB. Sourced from + l.isis.metric with l.isis.cost as the fallback, the same pair templates/isis/eos.j2 + and templates/isis/frr.j2 read. -#} +{% if l.isis.cost is defined or l.isis.metric is defined %} +{% set _metric = l.isis.metric|default(l.isis.cost) %} +{% if ityp in ["level-1","level-1-2"] %} + interface {{ l.ifname }} level 1 metric {{ _metric }} +{% endif %} +{% if ityp in ["level-2","level-1-2"] %} + interface {{ l.ifname }} level 2 metric {{ _metric }} +{% endif %} +{% endif %} +{% endif %} +{% endfor %} + +{#- IS-IS import/redistribution (isis.import): OpenConfig table-connection STATIC ISIS , + rendered for EACH address family the IS-IS instance carries (isis.af) so IPv4 and IPv6 + statics are both redistributed -- STATIC ISIS IPV6 commits identically to IPV4 (verified). + A static route redistributed into IS-IS appears in both nodes' IS-IS LSDB reachability + (control plane; FIB install is container-limited, same as SR-MPLS). Two ArcOS + rules: (1) the ISIS instance must be COMMITTED before the table-connection references it -> + #COMMIT split; (2) ISIS dst rejects `default-import-policy` and the BARE table-connection + redistributes nothing (implicit reject) -> an explicit accept import-policy is required. + The per-AF policy name matches templates/routing/arcos.j2's per-AF policy-definitions + (-ipv4 / -ipv6). connected(DIRECTLY_CONNECTED) has no addressable + src-instance (same gap as BGP import) -> feature declares import: [ static ]. -#} +{% if isis.import.static is defined %} +#COMMIT +{% set _base = isis.import.static.policy if (isis.import.static is mapping and isis.import.static.policy is defined) else "isis-redist-accept" %} +{% for _af in ['ipv4','ipv6'] if isis.af[_af]|default(false) %} +{% set _pol = _base ~ "-" ~ _af %} +{% if _base == "isis-redist-accept" %} +routing-policy policy-definition {{ _pol }} + statement s10 + actions accept-route + exit +! +{% endif %} +network-instance default table-connection STATIC ISIS {{ _af|upper }} src-dst-instance default i1 import-policy [ {{ _pol }} ] +! +{% endfor %} +{% endif %} +{% endif %} diff --git a/netsim/ansible/templates/lag/arcos.j2 b/netsim/ansible/templates/lag/arcos.j2 new file mode 100644 index 0000000000..66744dca1d --- /dev/null +++ b/netsim/ansible/templates/lag/arcos.j2 @@ -0,0 +1,18 @@ +{# + ArcOS LAG template. Aggregate = `interface bond` + `aggregation lag-type + LACP|STATIC` (+ min-links); members = `ethernet aggregate-id bond`. The bond's L3 + addressing is rendered by the initial template (bond appears as a regular interface with + type lag). +#} +{% for intf in interfaces if intf.type == 'lag' %} +interface {{ intf.ifname }} + enabled true + aggregation lag-type {{ 'STATIC' if intf.lag.lacp|default('') == 'off' else 'LACP' }} +! +{% for ch in interfaces if ch.lag._parentindex|default(None) == intf.lag.ifindex %} +interface {{ ch.ifname }} + enabled true + ethernet aggregate-id {{ intf.ifname }} +! +{% endfor %} +{% endfor %} diff --git a/netsim/ansible/templates/mpls/arcos.j2 b/netsim/ansible/templates/mpls/arcos.j2 new file mode 100644 index 0000000000..33c90906e6 --- /dev/null +++ b/netsim/ansible/templates/mpls/arcos.j2 @@ -0,0 +1,14 @@ +{# + ArcOS MPLS/LDP. LDP over the IGP underlay. Native container has NO kernel MPLS + (net.mpls.platform_labels absent) so label FORWARDING is not testable here -- this brings up + the LDP control plane (sessions + FEC/label bindings), same datapath caveat as SR. +#} +{% if ldp is defined %} +network-instance default mpls signaling-protocols ldp global enable true +network-instance default mpls signaling-protocols ldp global lsr-id {{ ldp.router_id }} +{% for l in interfaces if l.ldp is defined and not l.ldp.passive|default(false) %} +network-instance default mpls signaling-protocols ldp interface-attributes interface {{ l.ifname }} link-hello true +{# the AF enable is REQUIRED -- without it hellos flow but the session never forms #} +network-instance default mpls signaling-protocols ldp interface-attributes interface {{ l.ifname }} address-family IPV4 enabled true +{% endfor %} +{% endif %} diff --git a/netsim/ansible/templates/ospf/arcos.j2 b/netsim/ansible/templates/ospf/arcos.j2 index 1fe97d1ad6..2f5d6e223d 100644 --- a/netsim/ansible/templates/ospf/arcos.j2 +++ b/netsim/ansible/templates/ospf/arcos.j2 @@ -1,18 +1,97 @@ -{% set pid = ospf.process|default(1) %} +{# + ArcOS OSPFv2 + OSPFv3. Same area/area-interface shape for both AFs modulo two differences: + the protocol keyword (OSPF vs OSPF3), and the area ID is written as a plain integer ("area 0", + from ospf.areas[]._area_int) in OSPFv3, not dotted-decimal ("area 0.0.0.0") like OSPFv2. -network-instance default protocol OSPF p{{ pid }} - global log-adjacency-changes LOG_ADJ_ENABLE_DETAILED - area {{ ospf.area }} - interface Loopback0 - {% for l in interfaces|default([]) %} - interface {{ l.ifname }} - {% if l.type|default("") == "p2p" %} - network-type POINT_TO_POINT_NETWORK - {% endif %} - {% if l.ospf.cost is defined %} - metric {{ l.ospf.cost }} - {% endif %} - {% endfor %} + MULTI-AREA: one `area ` block is emitted per area the node participates in (netlab + pre-computes ospf.areas), and every interface is filed under its own ospf.area (an ABR then + carries interfaces in several areas under the same OSPF instance -- ArcOS accepts multiple + area blocks per instance, verified). The system loopback goes into loopback.ospf.area. An area + block is emitted only when it actually has an interface (or the loopback) in this AF, so a + v4-only area does not produce an empty OSPFv3 area block (or vice-versa). + + Known ArcOS limit (see docs/caveats.md): the ABR floods correct inter-area Type-3 summaries for + transit networks, but summarizes a /32 loopback host route with a /0 mask, so a loopback in a + NON-backbone area is not reachable from other areas. Adjacencies, intra-area routes and + inter-area transit prefixes are unaffected. Nothing to configure around it here -- it is an LSA + origination bug in this build. - - + MTU interop note: an ArcOS<->FRR OSPF adjacency needs both sides to advertise the same MTU in + their DBD exchange. ArcOS reprograms the real kernel netdev MTU (and the value OSPF advertises) + to (configured-leaf - 26) on every commit, so netsim/ansible/templates/initial/arcos.j2 sets + the interface MTU leaf to `l.mtu + 26` to land exactly on netlab's intended value. With that + device-side compensation the adjacency converges without any containerlab/core change. +#} +{% set pid = ospf.process|default(1) %} +{% set _dflt_area = ospf.area|default('0.0.0.0') %} +{% set _areas = ospf.areas|default([{'area': _dflt_area, '_area_int': 0}]) %} +{% macro area_interfaces(af, area_dotted) %} +{% for l in interfaces|default([]) if l.ospf is defined and l[af] is defined and (l.ospf.area|default(_dflt_area)) == area_dotted %} + interface {{ l.ifname }} +{# network-type selection (matches netlab's cross-vendor OSPF semantics): + 1) netlab's canonical ospf.network_type value is "point-to-point" (hyphen) -- see + netsim/modules/ospf.py's network_type() allowed-value list; match that spelling. + 2) fall back to point-to-point on a 2-node p2p link ONLY when ospf.network_type was not set + explicitly, so an explicit `ospf.network_type: broadcast` is honoured (a 2-node link is + always netlab type p2p) rather than forced to point-to-point, which would stall an + FRR<->ArcOS adjacency in ExStart on a network-type mismatch. An explicit "broadcast" + renders nothing (ArcOS's own default network-type). #} +{% set nt = l.ospf.network_type|default('point-to-point' if l.type|default('') == 'p2p' else '') %} +{% if nt == 'point-to-point' %} + network-type POINT_TO_POINT_NETWORK +{% endif %} +{% if l.ospf.cost is defined %} + metric {{ l.ospf.cost }} +{% endif %} +{#- Passive interface. netlab sets ospf.passive both explicitly (link-level ospf.passive) + and implicitly on a stub link, and the flag is a plain bool, so it is rendered whenever + it is true. This macro serves OSPFv2 and OSPFv3 alike, so both address families get it. -#} +{% if l.ospf.passive|default(false) %} + passive true +{% endif %} +{% if l.ospf.bfd|default(ospf.bfd|default(false)) %} + bfd enabled true +{% endif %} +{% if l.ospf.priority is defined %} + priority {{ l.ospf.priority }} +{% endif %} +{% if l.ospf.timers.hello is defined %} + timers hello-interval {{ l.ospf.timers.hello }} +{% endif %} +{% if l.ospf.timers.dead is defined %} + timers dead-interval {{ l.ospf.timers.dead }} +{% endif %} +{% endfor %} +{% endmacro %} +{% if ospf.router_id is defined and ospf.af.ipv4|default(false) %} +network-instance default protocol OSPF p{{ pid }} + global router-id {{ ospf.router_id }} + global log-adjacency-changes LOG_ADJ_ENABLE_DETAILED +{% for a in _areas %} +{% set _has_lb = loopback.ipv4 is defined and (loopback.ospf.area|default(_dflt_area)) == a.area %} +{% set _body %}{{ area_interfaces('ipv4', a.area) }}{% endset %} +{% if _has_lb or _body.strip() %} + area {{ a.area }} +{% if _has_lb %} + interface loopback0 +{% endif %} +{{ _body }} +{% endif %} +{% endfor %} +{% endif %} +{% if ospf.router_id is defined and ospf.af.ipv6|default(false) %} +network-instance default protocol OSPF3 p{{ pid }} + global router-id {{ ospf.router_id }} + global log-adjacency-changes LOG_ADJ_ENABLE_DETAILED +{% for a in _areas %} +{% set _has_lb = loopback.ipv6 is defined and (loopback.ospf.area|default(_dflt_area)) == a.area %} +{% set _body %}{{ area_interfaces('ipv6', a.area) }}{% endset %} +{% if _has_lb or _body.strip() %} + area {{ a._area_int }} +{% if _has_lb %} + interface loopback0 +{% endif %} +{{ _body }} +{% endif %} +{% endfor %} +{% endif %} diff --git a/netsim/ansible/templates/routing/arcos.j2 b/netsim/ansible/templates/routing/arcos.j2 new file mode 100644 index 0000000000..d5611af522 --- /dev/null +++ b/netsim/ansible/templates/routing/arcos.j2 @@ -0,0 +1,224 @@ +{# + ArcOS generic-routing template (statics, prefix-sets, policies). OpenConfig-CLI: + defined-sets prefix-set, policy-definition/statement, protocol STATIC static-route/ + next-hop-index. The routing-policy defined-sets prefix-set container is AF-agnostic (accepts + either family in the same 'prefix' leaf) -- so the same rendering loop covers ipv4 and ipv6. + STATIC route-install is likewise AF-agnostic (one STATIC instance, IPv4 and IPv6 static-routes + rendered from one loop over both AFs). + + COMMUNITY LISTS become OpenConfig bgp-defined-sets: a netlab standard list is a + `community-set` with `community-member`s, a large list a `large-community-set` with + `large-community-member`s. Members may be literals or regular expressions -- both verified by + effect on a peer. An ArcOS set has NO per-member action, only a whole-set match option + (ANY / ALL / INVERT), so a netlab list that mixes permit and deny entries has no faithful + rendering; see _match_opts below and the arcos_community_list quirk in netsim/devices/arcos.py. + + PREFIX SETS carry no per-entry action -- an ArcOS prefix-set is a flat list of members and the + accept/reject decision lives on the policy statement that references it. A netlab prefix list + has an action per entry, where a 'deny' entry means "this prefix does NOT match the list", so + the faithful rendering is to emit the PERMIT entries only; the deny entries are simply absent + and therefore do not match. A list with no permit entries at all matches nothing, so the set is + not emitted and the statement referencing it is skipped entirely (an empty prefix-set is not a + no-op -- it would leave the statement with no condition, i.e. matching everything). The one + ordering that cannot be expressed -- a deny entry shadowing a later permit entry it overlaps -- + is refused by the prefix_list_shape quirk in netsim/devices/arcos.py. + + POLICIES: netlab route policies apply to whichever AF references them, so each policy is + rendered once per AF as `-ipv4` and `-ipv6` (the bgp/isis import table-connections + are per-AF and pick the matching name). ArcOS routing-policy is otherwise AF-agnostic (a + prefix-set holds either family, bgp-actions are AF-independent), so the two variants differ + only in name; a policy that matches an IPv4-only prefix-set simply matches nothing in its + -ipv6 form (and vice-versa), which is correct. +#} +{% if routing.prefix|default({}) %} +{% for pf_af in ['ipv4', 'ipv6'] if pf_af in routing._prefix|default({}) %} +{% for p_name, p_all in routing._prefix[pf_af].items() %} +{#- Permit entries only -- see the header. A deny entry is not a member, so it does not + match; a list with none left over is emitted at all. -#} +{% set p_value = p_all|rejectattr('action','eq','deny')|list %} +{% if p_value %} +routing-policy defined-sets prefix-set {{ p_name }} +{% for p_entry in p_value %} +{% set lo = p_entry.min|default(p_entry[pf_af]|ansible.utils.ipaddr('prefix')) %} +{% set hi = p_entry.max|default(128 if pf_af == 'ipv6' else 32) %} +{% if p_entry.min is defined or p_entry.max is defined %} + prefix {{ p_entry[pf_af] }} {{ lo }}..{{ hi }} +{% else %} + prefix {{ p_entry[pf_af] }} exact +{% endif %} +{% endfor %} + exit +! +{% endif %} +{% endfor %} +{% endfor %} +{% endif %} +{#- A netlab community list carries a per-entry action; an ArcOS community-set does not -- it has + one match option for the whole set. Two shapes are expressible and nothing else: + * every entry permits -> the members ARE the set, match-set-options ANY + * deny entries + a catch-all '.*' permit -> the DENIED members are the set, and the sense is + flipped with match-set-options INVERT ("matches none of") + The quirk in netsim/devices/arcos.py refuses any other mix up front, so these two macros never + have to guess. -#} +{% macro _match_opts(c_data) %}{{ 'INVERT' if c_data.value|selectattr('action','eq','deny')|list else 'ANY' }}{% endmacro %} +{% macro _members(c_data) %} +{%- set _deny = c_data.value|selectattr('action','eq','deny')|list -%} +{{- (_deny if _deny else c_data.value)|map(attribute='_value')|join(' ') -}} +{% endmacro %} +{#- Community lists, and the large-community-sets the set.community.large action has to reference + (ArcOS has no INLINE form for large communities -- only 'method REFERENCE' -- so a literal + netlab value list is materialised into a set named SETLC__). Both are + emitted before the policy-definitions that use them, and a #COMMIT separates them: ArcOS + rejects a reference to a named list created in the same transaction (same trap as + templates/vlan/arcos.j2). Emitted only when there is something to emit, so a topology with no + communities renders exactly as before. -#} +{% set _lsets = [] %} +{% for p_name, p_entries in (routing.policy|default({})).items() %} +{% for p_entry in p_entries if p_entry.set.community.large is defined %} +{% set _ = _lsets.append((p_name,p_entry.sequence,p_entry.set.community.large)) %} +{% endfor %} +{% endfor %} +{% if routing.community|default({}) or _lsets %} +{% for c_name, c_data in (routing.community|default({})).items() %} +{% set _kind = 'large-community-set' if c_data.type == 'large' else 'community-set' %} +{% set _leaf = 'large-community-member' if c_data.type == 'large' else 'community-member' %} +routing-policy defined-sets bgp-defined-sets {{ _kind }} {{ c_name }} + {{ _leaf }} [ {{ _members(c_data) }} ] + exit +! +{% endfor %} +{% for (p_name, p_seq, p_val) in _lsets %} +routing-policy defined-sets bgp-defined-sets large-community-set SETLC_{{ p_name }}_{{ p_seq }} + large-community-member [ {{ p_val|join(' ') }} ] + exit +! +{% endfor %} +#COMMIT +{% endif %} +{% if routing.policy|default({}) %} +{% for p_name, p_entries in routing.policy.items() %} +{% for af in ['ipv4', 'ipv6'] %} +routing-policy policy-definition {{ p_name }}-{{ af }} +{% for p_entry in p_entries %} +{#- A statement whose match.prefix names a list with no permit entries in THIS address family + can never match, so it is skipped rather than rendered without its condition -- an + unconditional statement would match every prefix and apply the opposite outcome. -#} +{% set _pfx = p_entry.match.prefix|default('') %} +{% set _pfx_empty = _pfx and not (routing._prefix[af][_pfx]|default([])|rejectattr('action','eq','deny')|list) %} +{% if not _pfx_empty %} + statement s{{ p_entry.sequence|default(loop.index * 10) }} +{% if p_entry.match.prefix is defined %} + conditions match-prefix-set prefix-set {{ p_entry.match.prefix }} +{% endif %} +{#- match.community: the standard and large families are separate ArcOS conditions. The match + option comes from the shape of the referenced list (see _match_opts). -#} +{% if p_entry.match.community.standard is defined %} +{% set _cl = routing.community[p_entry.match.community.standard] %} + conditions bgp-conditions match-community-set community-set {{ p_entry.match.community.standard }} + conditions bgp-conditions match-community-set match-set-options {{ _match_opts(_cl) }} +{% endif %} +{% if p_entry.match.community.large is defined %} +{% set _cl = routing.community[p_entry.match.community.large] %} + conditions bgp-conditions match-large-community-set large-community-set {{ p_entry.match.community.large }} + conditions bgp-conditions match-large-community-set match-set-options {{ _match_opts(_cl) }} +{% endif %} +{% if p_entry.set.locpref is defined %} + actions bgp-actions set-local-pref {{ p_entry.set.locpref }} +{% endif %} +{% if p_entry.set.med is defined %} + actions bgp-actions set-med {{ p_entry.set.med }} +{% endif %} +{# set-community: ArcOS has ONLY standard communities (inline communities leaf-list). No + set-large-community / set-ext-community action exists -> not declared in features. #} +{% if p_entry.set.community.standard is defined %} + actions bgp-actions set-community method INLINE + actions bgp-actions set-community options {{ 'ADD' if p_entry.set.community.append|default(false) else 'REPLACE' }} + actions bgp-actions set-community inline communities [ {{ p_entry.set.community.standard|join(' ') }} ] +{% endif %} +{#- set.community.large: ArcOS has no INLINE form for large communities, so the values are + referenced through the SETLC_ set emitted above. (An earlier version of this template claimed + ArcOS has no set-large-community action at all and rendered a comment instead -- measured + wrong: 'method REFERENCE options ADD' adds the community, observed on a peer.) -#} +{% if p_entry.set.community.large is defined %} + actions bgp-actions set-large-community method REFERENCE + actions bgp-actions set-large-community options {{ 'ADD' if p_entry.set.community.append|default(false) else 'REPLACE' }} + actions bgp-actions set-large-community reference large-community-set-ref SETLC_{{ p_name }}_{{ p_entry.sequence }} +{% endif %} +{#- delete.community.list.: removal is a REFERENCE to a community-set with options + REMOVE, on the "-additional" leaf so that a statement which also SETS communities does not + collide with itself on the same leaf. netlab turns every delete request -- literal values as + well as an explicit list -- into a community list for us, because the device declares + features.routing.policy.delete.community: clist. -#} +{% if p_entry.delete.community.list.standard is defined %} + actions bgp-actions set-community-additional method REFERENCE + actions bgp-actions set-community-additional options REMOVE + actions bgp-actions set-community-additional reference community-set-ref {{ p_entry.delete.community.list.standard }} +{% endif %} +{% if p_entry.delete.community.list.large is defined %} + actions bgp-actions set-large-community-additional method REFERENCE + actions bgp-actions set-large-community-additional options REMOVE + actions bgp-actions set-large-community-additional reference large-community-set-ref {{ p_entry.delete.community.list.large }} +{% endif %} + actions {{ 'accept-route' if p_entry.action|default('permit') == 'permit' else 'reject-route' }} + exit +{% endif %} +{% endfor %} + exit +! +{% endfor %} +{% endfor %} +{% endif %} +{#- STATIC routes, global table and VRFs, from one macro so the two render identically. + + ArcOS scopes static routes by network-instance -- "network-instance protocol STATIC + " -- so a VRF static route is the same construct as a global one with a different + network-instance. Verified on the device: an IPv4 and an IPv6 static route committed under + "network-instance red protocol STATIC red" install into red's table only + ("10.99.99.0/24 via 172.18.0.1 dev swp1 proto static", nothing in any other VRF). + + Only ONE STATIC instance may exist per network-instance -- a second one is rejected with + "Exceeded max STATIC instances" -- so the instance tag is simply the VRF name (and "default" + in the global table, unchanged). Unlike the OSPF/IS-IS instance tags, a STATIC tag may repeat + a tag used by another protocol: "protocol STATIC red" and "protocol BGP red" coexist in the + same VRF (verified), so no p-/i-style prefix is needed. + + A discard (blackhole) route is "next-hop DROP": it installs as "blackhole proto + static" (verified). DROP is the only accepted keyword -- DISCARD, discard, blackhole, REJECT + and Null0 are all rejected by the parser. + + Inter-VRF static routes (nexthop.vrf) are NOT rendered and NOT declared: ArcOS expresses them + as "next-network-instance-name ", which takes host routes only -- a shorter IPv4 prefix + is rejected with "Expect /32 v4 prefix when configuring static route with next-network-instance + option" and IPv6 likewise needs a /128. netlab's static routes are prefix-based, so the + construct cannot carry them. See the caveats in netsim/devices/arcos.yml. -#} +{% macro static_routes(sr_list,af) %} +{% for sr_data in sr_list if sr_data[af] is defined and (sr_data.nexthop[af] is defined or sr_data.nexthop.discard is defined) %} + static-route {{ sr_data[af] }} + next-hop-index nh{{ sr_data.nexthop.idx|default(0) }} + next-hop {{ 'DROP' if sr_data.nexthop.discard is defined else sr_data.nexthop[af] }} + exit + exit +{% endfor %} +{% endmacro %} +{#- an entry with an inter-VRF next hop is skipped everywhere (see above) -#} +{% set sr_all = routing.static|default([])|rejectattr('nexthop.vrf','defined')|list %} +{% set sr_global = sr_all|rejectattr('vrf','defined')|list %} +{% if sr_global %} +network-instance default protocol STATIC default +{% for af in ['ipv4', 'ipv6'] %} +{{ static_routes(sr_global,af) -}} +{% endfor %} + exit +! +{% endif %} +{% for vname in vrfs|default({}) %} +{% set sr_vrf = sr_all|selectattr('vrf','defined')|selectattr('vrf','eq',vname)|list %} +{% if sr_vrf %} +network-instance {{ vname }} protocol STATIC {{ vname }} +{% for af in ['ipv4', 'ipv6'] %} +{{ static_routes(sr_vrf,af) -}} +{% endfor %} + exit +! +{% endif %} +{% endfor %} diff --git a/netsim/ansible/templates/sr/arcos.j2 b/netsim/ansible/templates/sr/arcos.j2 new file mode 100644 index 0000000000..4f66b0b135 --- /dev/null +++ b/netsim/ansible/templates/sr/arcos.j2 @@ -0,0 +1,25 @@ +{# + ArcOS Segment Routing (SR-MPLS via IS-IS). ALL FOUR pieces are required for the + SID to actually appear on the wire: + 1. reserved-label-block as ONE LINE (mode-descent -> "incomplete path") and it MUST carry + protocol-name : without it the SRGB never binds to ISIS and the prefix-SID + subTLV + SR_CAPABILITY are silently omitted from flooded LSPs (the LOCAL lsdb display + still shows them -- trust the receiver's copy, not the origin's). + 2. global traffic-engineering ipv4-router-id -- required to trigger the router capabilities + TLV with SR capabilities. + 3. global segment-routing enabled true. + 4. per-loopback prefix-sid (sid-type INDEX + value). + The ISIS instance name must match templates/isis/arcos.j2's ("i1"). +#} +{% if 'isis' in sr.protocol|default([]) %} +network-instance default mpls global reserved-label-block ISIS-SRGB lower-bound {{ sr.srgb.start|default(16000) }} upper-bound {{ sr.srgb.start|default(16000) + sr.srgb.size|default(8000) - 1 }} usage ISIS_SRGB protocol-identifier ISIS protocol-name i1 +! +network-instance default protocol ISIS i1 global traffic-engineering ipv4-router-id {{ loopback.ipv4|ipaddr('address') }} +network-instance default protocol ISIS i1 global segment-routing enabled true +! +{% if sr.node_sid.ipv4 is defined %} +network-instance default protocol ISIS i1 interface loopback0 af IPV4 UNICAST prefix-sid SPF sid-type INDEX +network-instance default protocol ISIS i1 interface loopback0 af IPV4 UNICAST prefix-sid SPF value {{ sr.node_sid.ipv4 }} +! +{% endif %} +{% endif %} diff --git a/netsim/ansible/templates/srv6/arcos.j2 b/netsim/ansible/templates/srv6/arcos.j2 new file mode 100644 index 0000000000..719074d7bd --- /dev/null +++ b/netsim/ansible/templates/srv6/arcos.j2 @@ -0,0 +1,18 @@ +{# + ArcOS SRv6 template. SRv6 lives under `network-instance default srv6` + (encapsulation + locator) and binds to IS-IS via `protocol ISIS i1 global srv6 + enabled true|locator `. ArcOS derives an End SID from the locator prefix automatically. + netlab data: srv6.locator (per-node /48 from srv6.locator_pool), srv6.igp. Only isis is + declared -- bgp/vpn/ospf-over-SRv6 not exercised. +#} +{% if srv6.locator is defined %} +{% set locname = inventory_hostname ~ "-loc" %} +network-instance default srv6 encapsulation source-address {{ loopback.ipv6|ansible.utils.ipaddr("address") }} +network-instance default srv6 locator {{ locname }} prefix {{ srv6.locator }} +{% if "isis" in srv6.igp|default([]) %} +network-instance default protocol ISIS i1 global af IPV6 UNICAST enabled true +network-instance default protocol ISIS i1 interface loopback0 af IPV6 UNICAST enabled true +network-instance default protocol ISIS i1 global srv6 enabled true +network-instance default protocol ISIS i1 global srv6 locator {{ locname }} +{% endif %} +{% endif %} diff --git a/netsim/ansible/templates/stp/arcos.j2 b/netsim/ansible/templates/stp/arcos.j2 new file mode 100644 index 0000000000..18cf00568b --- /dev/null +++ b/netsim/ansible/templates/stp/arcos.j2 @@ -0,0 +1,51 @@ +{#- + ArcOS spanning tree -- Rapid PVST only. + + The "stp enabled-protocol" leaf's own value list is { NONE | RAPID_PVST } (read off the leaf, not + guessed from a "?" completion), and RAPID_PVST is the device default -- ArcOS runs spanning tree + out of the box. There is no 802.1D, no plain RSTP and no MSTP on this image, so + features.stp.supported_protocols declares 'pvrst' alone and netlab rejects every other protocol + before this template ever runs. That is why the enabled branch below hard-codes RAPID_PVST + instead of mapping stp.protocol: any other value is already an error. + + Two shapes of the ArcOS model differ from netlab's and are converted here: + + * there is NO global bridge priority -- "stp rapid-pvst vlan bridge-priority" is the only + one there is -- so a node-level stp.priority is fanned out to every VLAN the node knows; + * "port-priority" is the 8-bit on-the-wire form (0..240) while netlab's stp.port_priority is the + 4-bit field (0..15), hence the x16, and it lives per VLAN per interface rather than per + interface, so it is fanned out across the VLANs the port carries. + + NOT rendered, deliberately: + * per-port enable/disable -- "stp interface " has no 'enabled' leaf, the only off switch is + the global one below, and features.stp.enable_per_port: False makes netlab say so; + * per-VLAN enable/disable -- "stp rapid-pvst vlan " has no 'enabled' leaf either. netlab core + does NOT gate vlans..stp.enable against any device feature, so that request would transform + and deploy and leave the device in a state the topology does not describe; the ArcOS quirk in + netsim/devices/arcos.py refuses it instead; + * stp.port_type -- edge/auto/normal would map onto "stp interface edge-port", but 'network' + has no per-port equivalent (ArcOS bridge assurance is the global "stp global bridge-assurance"), + and features.stp.port_type is a bare boolean that cannot express the subset. Undeclared, so + netlab refuses it; + * bpdu-guard -- "stp global bpdu-guard true" commits and appears in the running config, but + "show stp global state bpdu-guard" still reads false. Config the device accepts and does not + act on is worse than no config at all. +-#} +{% if not stp.enable|default(True) %} +stp enabled-protocol NONE +{% else %} +stp enabled-protocol RAPID_PVST +{% for vdata in (vlans|default({})).values() %} +{#- a per-VLAN priority wins over the node-level one; the node-level value has to be repeated on + every VLAN because ArcOS has no bridge-wide priority to put it on -#} +{% set priority = vdata.stp.priority|default(stp.priority|default(None)) %} +{% if priority is not none %} +stp rapid-pvst vlan {{ vdata.id }} bridge-priority {{ priority }} +{% endif %} +{% endfor %} +{% for l in interfaces|default([]) if l.stp.port_priority is defined %} +{% for vlan_id in l.vlan.trunk_id|default([l.vlan.access_id] if l.vlan.access_id is defined else []) %} +stp rapid-pvst vlan {{ vlan_id }} interface {{ l.ifname }} port-priority {{ l.stp.port_priority * 16 }} +{% endfor %} +{% endfor %} +{% endif %} diff --git a/netsim/ansible/templates/vlan/arcos.j2 b/netsim/ansible/templates/vlan/arcos.j2 new file mode 100644 index 0000000000..88f89c4655 --- /dev/null +++ b/netsim/ansible/templates/vlan/arcos.j2 @@ -0,0 +1,56 @@ +{#- + ArcOS VLAN -- NATIVE model. The OpenConfig-ish form (network-instance L2VLAN + + routed-vlan addressing) commits and even shows in oper state, but NEVER programs the kernel + IP => dead datapath in the native container. The working model: + vlan name -- top-level VLAN + interface swpX ethernet switched-vlan interface-mode ACCESS|TRUNK ... + interface vlan subinterface 0 ipv4 -- SVI (autostate false = up w/o members) + The two models CONFLICT ("illegal reference 'vlan vlan-id'") -- never mix. +-#} +{% if vlans is defined %} +{% for vname, vdata in vlans.items() %} +{# each named vlan needs its OWN commit (two in one transaction -> "illegal reference + 'vlan N vlan-id'") and must be in running before trunk-vlans references it #} +vlan {{ vdata.id }} name {{ vname }} +#COMMIT +{% endfor %} +{% endif %} +{% for l in interfaces|default([]) %} +{#- the switched-vlan container hangs off `ethernet` on a physical port but off `aggregation` + on a LAG bond (the OpenConfig model ArcOS follows) -- addressing a bond through `ethernet` + is rejected with "syntax error: element does not exist" -#} +{% set sw = 'aggregation' if l.type|default('') == 'lag' else 'ethernet' %} +{% if l.vlan is defined and l.vlan.access is defined %} +interface {{ l.ifname }} {{ sw }} switched-vlan interface-mode ACCESS access-vlan {{ l.vlan.access_id }} +{% elif l.vlan is defined and l.vlan.trunk is defined %} +interface {{ l.ifname }} {{ sw }} switched-vlan interface-mode TRUNK +{# native VLAN on a trunk: a SEPARATE leaf alongside trunk-vlans. Live probe found it MUST be + excluded from trunk-vlans ("Native vlan N cannot be in trunk-vlans list" on commit) even + though netlab's own vlan.native/vlan.trunk model keeps the native vlan IN the trunk list -- + so the trunk-vlans loop below filters it out. #} +{% if l.vlan.native is defined %} +interface {{ l.ifname }} {{ sw }} switched-vlan native-vlan {{ vlans[l.vlan.native].id }} +{% endif %} +interface {{ l.ifname }} {{ sw }} switched-vlan trunk-vlans [ {% for vname in l.vlan.trunk if vname != l.vlan.native|default("") %}{{ vlans[vname].id }} {% endfor %}] +{% endif %} +{% endfor %} +{% for l in interfaces|default([]) %} +{% if l.type|default("") == "svi" %} +interface {{ l.ifname }} enabled true +interface {{ l.ifname }} autostate false +{# An SVI needs the same treatment as any other L3 interface (templates/initial/arcos.j2): + IPv4 and IPv6 addresses rendered side by side from the same interface data, and the MTU + leaf set to l.mtu + 26 because ArcOS programs the real netdev (and the MTU OSPF advertises) + as (leaf - 26). Without the MTU leaf an SVI silently inherits the widest member port's MTU, + which stalls an OSPF adjacency over that VLAN in ExStart. #} +{% if l.mtu is defined %} +interface {{ l.ifname }} mtu {{ l.mtu + 26 }} +{% endif %} +{% if l.ipv4 is defined and l.ipv4 is string %} +interface {{ l.ifname }} subinterface 0 ipv4 address {{ l.ipv4 | ansible.utils.ipaddr('address') }} prefix-length {{ l.ipv4 | ansible.utils.ipaddr('prefix') }} +{% endif %} +{% if l.ipv6 is defined and l.ipv6 is string %} +interface {{ l.ifname }} subinterface 0 ipv6 address {{ l.ipv6 | ansible.utils.ipaddr('address') }} prefix-length {{ l.ipv6 | ansible.utils.ipaddr('prefix') }} +{% endif %} +{% endif %} +{% endfor %} diff --git a/netsim/ansible/templates/vrf/arcos.j2 b/netsim/ansible/templates/vrf/arcos.j2 new file mode 100644 index 0000000000..2b5f14bded --- /dev/null +++ b/netsim/ansible/templates/vrf/arcos.j2 @@ -0,0 +1,241 @@ +{% import "templates/bgp/arcos.macro.j2" as bgpcfg %} +{#- ArcOS per-VRF config: L3VRF network-instance + interface binding + per-VRF OSPFv2/v3/BGP/ISIS. + NOT declared (measured as unsupported, not just untested): import/export route-target leaking + between VRFs -- the export half can be driven to "Export ready" and puts correctly targeted VPN + routes in the per-VRF exported-rib, but a locally exported VPN path never reaches the local + import machinery (default-instance L3VPN loc-rib stays empty, every VRF stays at + "paths-imported 0"), so nothing leaks on a single device. See netsim/devices/arcos.yml. + + Redistribution of connected subnets into a VRF BGP instance IS supported, but not through a + table-connection: DIRECTLY_CONNECTED has no addressable source instance on this image, so the + connected() macro in templates/bgp/arcos.macro.j2 renders one BGP network statement per + connected prefix instead. + + Kept CONSISTENT with the global-table templates (theme: global vs VRF parity): the per-VRF + OSPF renders the same per-interface parameters (network-type/metric/bfd/priority/timers) and + the same multi-area layout as templates/ospf/arcos.j2, and the per-VRF IS-IS carries the same + dual-AF handling (IPv4 and IPv6 unicast, each gated on vdata.isis.af.) as + templates/isis/arcos.j2. The IPv6 IS-IS single-topology caveat there applies here too + (cross-vendor IPv6 IS-IS is unsupported). + + A protocol *instance tag* ("p1", "i1") is a GLOBAL namespace + across every network-instance on this ArcOS build, NOT scoped per network-instance -- reusing + the default instance's tag for a VRF's protocol instance aborts the commit ("Protocol instance + 'p1'/'i1' already configured under network-instance 'default'"). Tag every VRF protocol + instance with the VRF name so it can never collide with the default instance or another VRF's + instance of the same protocol. -#} +{%- macro ospf_if_params(l, ospf) %} +{# same fix as templates/ospf/arcos.j2: only fall back to l.type == 'p2p' when + ospf.network_type wasn't explicitly set, so an explicit "broadcast" isn't overridden #} +{% set nt = l.ospf.network_type|default('point-to-point' if l.type|default('') == 'p2p' else '') %} +{% if nt == 'point-to-point' %} + network-type POINT_TO_POINT_NETWORK +{% endif %} +{% if l.ospf.cost is defined %} + metric {{ l.ospf.cost }} +{% endif %} +{#- Passive interface, same as templates/ospf/arcos.j2 (global/VRF parity). -#} +{% if l.ospf.passive|default(false) %} + passive true +{% endif %} +{% if l.ospf.bfd|default(ospf.bfd|default(false)) %} + bfd enabled true +{% endif %} +{% if l.ospf.priority is defined %} + priority {{ l.ospf.priority }} +{% endif %} +{% if l.ospf.timers.hello is defined %} + timers hello-interval {{ l.ospf.timers.hello }} +{% endif %} +{% if l.ospf.timers.dead is defined %} + timers dead-interval {{ l.ospf.timers.dead }} +{% endif %} +{%- endmacro %} +{% if vrfs is defined %} +{% for vname, vdata in vrfs.items() %} +network-instance {{ vname }} type L3VRF +{# templates/initial/arcos.j2 already created the instance and bound every interface it + configures itself (it has to: an address is validated against the network-instance the + interface is in). Repeating those bindings is a no-op and picks up the interfaces created by + the modules that run in between -- an SVI cannot be bound before the VLAN template creates + it ("Interface vlanN subinterface 0 does not exist"). #} +{% for l in interfaces|default([]) %} +{% if l.vrf|default("") == vname %} +network-instance {{ vname }} interface {{ l.ifname }} +{% endif %} +{% endfor %} +{#- per-VRF OSPFv2, gated on vdata.ospf.af.ipv4 exactly like the OSPFv3 block below: netlab + only sets vrf.ospf.af. for the families the VRF has, so without the gate an + IPv6-only VRF got an OSPFv2 instance with a router ID and no areas. -#} +{% if vdata.ospf is defined and vdata.ospf.af.ipv4|default(false) %} +{% set _vdflt = vdata.ospf.area|default('0.0.0.0') %} +{% set _vareas = vdata.ospf.areas|default([{'area': _vdflt, '_area_int': 0}]) %} +network-instance {{ vname }} protocol OSPF p-{{ vname }} + global router-id {{ vdata.ospf.router_id }} +{% for a in _vareas %} +{% set _body %} +{% for l in vdata.ospf.interfaces|default([]) if l.ipv4 is defined and (l.ospf.area|default(_vdflt)) == a.area %} + interface {{ l.ifname }} +{{ ospf_if_params(l, ospf) }} +{% endfor %} +{% endset %} +{% if _body.strip() %} + area {{ a.area }} +{{ _body }} +{% endif %} +{% endfor %} +{% endif %} +{#- per-VRF OSPFv3: SEPARATE `protocol OSPF3` instance inside the SAME L3VRF network-instance, + mirroring the default-instance dual-stack pattern in templates/ospf/arcos.j2 (area written + as a bare integer, not dotted-decimal). Gated on vdata.ospf.af.ipv6 (netlab vrf.ospf.af, + feature vrf.ospfv3) so v4-only VRF OSPF topologies are unaffected. -#} +{% if vdata.ospf is defined and vdata.ospf.af.ipv6|default(false) %} +{% set _vdflt = vdata.ospf.area|default('0.0.0.0') %} +{% set _vareas = vdata.ospf.areas|default([{'area': _vdflt, '_area_int': 0}]) %} +network-instance {{ vname }} protocol OSPF3 p-{{ vname }} + global router-id {{ vdata.ospf.router_id }} +{% for a in _vareas %} +{% set _body %} +{% for l in vdata.ospf.interfaces|default([]) if l.ipv6 is defined and (l.ospf.area|default(_vdflt)) == a.area %} + interface {{ l.ifname }} +{{ ospf_if_params(l, ospf) }} +{% endfor %} +{% endset %} +{% if _body.strip() %} + area {{ a._area_int }} +{{ _body }} +{% endif %} +{% endfor %} +{% endif %} +{#- per-VRF BGP: a BGP protocol instance inside the L3VRF network-instance, rendered with the + same macros as the default-instance BGP in templates/bgp/arcos.j2 (global/VRF parity: same + router-id / address-family / network / neighbor handling, IPv4 and IPv6 alike). + + ArcOS specifics, all verified on the device: + * the AS number is NOT part of the per-VRF model ("global as" does not exist inside an + L3VRF) -- the VRF instance inherits the AS from the default instance, which the netlab + data model guarantees is there (a VRF BGP neighbor implies the node runs the bgp module); + * "global afi-safi _UNICAST" must be present before a neighbor may activate that AF + ("BGP global AF must be configured first"), but instance, AFs and neighbors all commit in + a SINGLE transaction -- unlike the table-connections, this needs no #COMMIT split; + * the protocol instance tag is "{{ vname }}" to match templates/evpn/arcos.j2, which adds + the L3VNI route-distinguisher and the L3VPN AF to that same instance. + + VRF route-target import/export leaking is NOT rendered: local RT/L3VPN leaking does not work + on this image, and the one local mechanism that does work (a static route with + next-network-instance-name) is /32-only and cannot carry netlab's prefix lists. Neither is a + declared feature -- see the caveats in netsim/devices/arcos.yml for both measurements. + + The VRF module sets vrfs..bgp on every VRF once it needs an AS number to build the + route distinguishers, so 'vdata.bgp is defined' is not enough to decide there is anything to + configure -- the guard right below spells out what actually counts. The node also has to run + the BGP module, because a VRF instance inherits its AS from the default instance. -#} +{#- Render the instance when the VRF has anything for it to carry: neighbors, networks to + advertise, or a route import (netlab auto-generates bgp.import for a VRF that runs an IGP, and + an EVPN L3VNI VRF can have an import and no neighbors at all -- evpn/22-ospf-ce-router). The + AF check keeps an address-less VRF from producing a BGP instance with no address family. -#} +{% if 'bgp' in module and (vdata.bgp.neighbors|default([]) or vdata.bgp.advertise|default([]) + or (vdata.bgp.import|default({}) and vdata.af|default({}))) %} +network-instance {{ vname }} protocol BGP {{ vname }} + global router-id {{ vdata.bgp.router_id|default(bgp.router_id) }} + global route-distinguisher {{ vdata.rd }} +{{ bgpcfg.afi_safi(vdata.af|default({})) -}} +{{ bgpcfg.advertise(vdata.bgp.advertise|default([]),vdata.af|default({})) -}} +{#- bgp.import.connected inside the VRF: the connected subnets of the interfaces bound to this + VRF, rendered as BGP network statements by the same macro the default instance uses. netlab + generates this import automatically for every VRF (features.bgp.import carries 'connected' + and 'vrf'), which is what makes a CE-to-CE path across the PE work: without the PE-CE subnets + in VRF BGP a CE has no route back to the far CE's interface address. -#} +{% if vdata.bgp.import.connected is defined %} +{{ bgpcfg.connected( + interfaces|default([])|selectattr('vrf','defined')|selectattr('vrf','eq',vname)|list, + vdata.af|default({}),vdata.bgp.import.connected,vdata.bgp.advertise|default([])) -}} +{% endif %} +{{ bgpcfg.neighbors(vdata.bgp.neighbors|default([]),vdata.af|default({})) -}} +{% endif %} +{#- per-VRF IS-IS: network-instance protocol ISIS -- verified: instance/net/level + + interface bindings commit inside an L3VRF, adjacency forms in-VRF. IPv4/IPv6 unicast each + gated on vdata.isis.af. (same as templates/isis/arcos.j2, incl. the single-topology + IPv6 caveat). -#} +{% if vdata.isis is defined %} +{% set vcaps = {"level-1":"LEVEL_1","level-2":"LEVEL_2","level-1-2":"LEVEL_1_2"} %} +{% set vtyp = vdata.isis.type|default("level-2") %} +network-instance {{ vname }} protocol ISIS i-{{ vname }} + global net {{ vdata.isis.net }} + global level-capability {{ vcaps[vtyp]|default("LEVEL_2") }} +{% if vdata.isis.af.ipv4|default(false) %} + global af IPV4 UNICAST enabled true +{% endif %} +{% if vdata.isis.af.ipv6|default(false) %} + global af IPV6 UNICAST enabled true +{% endif %} +{% set vdo_l1 = vtyp in ["level-1","level-1-2"] %} +{% set vdo_l2 = vtyp in ["level-2","level-1-2"] %} +{% if vdo_l1 %} + level 1 enabled true +{% endif %} +{% if vdo_l2 %} + level 2 enabled true +{% endif %} +{% for l in vdata.isis.interfaces|default([]) %} + interface {{ l.ifname }} enabled true +{% if vdata.isis.af.ipv4|default(false) %} + interface {{ l.ifname }} af IPV4 UNICAST enabled true +{% endif %} +{% if vdata.isis.af.ipv6|default(false) %} + interface {{ l.ifname }} af IPV6 UNICAST enabled true +{% endif %} +{% set vnt = l.isis.network_type|default('point-to-point' if l.type|default('') == 'p2p' else '') %} +{% if vnt == 'point-to-point' %} + interface {{ l.ifname }} network-type POINT_TO_POINT +{% endif %} +{% if vdo_l1 %} + interface {{ l.ifname }} level 1 enabled true +{% endif %} +{% if vdo_l2 %} + interface {{ l.ifname }} level 2 enabled true +{% endif %} +{#- Per-interface metric, same shape as templates/isis/arcos.j2 (global/VRF parity): + ArcOS keeps it on the interface's level container, so it is emitted per level. -#} +{% if l.isis.cost is defined or l.isis.metric is defined %} +{% set _metric = l.isis.metric|default(l.isis.cost) %} +{% if vdo_l1 %} + interface {{ l.ifname }} level 1 metric {{ _metric }} +{% endif %} +{% if vdo_l2 %} + interface {{ l.ifname }} level 2 metric {{ _metric }} +{% endif %} +{% endif %} +{% if l.isis.passive|default(false) %} + interface {{ l.ifname }} passive true +{% endif %} +{% endfor %} +{% endif %} +{% endfor %} +{#- IGP -> VRF BGP redistribution, in a SECOND pass over the VRFs. + + It cannot go in the loop above: ArcOS rejects a table-connection that references a protocol + instance created in the same transaction ("SRC/DST instance does not exist"), so every VRF's + OSPF/OSPF3/ISIS/BGP instance has to be committed first. One #COMMIT marker (see + provider/clab/arcos/netlab-config.j2) splits the whole module render in two, and this pass + then renders the table-connections for every VRF with the shared macro the default routing + table uses. The per-VRF instance tags are the ones the loop above emitted: "p-" for + OSPFv2/OSPFv3, "i-" for IS-IS, "" for BGP. -#} +{% set _vrf_redist = [] %} +{% for vname, vdata in vrfs.items() if 'bgp' in module %} +{% if vdata.bgp.import.ospf is defined and vdata.ospf is defined + or vdata.bgp.import.isis is defined and vdata.isis is defined %} +{% set _ = _vrf_redist.append((vname,vdata)) %} +{% endif %} +{% endfor %} +{% if _vrf_redist %} +#COMMIT +{% set _vrf_pol = [] %} +{% for vname, vdata in _vrf_redist %} +{{ bgpcfg.redistribute_igp(vname,vdata.bgp.import,vdata.af|default({}),vname, + {'ospf': 'p-' ~ vname if vdata.ospf is defined else none, + 'isis': 'i-' ~ vname if vdata.isis is defined else none}, + _vrf_pol) -}} +{% endfor %} +{% endif %} +{% endif %} diff --git a/netsim/ansible/templates/vxlan/arcos.j2 b/netsim/ansible/templates/vxlan/arcos.j2 new file mode 100644 index 0000000000..06eb243eb7 --- /dev/null +++ b/netsim/ansible/templates/vxlan/arcos.j2 @@ -0,0 +1,56 @@ +{# + ArcOS VXLAN. VTEP = `overlay local-tunnel-endpoint 0 source-interface loopback0`, committed + separately (#COMMIT) before anything references it. + + The two flooding models are MUTUALLY EXCLUSIVE on ArcOS -- static VXLAN is gated by the global + `overlay static-vxlan` knob, and while it is on the device refuses the EVPN configuration + templates/evpn/arcos.j2 emits ("'network-instance advertise-mac-routes' must not be true when + 'overlay static-vxlan' is true"). Exactly one of the two blocks below therefore renders, and + netsim/devices/arcos.py refuses a topology that asks for both. + + * vxlan.flooding == evpn: the VNI lives on the auto-created per-VLAN L2VLAN network-instance + (`network-instance vlan vni `); BUM replication comes from the EVPN type-3 routes + templates/evpn/arcos.j2 sets up. + * vxlan.flooding == static (netlab's default): head-end replication towards a configured + remote-VTEP list. ArcOS does NOT keep that list on the per-VLAN L2VLAN instance -- it uses a + dedicated L2VLAN_AWARE_BUNDLE network-instance carrying `static-vxlan remote-vteps` plus the + VNI-to-VLAN mapping of every VLAN sharing that VTEP set. netlab tracks the remote VTEPs per + VLAN (vlans..vtep_list), so the VLANs are grouped by their VTEP list and one bundle is + rendered per distinct list -- the exact shape ArcOS documents for that case. +#} +{% if vxlan.flooding|default('static') != 'evpn' %} +overlay static-vxlan true +{% endif %} +overlay local-tunnel-endpoint 0 source-interface {{ vxlan.vtep_interface|default('loopback0') }} +#COMMIT +{% if vxlan.flooding|default('static') == 'evpn' %} +{% for vname in vxlan.vlans|default([]) %} +network-instance vlan{{ vlans[vname].id }} vni {{ vlans[vname].vni }} local-tunnel-endpoint-id 0 +{% endfor %} +{% else %} +{% set bundles = {} %} +{% for vname in vxlan.vlans|default([]) %} +{% set vteps = vlans[vname].vtep_list|default([])|join(' ') %} +{% set _ = bundles.update({ vteps: bundles.get(vteps,[]) + [ vname ] }) %} +{% endfor %} +{% for vteps, vnames in bundles.items() %} +{#- Name the bundle after the lowest VNI it carries rather than the enumeration index, so the + same VTEP set always lands on the same network-instance no matter how vxlan.vlans is + ordered. -#} +{% set vni_list = [] %} +{% for vname in vnames %}{% set _ = vni_list.append(vlans[vname].vni) %}{% endfor %} +{% set bundle = 'static-vxlan-' ~ vni_list|min %} +network-instance {{ bundle }} type L2VLAN_AWARE_BUNDLE +network-instance {{ bundle }} local-tunnel-endpoint-id 0 +{# A VLAN with no remote VTEP still needs its VNI-to-VLAN mapping, so the bundle is rendered + without a `static-vxlan remote-vteps` leaf. The vendor guide never shows that shape, but + ArcOS accepts it and vxlan/04-vxlan-irb-ospf (which has exactly one such VLAN) passes 7/7 + with it -- deliberate, not an oversight. #} +{% if vteps %} +network-instance {{ bundle }} static-vxlan remote-vteps [ {{ vteps }} ] +{% endif %} +{% for vname in vnames %} +network-instance {{ bundle }} vni {{ vlans[vname].vni }} vlan-id {{ vlans[vname].id }} +{% endfor %} +{% endfor %} +{% endif %} diff --git a/netsim/devices/arcos.py b/netsim/devices/arcos.py new file mode 100644 index 0000000000..96b8550fd1 --- /dev/null +++ b/netsim/devices/arcos.py @@ -0,0 +1,138 @@ +# +# Arrcus ArcOS quirks +# +import ipaddress + +from box import Box + +from ..utils import log +from . import _Quirks, report_quirk + + +class ARCOS(_Quirks): + + @classmethod + def device_quirks(self, node: Box, topology: Box) -> None: + mods = node.get('module', []) + + # Static VXLAN flooding and the EVPN control plane are mutually exclusive on ArcOS: the global + # 'overlay static-vxlan' knob switches the whole box into flood-and-learn mode, and the device + # then refuses the EVPN configuration templates/evpn/arcos.j2 emits -- + # Aborted: 'overlay static-vxlan' (value "true"): 'network-instance advertise-mac-routes' + # must not be true when 'overlay static-vxlan' is true + # netlab core allows the combination (an EVPN node keeps 'static' among vxlan.flooding's valid + # values), so without this check the topology transforms and deploys and only dies partway + # through 'netlab initial' with a raw vendor error. Refuse it up front instead: rendering the + # EVPN half and silently dropping the static request -- or the reverse -- would leave the user + # with a device configured differently from the topology they wrote. + # + # Gated on the vxlan module as well: an EVPN node without it (a pure EVPN route reflector, or + # an EVPN/MPLS PE) has no VTEP and no flooding mode, so there is nothing to conflict with. + if 'evpn' in mods and 'vxlan' in mods and node.get('vxlan.flooding', 'static') != 'evpn': + report_quirk( + text=f'ArcOS cannot combine EVPN with static VXLAN flooding (node {node.name})', + more_hints=[ + 'ArcOS "overlay static-vxlan" is a global flood-and-learn mode that the device refuses ' + 'to run together with the EVPN control plane', + f'Use vxlan.flooding: evpn on node {node.name}, or remove the evpn module from it' ], + node=node, + quirk='evpn_static_flooding', + category=log.IncorrectValue) + + # An ArcOS community-set has no per-member action -- it carries a single match option for the + # whole set (ANY / ALL / INVERT). A netlab community list has an action per entry, so only two + # shapes have a faithful rendering (templates/routing/arcos.j2): + # + # * every entry permits -> the members are the set, match-set-options ANY + # * deny entries plus a catch-all '.*' permit -> the denied members are the set, and the sense + # is flipped with match-set-options INVERT + # + # Anything else -- a permit and a deny of different values in one list, or deny entries with no + # catch-all -- would have to be approximated, and an approximated route-map match is the kind of + # thing nobody notices until traffic goes the wrong way. Refuse it instead. + if node.get('routing.community', None): + for cl_name, cl_data in node.routing.community.items(): + entries = cl_data.get('value', []) + deny = [e for e in entries if e.get('action', 'permit') == 'deny'] + if not deny: # All-permit list: ANY, always fine + continue + other = [e for e in entries if e.get('action', 'permit') != 'deny'] + # The deny form is expressible only as "matches none of these", i.e. every non-deny entry + # has to be the catch-all that makes the rest of the list a pure negation. + if other and all(e.get('_value', '') == '.*' for e in other): + continue + report_quirk( + text=f'ArcOS cannot express BGP community list {cl_name} on node {node.name}', + more_hints=[ + 'An ArcOS community-set has one match option for the whole set (ANY/ALL/INVERT) and no ' + 'per-member action, so it can express "matches any of these" or "matches none of these" ' + 'and nothing in between', + f'Rewrite {cl_name} as a list of permit entries, or as deny entries followed by a ' + 'catch-all permit' ], + node=node, + quirk='community_list_shape', + category=log.IncorrectValue) + + # An ArcOS prefix-set is a flat member list: the accept/reject decision lives on the policy + # statement that references it, not on the individual members. A netlab prefix list carries an + # action per entry, and its semantics are first-match-wins, so a 'deny' entry means "this + # prefix does NOT match the list". + # + # templates/routing/arcos.j2 renders the PERMIT entries only, which is faithful as long as no + # deny entry shadows a permit entry that comes after it. When they overlap, order is the only + # thing separating them and a flat set cannot carry order: a prefix caught by the earlier deny + # would still match the later permit's member and be treated as matching the list. That is a + # route-map matching the wrong routes -- silent, and visible only as traffic taking the wrong + # path -- so refuse it rather than approximate it. + for pfx_name, pfx_list in node.get('routing.prefix', {}).items(): + for (idx, deny) in enumerate(pfx_list): + if deny.get('action', 'permit') != 'deny': + continue + for later in pfx_list[idx + 1:]: + if later.get('action', 'permit') == 'deny': + continue + for af in ('ipv4', 'ipv6'): # Same family, and do the nets overlap? + if af not in deny or af not in later: + continue + if not ipaddress.ip_network(deny[af]).overlaps(ipaddress.ip_network(later[af])): + continue + report_quirk( + text=f'ArcOS cannot express prefix list {pfx_name} on node {node.name}: ' + f'the deny entry {deny[af]} shadows the later permit entry {later[af]}', + more_hints=[ + 'An ArcOS prefix-set is a flat member list with no per-entry action and no ordering, ' + 'so a deny entry can only be expressed by leaving the prefix out of the set -- which ' + 'does not work when a later permit entry covers it as well', + f'Reorder {pfx_name} so no deny entry overlaps a permit entry below it, or split it ' + 'into separate lists' ], + node=node, + quirk='prefix_list_shape', + category=log.IncorrectValue) + + # ArcOS can only turn spanning tree off for the WHOLE box: the off switch is the global + # "stp enabled-protocol NONE". "stp rapid-pvst vlan " has no 'enabled' leaf, and neither + # does "stp interface " -- there is nowhere to put a per-VLAN disable request. + # + # netlab core gates the per-interface form (intf.stp.enable) against features.stp.enable_per_port, + # which ArcOS declares False, so that one is already refused. It does NOT gate the per-VLAN form + # against anything, so vlans..stp.enable: False transforms, renders and deploys -- and + # templates/stp/arcos.j2 has nothing to render for it, leaving spanning tree RUNNING on a VLAN + # the topology says it should be off on. That is a wrong topology rather than an error, and on + # a node that also carries stp.priority the rendered config makes the device the root bridge of + # exactly the VLAN the user asked to exclude. Refuse the combination up front instead. + if 'stp' in mods: + no_stp_vlans = [ + vname for vname, vdata in node.get('vlans', {}).items() + if vdata.get('stp.enable', True) is False ] + if no_stp_vlans: + report_quirk( + text=f'ArcOS cannot disable STP on individual VLANs (node {node.name}, ' + f'VLAN(s) {",".join(no_stp_vlans)})', + more_hints=[ + 'ArcOS has no per-VLAN "enabled" leaf -- the only off switch is the global ' + '"stp enabled-protocol NONE", which stops spanning tree on the whole device', + f'Use stp.enable: False on node {node.name} to stop STP everywhere, or remove the ' + 'per-VLAN stp.enable setting' ], + node=node, + quirk='stp_vlan_disable', + category=log.IncorrectValue) diff --git a/netsim/devices/arcos.yml b/netsim/devices/arcos.yml new file mode 100644 index 0000000000..562552806e --- /dev/null +++ b/netsim/devices/arcos.yml @@ -0,0 +1,447 @@ +--- +# Arrcus ArcOS device definition -- a containerlab (arrcus_arcos) network device. +# +# CONFIG DEPLOYMENT +# +# netlab's native "sh" config mode (netlab.tools/dev/config/deploy/#linux-configuration-scripts): +# each module's configuration is rendered into /config/netlab/NN-.sh with a +# '#!/config/netlab/netlab-config.sh' shebang and deployed with `docker exec