virtual router: Add route-maps to BGP peers for Routed Mode by wido · Pull Request #9964 · apache/cloudstack

wido · 2024-11-22T15:53:39Z

It is best practice, and mandatory in newer version of FRR, that route-maps should be applied to BGP peers. This is to prevent that mistakes can propogate through a network and cause outages.

This change changes the route-maps where the VR will only accept IPv4 and IPv4 default gateways (0.0.0.0/0 and ::/0) to be sent by the upstream router to the VR.

The other way around this change makes sure that FRR will not allow announcing anything else than the locally defined subnets to the upstream BGP router.

codecov · 2024-11-22T16:02:53Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 19.64%. Comparing base (75748a5) to head (d2f1f04).

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #9964      +/-   ##
============================================
- Coverage     19.65%   19.64%   -0.01%     
  Complexity    19792    19792              
============================================
  Files          6368     6368              
  Lines        575107   575107              
  Branches      70370    70370              
============================================
- Hits         113016   113002      -14     
- Misses       449808   449823      +15     
+ Partials      12283    12282       -1

Flag	Coverage Δ
uitests	`3.41% <ø> (ø)`
unittests	`20.92% <ø> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

weizhouapache · 2024-11-24T15:39:24Z

thanks @wido !

weizhouapache · 2024-11-24T15:39:31Z

@blueorangutan package

blueorangutan · 2024-11-24T15:40:04Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-11-24T16:47:43Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11599

weizhouapache · 2024-11-25T07:51:45Z

@blueorangutan test ubuntu24 kvm-ubuntu24

blueorangutan · 2024-11-25T07:54:04Z

@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu24 mgmt + kvm-ubuntu24) has been kicked to run smoke tests

blueorangutan · 2024-11-25T23:28:50Z

[SF] Trillian test result (tid-11794)
Environment: kvm-ubuntu24 (x2), Advanced Networking with Mgmt server u24
Total time taken: 54170 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9964-t11794-kvm-ubuntu24.zip
Smoke tests completed. 139 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_oobm_background_powerstate_sync	`Failure`	20.83	test_outofbandmanagement.py
test_oobm_enabledisable_across_clusterzones	`Error`	38.04	test_outofbandmanagement.py
test_oobm_issue_power_cycle	`Error`	19.81	test_outofbandmanagement.py
test_oobm_issue_power_off	`Error`	19.77	test_outofbandmanagement.py
test_oobm_issue_power_on	`Error`	19.80	test_outofbandmanagement.py
test_oobm_issue_power_reset	`Error`	19.84	test_outofbandmanagement.py
test_oobm_issue_power_soft	`Error`	19.80	test_outofbandmanagement.py
test_oobm_issue_power_status	`Error`	18.79	test_outofbandmanagement.py
test_oobm_multiple_mgmt_server_ownership	`Failure`	28.12	test_outofbandmanagement.py
test_oobm_zchange_password	`Error`	7.45	test_outofbandmanagement.py
test_hostha_kvm_host_degraded	`Error`	12.65	test_hostha_kvm.py
test_hostha_kvm_host_fencing	`Error`	10.14	test_hostha_kvm.py
test_hostha_kvm_host_recovering	`Error`	10.41	test_hostha_kvm.py

weizhouapache · 2024-11-30T07:57:10Z

@blueorangutan package

blueorangutan · 2024-11-30T07:58:03Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-11-30T08:44:57Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11666

weizhouapache

code lgtm

need testing

wido · 2026-01-13T11:58:23Z

@wido Since this is for the 4.22.1 release, could you retarget the PR to the 4.22 branch?

When I change the branch to 4.22 a lot of commits seem to break. I'm fine with putting this into main.

wido · 2026-07-01T09:07:55Z

This one seems ready to merge, agree @weizhouapache ? If so, can we merge it?

weizhouapache · 2026-07-01T09:12:11Z

This one seems ready to merge, agree @weizhouapache ? If so, can we merge it?

@wido
I will test it

weizhouapache · 2026-07-01T10:05:44Z

@blueorangutan package

blueorangutan · 2026-07-01T10:08:04Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-07-01T11:02:41Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18428

weizhouapache · 2026-07-01T11:12:25Z

@blueorangutan test

blueorangutan · 2026-07-01T11:14:04Z

@weizhouapache a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2026-07-02T03:03:46Z

[SF] Trillian test result (tid-16457)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 54106 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9964-t16457-kvm-ol8.zip
Smoke tests completed. 150 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestClusterDRS>:setup	`Error`	0.00	test_cluster_drs.py
test_list_system_vms_metrics_history	`Failure`	1.31	test_metrics_api.py

Copilot

Copilot wasn't able to review any files in this pull request.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

weizhouapache · 2026-07-13T12:11:33Z

without the change

root@r-269-VM:~# cat /etc/frr/frr.conf 
frr version 6.0
frr defaults traditional
hostname r-269-VM
service integrated-vtysh-config
ip nht resolve-via-default
router bgp 10010
 bgp router-id 10.200.0.12
 bgp default ipv6-unicast
 neighbor 10.200.0.1 remote-as 64999
 neighbor 10.200.0.1 password password3
 neighbor fc00:2024:9:7::1 remote-as 64999
 neighbor fc00:2024:9:7::1 password password3
 address-family ipv4 unicast
  network 192.168.120.0/28
 exit-address-family
 address-family ipv6 unicast
  network fd00:2026:2:19::/64
 exit-address-family
line vty
root@r-269-VM:~#

with the change

root@r-269-VM:~# cat /etc/frr/frr.conf 
frr defaults traditional
hostname r-269-VM
service integrated-vtysh-config
ip nht resolve-via-default
ip prefix-list all-v4 seq 1 permit any
ip prefix-list default-v4 seq 1 permit 0.0.0.0/0
ipv6 prefix-list all-v6 seq 1 permit any
ipv6 prefix-list default-v6 seq 1 permit ::/0
ip prefix-list local-v4 seq 1 permit 192.168.120.0/28
ipv6 prefix-list local-v6 seq 1 permit fd00:2026:2:19::/64
router bgp 10010
 bgp router-id 10.200.0.12
 bgp default ipv6-unicast
 neighbor 10.200.0.1 remote-as 64999
 neighbor {} route-map upstream-v4-in in
 neighbor {} route-map upstream-v4-out out
 neighbor 10.200.0.1 password password3
 neighbor fc00:2024:9:7::1 remote-as 64999
 neighbor {} route-map upstream-v6-in in
 neighbor {} route-map upstream-v6-out out
 neighbor fc00:2024:9:7::1 password password3
 address-family ipv4 unicast
  network 192.168.120.0/28
 exit-address-family
 address-family ipv6 unicast
  network fd00:2026:2:19::/64
 exit-address-family
route-map upstream-v4-in permit 10
  match ip address prefix-list default-v4
route-map upstream-v4-in deny 1000
  match ip address prefix-list all-v4
route-map upstream-v4-out permit 10
  match ip address prefix-list local-v4
route-map upstream-v4-out deny 1000
  match ip address prefix-list all-v4
route-map upstream-v6-in permit 10
  match ipv6 address prefix-list default-v6
route-map upstream-v6-in deny 1000
  match ipv6 address prefix-list all-v6
route-map upstream-v6-out permit 10
  match ipv6 address prefix-list local-v6
route-map upstream-v6-out deny 1000
  match ipv6 address prefix-list all-v6
line vty

weizhouapache · 2026-07-13T12:13:26Z

@wido
the configuration looks ok,
however. the VMs cannot reach the internet, even the default egress policy is Allow.
I need more time to figure out what causes the issue.
Would you mind moving this to next release (4.24.0) ?

It is best practice, and mandatory in newer version of FRR, that route-maps should be applied to BGP peers. This is to prevent that mistakes can propogate through a network and cause outages. This change changes the route-maps where the VR will only accept IPv4 and IPv4 default gateways (0.0.0.0/0 and ::/0) to be sent by the upstream router to the VR. The other way around this change makes sure that FRR will not allow announcing anything else than the locally defined subnets to the upstream BGP router.

wido · 2026-07-14T19:53:04Z

@wido the configuration looks ok, however. the VMs cannot reach the internet, even the default egress policy is Allow. I need more time to figure out what causes the issue. Would you mind moving this to next release (4.24.0) ?

Thanks for the test! Lets re-run the test and we can push this one for later.

In which env did you test it?

wido added component:virtual-router component:networking labels Nov 22, 2024

wido requested a review from weizhouapache November 22, 2024 15:53

boring-cyborg Bot added the Python Warning... Python code Ahead! label Nov 22, 2024

alexandremattioli approved these changes Dec 3, 2024

View reviewed changes

weizhouapache approved these changes Dec 3, 2024

View reviewed changes

weizhouapache self-assigned this Dec 3, 2024

weizhouapache added the status:needs-testing label Dec 3, 2024

sureshanaparti added this to the 4.21.0 milestone Jun 5, 2025

sureshanaparti added this to Apache CloudStack 4.21.0 Jun 5, 2025

sureshanaparti moved this to In Progress in Apache CloudStack 4.21.0 Jul 14, 2025

weizhouapache modified the milestones: 4.21.0, 4.22.0 Aug 29, 2025

sureshanaparti removed this from Apache CloudStack 4.21.0 Aug 29, 2025

sureshanaparti added this to Apache CloudStack 4.22.0 Aug 29, 2025

sureshanaparti moved this to In Progress in Apache CloudStack 4.22.0 Aug 29, 2025

harikrishna-patnala modified the milestones: 4.22.0, 4.22.1 Nov 12, 2025

harikrishna-patnala removed this from Apache CloudStack 4.22.0 Nov 12, 2025

boring-cyborg Bot added component:api component:build component:integration labels Jan 13, 2026

wido changed the base branch from 4.22 to main January 13, 2026 11:57

DaanHoogland removed this from Apache CloudStack 4.22.1 Jan 13, 2026

DaanHoogland modified the milestones: 4.22.1, 4.23.0 Jan 13, 2026

weizhouapache modified the milestones: 4.23.0, 4.24.0 Jun 27, 2026

weizhouapache modified the milestones: 4.24.0, 4.23.0 Jul 1, 2026

weizhouapache requested a review from Copilot July 8, 2026 09:29

Copilot AI reviewed Jul 8, 2026

View reviewed changes

weizhouapache self-requested a review July 13, 2026 12:13

weizhouapache modified the milestones: 4.23.0, 4.24.0 Jul 14, 2026

wido force-pushed the frr-conf-virtual-router branch from 70b6cb4 to d2f1f04 Compare July 14, 2026 19:52

Uh oh!

Conversation

wido commented Nov 22, 2024

Uh oh!

codecov Bot commented Nov 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

weizhouapache commented Nov 24, 2024

Uh oh!

weizhouapache commented Nov 24, 2024

Uh oh!

blueorangutan commented Nov 24, 2024

Uh oh!

blueorangutan commented Nov 24, 2024

Uh oh!

weizhouapache commented Nov 25, 2024

Uh oh!

blueorangutan commented Nov 25, 2024

Uh oh!

blueorangutan commented Nov 25, 2024

Uh oh!

weizhouapache commented Nov 30, 2024

Uh oh!

blueorangutan commented Nov 30, 2024

Uh oh!

blueorangutan commented Nov 30, 2024

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

wido commented Jan 13, 2026

Uh oh!

wido commented Jul 1, 2026

Uh oh!

weizhouapache commented Jul 1, 2026

Uh oh!

weizhouapache commented Jul 1, 2026

Uh oh!

blueorangutan commented Jul 1, 2026

Uh oh!

blueorangutan commented Jul 1, 2026

Uh oh!

weizhouapache commented Jul 1, 2026

Uh oh!

blueorangutan commented Jul 1, 2026

Uh oh!

blueorangutan commented Jul 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

weizhouapache commented Jul 13, 2026

Uh oh!

weizhouapache commented Jul 13, 2026

Uh oh!

wido commented Jul 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

codecov Bot commented Nov 22, 2024 •

edited

Loading