[axelJacquet]

Fixes #2749
The PF_PACKET socket used to send gratuitous ARPs is send-only, but is created with htons(ETH_P_ARP), which registers a kernel receive hook: every incoming ARP packet of the host is handed to packet_rcv() for this socket, then dropped by the "receive nothing" BPF filter. With many keepalived instances in separate namespaces (OpenStack Neutron L3 HA routers), this becomes quadratic and saturates a CPU.

This creates the socket with protocol 0 ("If protocol is set to zero, no packets are received", packet(7)). Sending is unc
hanged: with a destination address, the kernel uses sll_protocol, which keepalived already sets.

Details, history and measurements are in #2749.

Tested on 2.2.7 (Debian 12, Linux 6.12):

• socket no longer registered (/proc/net/packet: protocol 0000, R 0);
• gratuitous ARPs still sent on transition to MASTER;
• 236 namespaces with keepalived, broadcast ARP load: CPU at 700 ARP/s 65.5 % → 6.1 %, no drops up to 5 000 ARP/s (drops from ~1 000 ARP/s before).

[pqarmitage]

@axelJacquet Many thanks for the detailed explanation and the patch.

Based on the information you have provided, I think there are several changes that are missing:

1. Having set protocol in the socket() call to 0, there is no need to call if_setsockopt_no_receive()

diff --git a/keepalived/vrrp/vrrp_arp.c b/keepalived/vrrp/vrrp_arp.c
index 96700ad9..6c4326b9 100644
--- a/keepalived/vrrp/vrrp_arp.c
+++ b/keepalived/vrrp/vrrp_arp.c
@@ -221,9 +221,6 @@ gratuitous_arp_init(void)
        if (__test_bit(LOG_DETAIL_BIT, &debug))
                log_message(LOG_INFO, "Registering gratuitous ARP shared channel");
 
-       /* We don't want to receive any data on this socket */
-       if_setsockopt_no_receive(&garp_fd);
-
        /* Initalize shared buffer */
        garp_buffer = PTR_CAST(char, MALLOC(GARP_BUFFER_SIZE));
 

2. The same changes should be applied for IPv6 ndisc:

diff --git a/keepalived/vrrp/vrrp_ndisc.c b/keepalived/vrrp/vrrp_ndisc.c
index 1e2453e3..001759d8 100644
--- a/keepalived/vrrp/vrrp_ndisc.c
+++ b/keepalived/vrrp/vrrp_ndisc.c
@@ -312,7 +312,7 @@ ndisc_init(void)
                return true;
 
        /* Create the socket descriptor */
-       ndisc_fd = socket(PF_PACKET, SOCK_RAW | SOCK_CLOEXEC | SOCK_NONBLOCK, htons(ETH_P_IPV6));
+       ndisc_fd = socket(PF_PACKET, SOCK_RAW | SOCK_CLOEXEC | SOCK_NONBLOCK, 0);
 
        if (ndisc_fd < 0) {
                log_message(LOG_INFO, "Error %d while registering gratuitous NDISC shared channel", errno);
@@ -322,9 +322,6 @@ ndisc_init(void)
        if (__test_bit(LOG_DETAIL_BIT, &debug))
                log_message(LOG_INFO, "Registering gratuitous NDISC shared channel");
 
-       /* We don't want to receive any data on this socket */
-       if_setsockopt_no_receive(&ndisc_fd);
-
        return true;
 }
 

3. vrrp_open_send_socket() , which also calls if_setsockopt_no_receive() presumably can also benefit from this change:

diff --git a/keepalived/vrrp/vrrp.c b/keepalived/vrrp/vrrp.c
index 967181ca..f1d07c63 100644
--- a/keepalived/vrrp/vrrp.c
+++ b/keepalived/vrrp/vrrp.c
@@ -2733,7 +2733,7 @@ chk_min_cfg(vrrp_t *vrrp)
 
 /* open a VRRP sending socket */
 static int
-open_vrrp_send_socket(sa_family_t family, int proto, const interface_t *ifp,
+open_vrrp_send_socket(sa_family_t family, const interface_t *ifp,
 #ifdef _HAVE_VRF_
                        const interface_t *vrf_ifp,
 #endif
@@ -2750,7 +2750,7 @@ open_vrrp_send_socket(sa_family_t family, int proto, const interface_t *ifp,
        }
 
        /* Create and init socket descriptor */
-       fd = socket(family, SOCK_RAW | SOCK_CLOEXEC | SOCK_NONBLOCK, proto);
+       fd = socket(family, SOCK_RAW | SOCK_CLOEXEC | SOCK_NONBLOCK, 0);
        if (fd < 0) {
                log_message(LOG_INFO, "cant open raw socket. errno=%d", errno);
                return -1;
@@ -2793,8 +2793,6 @@ open_vrrp_send_socket(sa_family_t family, int proto, const interface_t *ifp,
 
        if_setsockopt_priority(&fd, family);
 
-       if_setsockopt_no_receive(&fd);
-
        if (fd < 0)
                return -1;
 
@@ -2961,7 +2959,7 @@ open_sockpool_socket(sock_t *sock)
        if (sock->fd_in == -1)
                sock->fd_out = -1;
        else
-               sock->fd_out = open_vrrp_send_socket(sock->family, sock->proto, sock->ifp,
+               sock->fd_out = open_vrrp_send_socket(sock->family, sock->ifp,
 #ifdef _HAVE_VRF_
                                                     sock->vrf_ifp,
 #endif

4. Now there are no remaining calls to if_setsockopt_no_receive():

diff --git a/keepalived/include/vrrp_if.h b/keepalived/include/vrrp_if.h
index e946e83b..dc0555d6 100644
--- a/keepalived/include/vrrp_if.h
+++ b/keepalived/include/vrrp_if.h
@@ -268,7 +268,6 @@ extern int if_setsockopt_mcast_hops(sa_family_t, int *);
 extern int if_setsockopt_mcast_if(sa_family_t, int *, const interface_t *);
 extern int if_setsockopt_priority(int *, int);
 extern int if_setsockopt_rcvbuf(int *, int);
-extern int if_setsockopt_no_receive(int *);
 extern void interface_up(interface_t *);
 extern void interface_down(interface_t *);
 extern void cleanup_lost_interface(interface_t *);
diff --git a/keepalived/vrrp/vrrp_if.c b/keepalived/vrrp/vrrp_if.c
index 6a15dc69..f9950fcf 100644
--- a/keepalived/vrrp/vrrp_if.c
+++ b/keepalived/vrrp/vrrp_if.c
@@ -1338,28 +1338,6 @@ if_setsockopt_rcvbuf(int *sd, int val)
        return *sd;
 }
 
-int
-if_setsockopt_no_receive(int *sd)
-{
-       int ret;
-       struct sock_filter bpfcode[1] = {
-               BPF_STMT(BPF_RET | BPF_K, 0),   /* ret #0 - means that all packets will be filtered out */
-       };
-       struct sock_fprog bpf = { sizeof(bpfcode) / sizeof(bpfcode[0]), bpfcode };
-
-       if (*sd < 0)
-               return -1;
-
-       ret = setsockopt(*sd, SOL_SOCKET, SO_ATTACH_FILTER, &bpf, sizeof(bpf));
-       if (ret < 0) {
-               log_message(LOG_INFO, "Can't set SO_ATTACH_FILTER option. errno=%d (%m)", errno);
-               close(*sd);
-               *sd = -1;
-       }
-
-       return *sd;
-}
-
 void
 interface_up(interface_t *ifp)
 {

I suspect the calls to sendmsg() in vrrp_send_pkt() and ndisc_send_na() my need to be changed to sendto(), but that needs further investigation.

[axelJacquet]

Thanks for the review!

I've pushed a second commit: the NDISC socket is now also created with protocol 0, and if_setsockopt_no_receive() is no longer called on either packet socket.

I don't think the change can be applied to open_vrrp_send_socket(), though. It is an AF_INET/AF_INET6 raw socket, not a packet socket, and for those protocol 0 is rejected by the kernel (inet_create() / inet6_create() only match the SOCK_RAW wildcard entry for a non-zero protocol):

socket(PF_PACKET, SOCK_RAW, 0)                 -> OK
socket(AF_INET, SOCK_RAW, IPPROTO_VRRP)        -> OK
socket(AF_INET, SOCK_RAW, 0)                   -> FAILED: Protocol not supported (errno 93)
socket(AF_INET6, SOCK_RAW, IPPROTO_VRRP)       -> OK
socket(AF_INET6, SOCK_RAW, 0)                  -> FAILED: Protocol not supported (errno 93)

I built three variants of current master (e922f97) and ran two keepalived instances (priority 150 and 100, same VRID) in two namespaces connected by a veth pair, Linux 6.12:

Variant	States	VRRP adverts sent	GARP / NA sent	VIP on both	Packet sockets subscribed
master	MASTER / BACKUP	yes	yes	no	2 (with BPF filter)
this PR	MASTER / BACKUP	yes	yes	no	0 (no BPF filter, nothing queued)
this PR + open_vrrp_send_socket() with protocol 0	MASTER / MASTER	none (cant open raw socket. errno=93)	yes	yes	0

So I've kept if_setsockopt_no_receive() for the VRRP send socket. That socket only receives VRRP/AH packets, so the filter there is still useful and the cost is not comparable to the ARP case.

The sendmsg() / sendto() question can be looked at separately.

[pqarmitage]

@axelJacquet Many thanks for identifying this issue and providing the resolution.