Newer RHEL compatible OS with network-manager having startup errors because of tentative state on the network interface

With the latest OS versions, RHEL 8 and up, the OS is now running the DAD (Duplicate Address Detection) algorithm right after a network interface is enabled because all interfaces will get an IPv6 address. While this process is ongoing, the IP address is set to tentative state and many applications listening on an IP address for a service will fail to start as the IP is not ready and not available for the system and processes. The DAD algorithm can take a few seconds to finish and it has been conflicting with many production systems that can’t handle that delay.

Even though a different algorithm called Optimistic DAD has been implemented in the newest kernels, it has shown to continue exhibiting the same behavior for the applications that I’ve tested it with.

Symptoms:

– Network address on IPv6 is set to tentative state for a few seconds during boot or network restart, due to the Duplicate Address Detection mechanism

– The application fails to bind to the IPv6 address and enters safe/fail mode requiring a service restart.

Root cause:

This scenario triggers when the following conditions are present:

– the network interface has an IPv6 configured

– the network interface with IPv6 has no connected cable or takes long to verify the IP address.

– DAD is enabled for the interface with ipv6 address (default configuration)

– Application is configured to listen on all configured IPs and one specific static address.

There are many possible ways to prevent this failure, but all require manual configuration or human intervention to configure the OS or application.

OS workarounds:

  • Unset the IPv6 from the network interface
  • Disable DAD algorithm on the problematic interface or all interfaces
# For testing (change interface name accordingly, example: all|eth1|ens5f0|enp4s0f1)
sysctl -w net.ipv6.conf.eth2.accept_dad=0

# For persistent config after reboot 
echo "net.ipv6.conf.eth2.accept_dad=0" >> /etc/sysctl.conf
  • Add a pre exec wait for a few seconds on the system.d override config file for your application .
systemctl edit <your_app>.service

[Service] 
ExecStartPre=/bin/sleep 3

Application workaround:

Configure it to listen only on an IP address which is configured statically instead of trying to listen on all the IP addresses.

If you want to try optimistic DAD look for the paramater /proc/sys/net/ipv6/conf/all/optimistic_dad and /proc/sys/net/ipv6/conf/all/use_optimistic to be available as it needs to be supported by the kernel. Try changing these values with sysctl for testing.

Note: there is a setting for every interface and a default setting for all interfaces.

Leave a comment