rec: Further improvements for slow authoritative NS #18032

• Program: Recursor
• Issue type: Feature request

Short description

Due to NSSpeeds cache decay an authoritative nameserver for a domain that is really slow is still forwarded "live" DNS queries every now and then leading to undesired long(er) response times for such queries.

Usecase

There are quite some domains where not all authoritative servers are similarly fast. Recursor learns slow nameservers but still forwards a few queries to these every now and then. However we'd usually prefer "stable" fast resolution times, especially when people measure response times, look closely at outliers and try to do comparisons.

Description

This is a follow-up to #17492 where bugs in NSSpeeds cache decay have been fixed but there still seems to be room for improvement:

• "Marking auths as "second class" if they consistently score high roundtrip times and not selecting them at all for a longer period of time is a separate issue"
• Why does roundtrip time probing need to happen with "live" traffic, at least if we already know that some NS were slower than others? There are already background tasks for example for refetching of almost expired records, couldn't those be used to probe auth servers as well, for example when the NSSpeeds cache entry is "sufficiently decayed" instead of later slowing down a "live" query, at least for auths with a sufficiently high query rate?

Even though it's common nowadays that domains are hosted on nearby Anycast NS there are still a lot of cases where either not all Anycast instances are similarily close (and therefore fast), Anycast NS are combined with "regular" non-anycasted NS somewhere on this planet (e.g. domain "telstra.com.au" [+/- 300ms here], "funet.fi" [+/- 30ms here]) or full unicast setups where NS are also not necessarily similarily close.
While that might not be a big deal for responses with high TTLs we've unfortunately seen cases where CDNs with their usual CNAME chains and minimal TTLs were affected and queries to slower NS quickly ended up impacting measured end user DNS response times.