DNS Flag Day: What It Is and How It Affects You
Reading time: 8 minutesDNS Flag day is almost here. That's why we decided to write about this important event, what EDNS is, the current problem with old broken DNS servers, and finally, we ran a massive EDNS test revealing that around 3.7 % of the top 100k websites ranked in Alexa will be down soon.
Like we've said before: the DNS system isn't as fast and secure as it should be to follow the Internet we're using today.
It was created decades ago, and has even been patched several times and improved in some ways to carry on with the new digital requirements of software developers, servers, ISPs, and new tech. Still, we find it lacking.
Speed and performance aren't the only disadvantages of the current DNS system. It's also a difficult platform for introducing innovative changes and features.
- What is EDNS?
- What's wrong with EDNS then?
- What's DNS Flag Day?
- Will my site go down?
- Running a massive EDNS test
- Conclusion
What is EDNS?
The DNS system is old, pretty old in fact. Its main technical specifications were defined in November 1983, and are described and documented in RFC 882.
More than 35 years have passed since then. The software world changed, and networks and the entire Internet are vastly different now. Our needs as software developers have changed as well.
The DNS needed a way to improve its technology by adding new features. But that was difficult because the DNS system only had space for 7 × 1-bit flags in its current UDP packet, and there was no easy space to add extra space.
That's why in 1999, the EDNS (also known as EDNS0) was introduced and documented in RFC 2671. EDNS is a mechanism that allows support for new DNS options, new response codes, and bypasses several size limits, among other things.
New implementations of the protocol were launched in 2013 and documented under a new RFC 6891 (which obsoletes RFC 2671). Thanks to EDNS, DNS servers were now able to communicate with other EDNS-based servers, that allowed bypassing the 512-byte package limit.
Looking for a faster and better way to handle then-current DNS usage, and at the same time seeking enough space to expand DNS growth in the future, software engineers wanted an alternative solution. That was when EDNS was created.
The idea worked pretty well. It was a DNS change that allowed better expansion, and at the same time was compatible with older DNS Servers.
What's wrong with EDNS then?
There is nothing wrong with EDNS essentially — this extension has allowed the DNS system to grow over the years.
However, some implementations and patches built by DNS providers were done with the main goal of keeping old DNS servers working.
As decades passed, quick "fixes" were implemented for broken servers, and the result hasn't proved as successful as expected. Many years later, these legacy DNS servers are not responding properly to today's EDNS requests.
This means that when a new EDNS request is made against these broken DNS authoritative servers, sometimes the DNS server won't answer the queries, and other times it will time out at the end, causing unnecessary delay in DNS communication.
Another issue found was the lack of standardized (non-FORMERR) error codes. Many of these patched servers respond using different error response codes, which makes the debugging task even more difficult for developers and system and network administrators.
Finally, there are misconfigured firewalls using strict rules that actually block valid DNS traffic, causing even more timeouts and delays.
What's DNS Flag Day?
In order to solve this timeout and DNS delay problem, the most popular DNS server software developers and several public DNS providers will soon be releasing a software update that will fix this, but at the same time, it has the potential to make a lot of sites go down.
On February 1, 2019, BIND, Knot, PowerDNS and Unbound DNS servers will remove support for old DNS patches, while at the same time implementing stricter EDNS handling.
In particular, they will remove the support for these poorly implemented workarounds in BIND 9.13.3 (development) and BIND 9.14.0 (production), Knot Resolver in all their stable versions, as well as the latest Unbound versions and PowerDNS Recursor 4.2.0.
This will only affect domain names that are using servers not following the standards.
Upon resuming February 1st, 2019, the DNS timeout will be interpreted as a server or network problem, meaning EDNS queries not answered within the timeout limit will be declared as not working/dead.
Will my site go down?
There are several ways to know if your site will be affected by this change.
You can use the ISC DNS Compliance Tester, which can easily tell you whether or not your DNS authoritative servers are following the standards.
We ran our own test against the DNS servers used here at SecurityTrails.com and we passed the test, as you see here:
Another great EDNS Compliance tool is made by CZ.NIC and follows the same principle: it will analyze and show you if your site will go down or not.
If you don't like GUI's and need some old-fashioned terminal action, you can also test this by yourself using the all-powerful dig command.
This is a simple terminal-based implementation of the same test ISC runs in their web-based interface.
In order to be EDNS compliant you must run a few DNS queries against your DNS authoritative servers. These range from basic tests to advanced ones:
- dig +norec +noedns soa zone @server
- dig +norec +edns=0 soa zone @server
- dig +norec +edns=100 +noednsneg soa zone @server
- dig +norec +ednsopt=100 soa zone @server
- dig +norec +ednsflags=0x80 soa zone @server
- dig +norec +dnssec soa zone @server
- dig +norec +dnssec +bufsize=512 +ignore dnskey zone @server
- dig +norec +edns=100 +noednsneg +ednsopt=100 soa zone @server
We ran some basic tests with our own DNS servers:
dig +norec +edns=0 soa securitytrails.com @ns07.domaincontrol.com
The DNS reply from our authoritative server follows the standards showing OPT PSEUDOSECTION with ENDS version 0, along with the NOERROR status in the dig response:
[research@securitytrails.com ~]$ dig +norec +edns=0 soa securitytrails.com @ns07.domaincontrol.com
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-10.P2.fc29 <<>> +norec +edns=0 soa securitytrails.com @ns07.domaincontrol.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64126
;; flags: qr aa; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 1
;; OPT PSEUDOSECTION
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;securitytrails.com. IN SOA
;; ANSWER SECTION:
securitytrails.com. 600 IN SOA ns07.domaincontrol.com. dns.jomax.net. 2018110903 28800 7200 604800 600
We tested another site, dnsowl.com, and no OPT record with version set to 0 wasn't found:
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-10.P2.fc29 <<>> +norec +edns=0 soa dnsowl.com @ns1.dnsowl.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51733
;; flags: qr aa; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 9
;; QUESTION SECTION:
;dnsowl.com. IN SOA
;; ANSWER SECTION:
dnsowl.com. 172800 IN SOA ns1.dnsowl.com. hostmaster.dnsowl.com. 1548076415 7200 1800 1209600 600
The same EDNS issue can be confirmed with full details in the ISC EDNS compliance test https://ednscomp.isc.org/ednscomp/fa33bd90a9
My site is going to be affected. What can I do?
If your site is affected there is only one way to fix it:
- Use a good DNS server or public resolver that follows the standards — Cloudflare is a good choice.
- If you are running your own DNS server, upgrade to the latest version as soon as possible.
- Tweak your firewall: some firewalls are configured to drop DNS packets that use EDNS extensions. This type of configuration should be avoided, as it creates higher latency for DNS connections. Make sure to remove any filtering for EDNS extensions.
Make the changes on your server, test again, and your site should pass the tests.
Running a massive EDNS test
Here at SecurityTrails, we hold DNS data from up to 805 million domain names. So it's easy for us to fetch domain lists.
However, as DNS Flag Day is almost here, it's almost impossible to test all our domain names in such a short time. That's why we isolated the top 100k Alexa domain names from our API and then ran the tests.
Our security researchers created a basic bash script to run a massive EDNS test, allowing us to explore all the current domain names within the top 100k domain names from Alexa's rankings:
#!/bin/bash
for i in $(cat dump.csv); do
ns=$(dig NS $i | grep NS | egrep -v 'flag|ANSWER|>' | awk '{print $5}' | sed '/^$/d' | head -n 1 | sed 's/\.$//')
if ! dig +norec +edns=0 soa $i @$ns | grep "EDNS: version: 0" -q && dig +norec +edns=0 soa $i @$ns | grep -q "NOERROR"; then
echo "$i minor problem detected"
elif dig +norec +edns=0 soa $i @$ns | grep "EDNS: version: 0" -q && dig +norec +edns=0 soa $i @$ns | grep -q "NOERROR"; then
echo "$i not affected"
else
echo "$i affected"
fi
done
Important: While there are more advanced ways to test full EDNS compliance, this script had the goal of providing us three basic results—domains with minor issues that will keep working after February 1st, affected domains that will stop working, and unaffected domains.
We parsed the results, and this is what we found:
- 3785 domains will be down after Feb. 1st, 2019.
- 5064 domains will have minor DNS issues but will keep working Feb. 1st, 2019.
- 91151 domain names will be unaffected.
Browsing the affected websites, we discovered a couple of sites such as a bank in Argentina (bancoprovincia.com.ar), Brazilian universities like ufc.br, the VLC media player main website videolan.org, the popular network and cell phone provider Verizon with their domain name verizonwireless.com and some government websites like dl.gov.cn.
If you want to explore the results by yourself, download the list of domain names here:
Conclusion
At this time there are only a few days left before DNS Flag Day's final change. Fortunately, you now know a way to test if your DNS servers are following the standards or not.
Here at SecurityTrails, we follow not only the best DNS standards but also the best cybersecurity practices to keep you up to date when it comes to intel gathering and data intelligence.
We're always scanning servers and IP ranges, analyzing DNS records, and much more, in order to give you access to the most accurate and complete DNS data on the Internet.
Book a SurfaceBrowser demo with our account team, or sign up for a free API account so you can keep your DNS servers, IPs and domain names safe at all times.
