In a DNS Bind

Recently a friend, who also works in IT, asked me, “What DNS server do you guys run?” I had to think for a second, because I wondered where to begin. It’s not a quick, easy answer.

Note that I use the term “domain” and “zone” more-or-less interchangeably in this post.

Internal

We run Windows Server 2012r2 DNS on our domain controllers, as many organizations with Active Directory infrastructure do: DNS is critical to its operation. We also make extensive use of dynamic DNS with DHCP integration. We don’t use external caching servers or anything like that for our internal network. Every workstation and IoT device is pointed at two of our domain controllers. You can get away with that when you work at a company with less than 5,000 devices. We did a Cisco Umbrella (formerly OpenDNS) pilot recently, but besides their on-premises server that augments the DNS a bit, it’s more-or-less Microsoft AD DNS as the star player.

“Semi-Private”

We have business partners that have trust relationships with our Active Directory. However, we don’t give them direct access to our internal Active Directory servers. They configure their DNS to perform “Conditional Forwarders” to us. What they actually hit are two Unbound DNS servers running on Windows Server 2012r2. Unbound is very flexible caching name server software. We use it because we want them to see “parts” of our internal network, but also “parts”of our public DNS... basically we use Unbound to override parts of our public space with private resolution, then whatever we don’t override gets sent on to our public DNS servers. We also do extensive NAT’ing, so we need to override the internal IP addresses as well.

Public

We used to use the free GoDaddy DNS service that came with domain ownership purchased through them. However, after the DDOS attacks on Dyn on October 16, 2016, we decided that we should diversify DNS providers. In addition, we had just purchased a new /24 IPv4 range as well as wanting to take control of the reverse lookups for the two /27 subnets we lease from one of our ISPs. We upgraded to GoDaddy’s Premium DNS and employed DNS Made Easy as our second provider. While they are providing secondary services for our main three zones, they are providing primary services for the PTR records for our two /27 subnets, our /24, as well as our IPv6 subnet’s ip6.arpa zone.

Notify Me!

We quickly discovered that there were some problems with this arrangement... namely in the way that our providers handle NOTIFY messages. In case you’re not aware, NOTIFY messages are an important component of keeping DNS servers in sync. If I change a record on my primary DNS server and NOTIFY is not functioning properly, it will take up to the refresh interval specified in the zone’s SOA record to replicate to the secondary. Actually, without NOTIFY, it’s the responsibility of the secondary server to “reach out” to the primary when it’s refresh interval is reached. If NOTIFY is working properly, the primary server reaches out to its secondaries to tap them on the shoulder and say, “Hey, I have some new information, ask me what it is,” at which point the secondary will issue an IXFR or AXFR transfer request.

Problems

The problem we found is that GoDaddy will only send NOTIFY messages to DNS servers defined as NS records for a particular domain. However, DNS Made Easy will not listen for notify messages on its authoritative servers--they have their own set of servers that listen for NOTIFY messages. GoDaddy offers no way to add those servers to an “also-notify” list. I sent trouble tickets to both companies, and (eventually) they both agreed that it was a problem. They also both said that this was a low priority thing as nobody else besides me seemed to care, and basically said they’ll get to it some day (insha’Allah, as our Arab friends would say).

BIND to the Rescue

ISC’s BIND software is insanely flexible. We ended up setting up a virtual private server with Linode whose $5/month role was primarily to act as a broker between two grumpy DNS providers who didn’t want to share nicely. It’s like the referee at the beginning of the boxing match that forces the boxers to shake hands. The BIND set up is pretty simple: in our options section, we have defined a max-refresh-time 120 that tells it to check the serial number on the primary server every two minutes. Then in our relevant configurations, we have also-notify { dnsmadeeasy-slaves; }; with the dnsmadeeasy-slaves defined in the masters section. In addition, we have notify-explicit; which ensures that the BIND server notifies the secondary (slave) servers upon discovery of a change. GoDaddy has a place to define an access-list that allows the BIND server to initiate a zone transfer. The BIND server also allows the DNS Made Easy servers to transfer the zones from BIND.

Final Answer

To answer my friend’s original question (which at this point he probably wished he hadn’t asked), I guess we’re running five different interfaces (Windows, Unbound, GoDaddy, DNS Made Easy, and BIND). I’m not sure which DNS software the providers are running, since the cute little BIND trick has been overridden (try nslookup -type=txt -class=chaos version.bind. 208.80.126.2). It’s amazing how much more complicated something can turn out to be beyond what you had originally intended.

The Point

Besides simply explaining what our crazy DNS set up is, my point is that if you know the technology well enough, you can leverage the right tool to get the job done. If you have to learn a new configuration file format, that’s not a big deal if you have an idea of what’s supposed to happen. One of the things I also tell my colleagues is that the software is supposed to serve you, you are not supposed to serve the software.