Fast and Reliable DNS Setup

Spark

Verified User
Joined
Aug 25, 2006
Messages
107
Okay I have read quite a few posts around here about proper DNS configuration and BIND and many of the answers did not sit well with me. So I decided to write this howto for you.


First off we are going to setup resolution using OpenDNS, a very fast and probably the largest DNS cache available which will provide significant speed increases to your DNS queries.

1) Go sign up for an OpenDNS account, this is very important because you need to edit OpenDNS configuration and turn off all their features such as typo correction which can cause problems for your dns server. ( you can disable these features for your IP blocks ).


2) Open up /etc/named.conf

and add these lines within the options block:
recursion no;
fetch-glue no;
forwarders { 208.67.222.222; 208.67.220.220; put.your.datacenter.dns.server.ip.here; };

Then restart named.

Because these are going to be public facing nameservers we want recursion turned off and since we are not going to be doing recursion there is no need to fetch the glue either :)

Now some of you would say oh but we can add an acl to allow for local recursion only, I have seen your examples there is no need for acl's to accomplish that allow-recursion {ipaddresses}; within your options block would work. THIS IS HIGHLY RECOMMENDED AGAINST!

ICS and various other parties have published several documents on why it is not a good idea to have a DNS server performing both authoritative and recursive lookups regadless if recursion is only done for the local network. I am not here to debate this with you if you dont get it then you should probably listen anyways because your thoughts are scewed.



3) last step edit /etc/resolv.conf

make it look like this:
nameserver 127.0.0.1
nameserver 208.67.222.222
nameserver 208.67.220.220
nameserver put.your.datacenter.dns.server.ip.here

the idea here is we first query the local dns server to see if we are hosting the domain if so return the result, if not the forwarders inside the named conf will take over. The only reason we add the other 3 nameservers to this resolv.conf file is in case your local dns server is stopped or errors out, this way we know we always have a proper route to domain resolution.



Thats it, enjoy a more responsive system, many speed improvements such as logging , updates, hostname lookups, spam detection will now improve plus it will also help reduce system load.
 
Last edited:
QuantumNet,

You make some good points, but I believe you miss a few relevant issues.
If you really want to use OpenDNS isntead of your upstream's nameservers, then you should use DNS the way it was designed, and use your local resolv.conf file to direct queries, and not forwarders.

Craig Hunt, author of Linux DNS Server Administration published by Sybex, points out:
The purpose of creating a forwarders list is twofold. First, because the forwarders handle so many queries, they develop a rich cache of answers. Many queries are rapidly answered from the forwarders' cache once this rich cache develops.

Second, using forwarders can reduce traffic over the wide area network. The forwarders are generally located within the organization's private network or local area network.
Craig's point about the rich cache is well-taken, but if you don't mind the time spent in actually communicating with OpenDNS servers, you'd probably be better served by putting the code into your resolv.conf file; where no time would be wasted first querying your own server at all. His second point of course is simply untrue in your scenario of using forwarders to a distant server on the Internet.

I've made some tests. From my office it takes over twice as long to get a response from OpenDNS's nameserver than from my connectivity provider's nameserver:
Code:
[jlasman@of1 ~]$ ping 206.13.29.12 # my upstream nameserver
PING 206.13.29.12 (206.13.29.12) 56(84) bytes of data.
64 bytes from 206.13.29.12: icmp_seq=1 ttl=249 time=13.7 ms
64 bytes from 206.13.29.12: icmp_seq=2 ttl=249 time=13.2 ms

--- 206.13.29.12 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 13.211/13.467/13.723/0.256 ms
[B][jlasman@of1 ~]$ping 208.67.222.222 # OpenDNS nameserver
PING 208.67.222.222 (208.67.222.222) 56(84) bytes of data.
64 bytes from 208.67.222.222: icmp_seq=1 ttl=52 time=28.7 ms
64 bytes from 208.67.222.222: icmp_seq=2 ttl=52 time=28.5 ms

--- 208.67.222.222 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 28.525/28.623/28.721/0.098 ms
[jlasman@of1 ~]$
At my data center the difference is orders of magnitude:
Code:
[jlasman@da12 ~]$ ping 67.30.130.157 # my upstream nameserver
PING 67.30.130.157 (67.30.130.157) 56(84) bytes of data.
64 bytes from 67.30.130.157: icmp_seq=0 ttl=63 time=0.411 ms
64 bytes from 67.30.130.157: icmp_seq=1 ttl=63 time=0.434 ms

--- 67.30.130.157 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.411/0.422/0.434/0.023 ms, pipe 2
[jlasman@da12 ~]$ ping 208.67.222.222 # OpenDNS nameserver
PING 208.67.222.222 (208.67.222.222) 56(84) bytes of data.
64 bytes from 208.67.222.222: icmp_seq=0 ttl=57 time=31.7 ms
64 bytes from 208.67.222.222: icmp_seq=1 ttl=57 time=29.3 ms

--- 208.67.222.222 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 29.379/30.572/31.765/1.193 ms, pipe 2
[jlasman@da12 ~]$
You wrote:
1) Go sign up for an OpenDNS account, this is very important because you need to edit OpenDNS configuration and turn off all their features such as typo correction which can cause problems for your dns server. ( you can disable these features for your IP blocks ).
Yes the default feature set can easily break your DNS in ways most of us would never imagine, mostly because OpenDNS was designed to be used by local networks for decisions as to what traffic to allow in; not where to send traffic. Yes you can change the default settings now, but you cannot guard against future changes to OpenDNS. Since you're advocating it's use for purposes never thought of, there's probably going to be a point in the future when they add another feature that completely breaks your system. For example, by default they automatically resolve all domains, whether they really resolve or not.

You also wrote:
2) Open up /etc/named.conf

and add these lines within the options block:
recursion no;
fetch-glue no;
forwarders { 208.67.222.222; 208.67.220.220; put.your.datacenter.dns.server.ip.here; };
Then restart named.

Because these are going to be public facing nameservers we want recursion turned off and since we are not going to be doing recursion there is no need to fetch the glue either :)
Why are you using forwarders here? Why not use the resolv.conf file for the purpose it was designed for?

According to Hunt (citation above):
The purpose of creating a forwarders list is twofold. First, because the forwarders handle so many queries, they develop a rich cache of answers. Many queries are rapidly answered from the forwarders' cache once this rich cache develops.

Second, using forwarders can reduce traffic over the wide area network. The forwarders are generally located within the organization's private network or local area network.
Craig's point about the rich cache is well-taken, but if you don't mind the time spent in actually communicating with a distant server or servers, you'd be better served by putting the code into your resolv.conf file, where no time would be wasted first querying your own server at all. His second point of course is simply untrue in your scenario of using forwarders to a distant server on the Internet.

You also wrote:
1) Go sign up for an OpenDNS account, this is very important because you need to edit OpenDNS configuration and turn off all their features such as typo correction which can cause problems for your dns server. ( you can disable these features for your IP blocks ).
Yes the default feature set can easily break your DNS in ways most of us would never imagine, mostly because OpenDNS was designed to be used by local networks for decisions as to what traffic to allow in; not where to send traffic. Yes you can change the default settings now, but you cannot guard against future changes to OpenDNS. Since you're advocating it's use for purposes never thought of, there's probably going to be a point in the future when they add another feature that completely breaks your system. For example, by default they automatically resolve all domains, whether they really resolve or not.
2) Open up /etc/named.conf

and add these lines within the options block:
recursion no;
fetch-glue no;
forwarders { 208.67.222.222; 208.67.220.220; put.your.datacenter.dns.server.ip.here; };
Then restart named.

Because these are going to be public facing nameservers we want recursion turned off and since we are not going to be doing recursion there is no need to fetch the glue either :)
Why are you using forwarders here? Why not use the resolv.conf file for the purpose it was designed for?

According to Craig Hunt (Linux DNS Server Administration, Sybex):
The purpose of creating a forwarders list is twofold. First, because the forwarders handle so many queries, they develop a rich cache of answers. Many queries are rapidly answered from the forwarders' cache once this rich cache develops.

Second, using forwarders can reduce traffic over the wide area network. The forwarders are generally located within the organization's private network or local area network.
Craig's point about the rich cache is well-taken, but if you don't mind the time spent in actually communicating with OpenDNS servers, you'd probably be better served by putting the code into your resolv.conf file; where no time would be wasted first querying your own server at all. His second point of course is simply untrue in your scenario of using forwarders to a distant server on the Internet.
Now some of you would say oh but we can add an acl to allow for local recursion only, I have seen your examples there is no need for acl's to accomplish that allow-recursion {ipaddresses}; within your options block would work. THIS IS HIGHLY RECOMMENDED AGAINST!
Mark Andrews of ISC, who's considered authoritative on BIND configuration, doesn't say to not use allow-recursion(look here), and while he also points out that there is a way to avoid cache-viewing by those not allowed to use recursion directly. In this post) he writes:
9.4.0 has allow-query cache. Prior to 9.4 you need to set allow-query on each zone if you don't want the cache visible and still leave the zones visible.
I agree with you that it's a bad idea for reasons you don't note:

The only reason you'd use allow-recursion would be if you were going to use your local nameserver for lookups for domains for which it lists itself as authoritative. This is NOT a good idea in a webhosting environment. Here's why:

You automatically set up authoritative DNS zones for every domain you host, but you really don't know, or control, whether or not your hosting client uses your nameserver(s). If your client is using his own, or some other nameserver, you really don't know if the one you've set up automatically actually reflects his domain or not. If you query your nameserver first, you have no idea if you're actually looking at his zone, or just looking at what your server is serving as his zone.

You also wrote:
The only reason we add the other 3 nameservers to this resolv.conf file is in case your local dns server is stopped or errors out, this way we know we always have a proper route to domain resolution.
For reasons explained above you'd be better off using your upstream's nameservers in your resolv.conf file, exactly as you do with your connection at home for your desktop system.

For those of us who don't have an on-network nameserver supplied by our upstream connectivity provider, I suppose OpenDNS is an option (after remembering to change the defaults). For the rest of us, it's probably not faster, and usually slower, than using our provider's recursive server for our lookups, using recursiion no in our named.conf file, and leaving localhost completely out of our resolv.conf file.

Jeff
 
QuantumNet,

You make some good points, but I believe you miss a few relevant issues.
If you really want to use OpenDNS isntead of your upstream's nameservers, then you should use DNS the way it was designed, and use your local resolv.conf file to direct queries, and not forwarders.

Craig Hunt, author of Linux DNS Server Administration published by Sybex, points out:

Craig's point about the rich cache is well-taken, but if you don't mind the time spent in actually communicating with OpenDNS servers, you'd probably be better served by putting the code into your resolv.conf file; where no time would be wasted first querying your own server at all. His second point of course is simply untrue in your scenario of using forwarders to a distant server on the Internet.

I've made some tests. From my office it takes over twice as long to get a response from OpenDNS's nameserver than from my connectivity provider's nameserver:
Code:
[jlasman@of1 ~]$ ping 206.13.29.12 # my upstream nameserver
PING 206.13.29.12 (206.13.29.12) 56(84) bytes of data.
64 bytes from 206.13.29.12: icmp_seq=1 ttl=249 time=13.7 ms
64 bytes from 206.13.29.12: icmp_seq=2 ttl=249 time=13.2 ms

--- 206.13.29.12 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 13.211/13.467/13.723/0.256 ms
[B][jlasman@of1 ~]$ping 208.67.222.222 # OpenDNS nameserver
PING 208.67.222.222 (208.67.222.222) 56(84) bytes of data.
64 bytes from 208.67.222.222: icmp_seq=1 ttl=52 time=28.7 ms
64 bytes from 208.67.222.222: icmp_seq=2 ttl=52 time=28.5 ms

--- 208.67.222.222 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 28.525/28.623/28.721/0.098 ms
[jlasman@of1 ~]$
At my data center the difference is orders of magnitude:
Code:
[jlasman@da12 ~]$ ping 67.30.130.157 # my upstream nameserver
PING 67.30.130.157 (67.30.130.157) 56(84) bytes of data.
64 bytes from 67.30.130.157: icmp_seq=0 ttl=63 time=0.411 ms
64 bytes from 67.30.130.157: icmp_seq=1 ttl=63 time=0.434 ms

--- 67.30.130.157 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.411/0.422/0.434/0.023 ms, pipe 2
[jlasman@da12 ~]$ ping 208.67.222.222 # OpenDNS nameserver
PING 208.67.222.222 (208.67.222.222) 56(84) bytes of data.
64 bytes from 208.67.222.222: icmp_seq=0 ttl=57 time=31.7 ms
64 bytes from 208.67.222.222: icmp_seq=1 ttl=57 time=29.3 ms

--- 208.67.222.222 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 29.379/30.572/31.765/1.193 ms, pipe 2
[jlasman@da12 ~]$

Actually a ping wont serve as a true time test for this operation, since OpenDNS is one of the largest caches available it does not have to look any firther then its own cache for most queries, where as your datacenters recursive name servers will spend alot more time looking at root servers and then authoritative servers in order to get its answer so you can triple the ping time for your local provider report since there are 3 sets of nameservers that need to be contacted.

Yes the default feature set can easily break your DNS in ways most of us would never imagine, mostly because OpenDNS was designed to be used by local networks for decisions as to what traffic to allow in; not where to send traffic. Yes you can change the default settings now, but you cannot guard against future changes to OpenDNS. Since you're advocating it's use for purposes never thought of, there's probably going to be a point in the future when they add another feature that completely breaks your system. For example, by default they automatically resolve all domains, whether they really resolve or not.

by default they do rewrite and resolv NXDOMAIN but that is why they provide an interface for you to control your subnets and turn all features off when you turn all features off they are off for all your subnets that you define. There used to be a problem with this and it wont be affected because unlike what you say about me "advocating it's use for things for purposes never thought of" SURBL worked with OpenDNS to resolve these issues for systems that emplyed SURBL to help fight spam. Now if OpenDNS and SURBL found it important to team up with eachother for this type of use than I dont get where you come off acting like this was my idea.


Mark Andrews of ISC, who's considered authoritative on BIND configuration, doesn't say to not use allow-recursion(look here), and while he also points out that there is a way to avoid cache-viewing by those not allowed to use recursion directly. In this post) he writes:

I agree with you that it's a bad idea for reasons you don't note:

Section 1.3 of this publication: http://www.isc.org/pubs/tn/isc-tn-2002-2.txt
BIND serves cached data non-authoritatively.

1.3. Security Considerations -- Although BIND can provide both functions
(authoritative and non-authoritative) in single instance, ISC recommends
always separating these two functions. In other words, you should run
one server for your authoritative zones, and another that caches non-
authoritative data. Reasons include:



You wrote:
The only reason you'd use allow-recursion would be if you were going to use your local nameserver for lookups for domains for which it lists itself as authoritative.
Which is what I was doing to improve speed for locally hosted sites.


This is NOT a good idea in a webhosting environment. Here's why:

You automatically set up authoritative DNS zones for every domain you host, but you really don't know, or control, whether or not your hosting client uses your nameserver(s). If your client is using his own, or some other nameserver, you really don't know if the one you've set up automatically actually reflects his domain or not. If you query your nameserver first, you have no idea if you're actually looking at his zone, or just looking at what your server is serving as his zone.

Care to elaborate on this one more and explain what could be added to the dns that the server would need to resolv in order to perform its functions as a web server? I know email can come into play here but thats why DA has a checkbox for externally hosted email servers.

You might have an IRC script or 2 that connects to a remote domain but that is the beutiful part of bind 9 customers or you can add NS records to the local zone... some of my customers did this when they switched over to DynDNS.

For reasons explained above you'd be better off using your upstream's nameservers in your resolv.conf file, exactly as you do with your connection at home for your desktop system.

I dont agree and Niether does SURBL or OpenDNS so ...

For those of us who don't have an on-network nameserver supplied by our upstream connectivity provider, I suppose OpenDNS is an option (after remembering to change the defaults). For the rest of us, it's probably not faster, and usually slower, than using our provider's recursive server for our lookups, using recursiion no in our named.conf file, and leaving localhost completely out of our resolv.conf file.

I would seriously consider doing extensive testing on your end before deciding this or not. We happen to do more than web hosting we do streaming and IRC related services where connection issues such as DNS latency are more noticeable than webhosting, after a weeks period of switching over to this solution we have yet to see any ping timeouts on the IRC networks hosted by us.



Just like you tell people with your SpamBlocker script Jeff "Monitor it for a week"



On another note I have been playing around with your spamblocker 3 and one of the other guys on these forums idea with SURBL at the MTA level. I should start another thread to discuss that though.
 
Actually a ping wont serve as a true time test for this operation, since OpenDNS is one of the largest caches available it does not have to look any firther then its own cache for most queries, where as your datacenters recursive name servers will spend alot more time looking at root servers and then authoritative servers in order to get its answer so you can triple the ping time for your local provider report since there are 3 sets of nameservers that need to be contacted.
When we used our own nameserver for cache resolution a few years ago we found that over 80% of our requests were coming from our cache. We believe that's true because the servers our servers need generally end up in our own cache; it's rich enough for us, though probably nowheres near as rich as the OpenDNS servers.

We believe that hasn't changed and for ourselves we won't be testing further. We believe that it's true in most webhosting environments.

We also don't believe in using a service we're not paying for in our commercial environment, since we know our providers need to at least cover expenses in order to stay in business to continue serving us.

We understand that the OpenDNS model is to get money from advertising for their hits for nonexistent domains, but since we have to turn that off we see ourselves as not paying for OpenDNS servers.

Nevertheless, I may decide to try it at some point in the future, on one of our servers, to see if there's a noticeable difference. Toward that end I've already set up an account.
by default they do rewrite and resolv NXDOMAIN
I and a lot of others on the bind-users forums see that as a bad thing.
but that is why they provide an interface for you to control your subnets and turn all features off when you turn all features off they are off for all your subnets that you define.
I wonder if they did that because of the bad press and pressure they were getting? Certainly it undermines their payment model. Nevertheless, that's a question for them and not for you.
There used to be a problem with this and it wont be affected because unlike what you say about me "advocating it's use for things for purposes never thought of" SURBL worked with OpenDNS to resolve these issues for systems that emplyed SURBL to help fight spam. Now if OpenDNS and SURBL found it important to team up with eachother for this type of use than I dont get where you come off acting like this was my idea.
I'm sorry if I gave you the impression it was your idea. I still believe in the people I quoted. I'm also going to find some more recent quotes, and see what Cricket Liu (co-author of the DNS and Bind book) says on the subject.
Care to elaborate on this one more and explain what could be added to the dns that the server would need to resolv in order to perform its functions as a web server? I know email can come into play here but thats why DA has a checkbox for externally hosted email servers.
The problem is that you have no control over and no knowledge of the changes made to DNS by the client; you have no idea what he's doing with his domain, and you can't tell from looking at your server. You could be trying to (for example) execute his curl script to what your server thinks is a nonexistent subdomain, simply because it's located on a different server. One of our bank clients does this; their backend is at their main branch, not on our servers, and they couldn't talk to it under your scenario because they didn't change DNS on our server and that's what we depend on.
I would seriously consider doing extensive testing on your end before deciding this or not. We happen to do more than web hosting we do streaming and IRC related services where connection issues such as DNS latency are more noticeable than webhosting, after a weeks period of switching over to this solution we have yet to see any ping timeouts on the IRC networks hosted by us.
Which doesn't change that you're depending on a foreign server on external networks, none of which are under your control, and all of which are subject to vagaries not under your control. I'm glad it works for you.
On another note I have been playing around with your spamblocker 3 and one of the other guys on these forums idea with SURBL at the MTA level. I should start another thread to discuss that though.
Please do; we're always open to changes that help all of us. Please realize that I wrote SpamBlocker for my own use and it "kind of got away from me" when John and Mark decided to use it in DirectAdmin.

You can of course (anyone can) use your own exim.conf file.

When I wrote SpamBlocker it was far advanced as compared to the DirectAdmin supplied file, which accepted all email, even undeliverable email, before checking it.

Lots of advances since then and I don't always have time to keep up with them. I look forward to help and ideas.

Jeff
 
Which doesn't change that you're depending on a foreign server on external networks, none of which are under your control, and all of which are subject to vagaries not under your control. I'm glad it works for you.

Most people that will be reading this howto do not have the luxury of being in business since the beginning of the internet like yourself and their businesses have not grown to the point where they are managing their own datacenter with their own redundant DNS cache servers. So for you to write based on your own personal setup that you actually have control of your own cache servers is beyond this document obvious people like you would not need to even read a howto like this.

Most people reading this document will be using their datacenters recursive name servers which they have no control over which in their case OpenDNS is a better solution. Also there are some datacenters that only provide recursive name servers on the local network and when DA is installed within a VPS that is not granted local network access then openDNS is the solution.

Whats to say that these datacenters recursive nameservers are not overloaded as it is by existing clients causing queries to slow down even more causing your system load to be just that much higher as the mail daemon is processing dns lookups.

When it comes to the forwarding nameservers and having 127.0.0.1 in the resolve file there is a toss up here you weigh speed loss for jeff's concerns but his concerns are only found in rare circumstances which a customer would clearly put in a support ticket asking for assistance.

So its up to the end user to decide because what works for Jeff does not always work for everyone. We had a problem we solved it with a solution and shared the results for others who might be in the same boat we were.


P.S.
your cache in your local nameservers expires with TTL so your recursive name servers have to look to the root servers then the authoritative, in regards to spam fighting with SURBL this is not the case because with 1000's of MTA's contacting OpenDNS for the same queries this cache pretty much stays upto date at all times.
 
We're probably going to try using an account at OpenDNS at some point, but we won't follow up on it unless it causes problems.

Jeff
 
I have tried running this (as described above) for the last couple of weeks and it has been painless :)

Everything works as it should and my spamassassin scan time per email has gone down by a couple of seconds (from 9 to 7) - all the DNSBLs and URIBLs I use are still functioning correctly.

Thumbs up from me!
 
How come I get
/etc/named.conf:15: option 'fetch-glue' is obsolete: 11 Time(s)

i did as exactly as the author said to do.
 
You may be using a version of BIND that no longer supports the fetch-glue option; it's not required for DirectAdmin i'm not familiar with it, but perhaps someone else is and will respond, or perhaps you can try a BIND support group for information as to how to replace it.

Jeff
 
Back
Top