Skip to main content

You are here

2009.1 January - Global Distributed Denial of Service Attack Causing Major Problems with Internet

Submitted by Admin on Fri, 30/01/2009 - 22:30

As a hosting provider, K-IT.ca Inc. has been experiencing slowed web page connections since the middle of January 2009, and the problems seemed to be escalating. When Bell Canada's services were disrupted by a power failure on January 27, we moved our sites to a backup location until the service could be restored. We found, to our disappointment, that the page requests being made to the backup servers were being responded to almost instantly by the web servers, but the name resolution for the web pages was showing a delay of up to 2 minutes before the request reached the web servers. When we looked at our name servers, we discovered that we were being victimized by a DDOS attack.

We restored our primary servers the following day, and the DDOS attack increased in intensity, to the point that our servers became inaccessible by Friday evening. We have now restored service and have taken measures to block this DDOS.

This DDOS is insidious, and attacks the infrastructure of the internet. Nameservers form the backbone for all internet servers, be they email, ftp, web servers etc. Without nameservers to look up the address of the resource you are requesting, nothing will work! When you configure your account with your Internet Service Provider, one of the first things you do, either manually, or automatically if you use an installation disk, is to tell your computer what nameservers the ISP provides for your use. Every request you make for a web page, or to send email, first goes through these nameservers.

How Nameservers Work

When you request a web page (eg http://k-it.ca/content/newsletters) through your browser, that request must be routed over the internet to the correct IP address. Your browser first sends the domain (k-it.ca) to the name server. The name server then makes a request to one of the ROOT nameservers around the world (there are many!) for the IP of the name server responsible for the .ca top level domain. In this case they will be directed to the nameservers at the Canadian Internet Registration Agency. The ISP nameserver then asks the .ca authority for the IP of the nameserver responsible for the k-it.ca domain, and CIRA will direct the ISP to our nameservers. Your ISP will then query our nameserver for the IP of the web server, and that IP is returned to your browser. Your browser then sends the full page request directly to the web server, and the web server returns the page directly to your browser.

The strength of the global nameserver network is that the nameservers are free to talk to each other, and requests can be distributed to even out the load. All this talking back and forth happens very quickly, and the majority of the delay in web page service is typically due to data volume from the web server.

How this DDOS is Disrupting Service

When a nameserver is set up, it is provided with a list of ROOT nameservers, and its program will occasionally update this list. The list is massive, but stable, and does not have to be updated often. If a nameserver loses its list of ROOT servers, it can request the list from another nameserver, which triggers both nameservers to update their list. The one receiving the request will use its own list to request a fresh list, and it will then send the fresh list to the server which made the original request. This very polite system has kept the nameserver network functional for a long time. These requests are usually only made on nameserver startup.

Another polite feature of the nameserver network is the fact that a nameserver under load can ask another nameserver to do the lookup, and the results can then be returned to the client machine by the overloaded server more quickly.

This DDOS attack is sending requests out to nameservers, requesting them to retrieve ROOT server lists from other nameservers, and exploits both the above polite features. The latency on a single ROOT server request can be as long as five seconds, and when your nameservers are receiving over forty requests a minute, it is understandable that legitimate requests are being delayed, and even dropped, by the nameservers. They are brought to their knees by the attack.

What K-IT.ca Has Done

The short term solution is to block the offending nameservers at the firewall, but this breaks the way the internet is supposed to work. If one of these nameservers receives a request for one of our hosted domains, or if we receive a request for one of theirs, the nameservers must be able to communicate with each other.

The ROOT server requests, if legitimate, should also be responded to, and so we cannot simply block that type of request.

The solution we have implemented allows a remote nameserver to make a ROOT server request, but we log the request. If another such request comes from the same source within 3 days, the server is blacklisted for three days. This way we hope to maintain the network, with only minor interruptions of communications.

If your nameserver logs are showing multiple repeated requests for   query: . IN NS + ,   and you are interested in our corrective measures, contact K-it.ca Inc. and we will be pleased to help.

Link to this page

To refer others to this page copy the following link code and paste it into your page, blog, text or email.

Premium Drupal Themes by Adaptivethemes