Web Server Clustering

by Richard Yoo, Co-founder and CTO of Rackspace Managed Hosting

There was a time when computers were simple and applications were even simpler. However, with the advent of the desktop computer and massive communication networks such as the Internet, the computing landscape has changed significantly. As people began to rely on their e-mail and favorite Web sites, system architects began looking for ways to fortify their server platforms, both to increase performance and reduce downtime. Clustering achieves these goals by using multiple servers instead of a single box.

In the past, when system operators required more horsepower, they would add memory or additional CPU's to systems. Other methods included tweaking or rewriting software or updating disk subsystems for better performance. Often, you couldn't get any more performance out of the server, or the application had simply reached its limit.

Scalability and availability are challenges in any environment. They are particularly difficult in a Web environment, because traffic patterns tend to be erratic and generally volatile. Major sites such as E-bay, Yahoo!, and Amazon.com have realized that the success of their organizations relies heavily on their robust platforms. They have also learned that hardware and software aren't perfect. In the quest for resilience and performance, many have switched gears and turned to a "more is better" approach.

Typically in computing, the term "clustering" is used to describe a set of servers that is collectively trying to accomplish a single goal. Lately, computational clustering has become very popular, particularly through the use of Linux as applied in the Beowulf Project . However, in the case of Web serving, the goal would be to deliver Web content to a given audience instead of, lets say, cracking the human genome.

The term "load-balancing" refers to the way in which requests for Web content are distributed across the cluster, and occasionally, across multiple clusters. There are many different ways to achieve this distribution. Some of them involve laying out content across different servers using DNS tricks or highly sophisticated algorithms that decide which server is most ready for the next request.

One of the simplest ways you can spread load across a group of servers is to use what is known as "round-robin DNS". It can be used to distribute a load across a group of servers containing identical content. In round-robin DNS, several servers share the same Fully-Qualified-Domain-Name (or FQDN). The DNS server then rotates through each of the IP addresses for the servers as it processes queries for the common FQDN.

For example, let's say we have a web site with the URL http://www.example.com. We'd like the site to be served by a trio of servers with addresses 192.168.0.101, 192.168.0.102, and 192.168.0.103.The FQDN of "www.example.com" would resolve to 192.168.0.101 first, then 192.168.0.102, followed by 192.168.0.103 on the third request. On the fourth request it would loop back to 192.168.0.101 and then repeat the pattern. This processes results in requests being distributed across multiple servers with minimal effort.

One problem with this method is that there is no logic to determine which servers are available for requests, so servers that are overwhelmed or otherwise down will still get requests sent to them. On the other hand, round-robin DNS is very simple to implement, because it does not require any additional software or hardware.

Another way to distribute traffic across multiple servers involves placing "redirectors" in front of the cluster. The most basic of these redirectors would be a "reverse proxy." Basically, the reverse proxy looks at the incoming request and decides which content server to pull from. Typically, in such scenarios the content is not identical across all servers, so the administrator has the flexibility of placing more popular content on the beefier servers.

Depending on the actual implementation, the reverse proxy may have the ability to dynamically choose where to send requests. Its decision-making process may be influenced by what content is being requested or even by the availability of the servers in the back end. This, in turn, adds a bit of intelligence compared to the seemingly brain-dead round-robin DNS.

Of course, every improvement comes at a price, and in this case we've introduced a single point of failure. If the reverse proxy has any difficulties (crashes, becomes overwhelmed, etc.) the entire cluster is rendered useless.

There are lots of different implementations of reverse proxies from both open-source developers and commercial vendors. Cost and effectiveness will vary greatly.

There have been several vendors who have made load-balancing a major project. Software companies like TurboLinux have produced packages that can transform ordinary servers into devices than can perform highly intelligent load-balancing. To further simplify and fortify the load-balancing product idea, appliance vendors like Cisco and Nortel have produced specialized chassis-based solutions that bring a whole new level of sophistication and robustness to directing traffic.

With commercial companies battling for market share, every vendor has a different spin on how load-balancing should be done. The common theme, however, is to perform this traffic direction as intelligently as possible while minimizing downtime. Most, if not all, commercial load-balancers will work in pairs (or more) for fault-tolerance and operate at amazing speeds. It's commonplace to see graphical configuration interfaces to simplify management and monitoring. Basically, you get all the bells and whistles you'd get with any refined and bundled product.

Specialized load-balancing products also use sophisticated algorithms that help decide which servers in the cluster receive a given Web request. In addition to round-robin, popular methods include least-connections, hashing, and minimum misses. Typically, threshold and weightings can be applied to bias traffic distribution to accommodate a heterogeneous cluster of servers. Additionally, backup or overflow servers can also be designated to adjust the size of the cluster on the fly.

Commercial load-balancers are very sexy with their endless feature lists. They work well and the performance and uptime numbers can be staggering. Unfortunately, all these flashy features come at a cost and it's not unusual to see five-digit price tags attached to some of these products.

Clustering has become one of the key methods of handling huge amounts of Web traffic and making sure that content is deliverable all the time. Since demand is high, the load-balancing landscape is changing at a rapid rate. We're starting to see more applications assuming the roles of an external chassis or software package. Applications are becoming more resistant to failure and beginning to work together with other nodes to increase performance. However, if your current implementations are lacking cluster-aware features, one of these traffic-direction techniques may work for you.

Originally from Houston, Texas, Richard Yoo attended Trinity University as a pre-med student. From the age of 12, computers were Yoo's obsession. In 1996, he left Trinity, found a small office space, and with four modems and four computers, started an Internet service provider company, Cymitar Network Systems.

Unlike other dial-up ISPs, Cymitar moved rapidly to embrace e-commerce and business consulting before moving firmly in the direction Rackspace has taken today: managed Web hosting. Immediate and positive customer response to Yoo's initial foray into the dedicated server market with a small Web site and a simple configurator confirmed his hunch about the market niche and the growing business demand for dedicated servers. In 1998, Richard incorporated Rackspace with partners Dirk Elmendorf and Patrick Condon.

Today as Rackspace Managed Hosting chief technology officer, Yoo's responsibilities include evaluating and developing the company's technical direction, as well as making internal technology purchases.