How to scale requests to a single IP (DC)

How to scale requests to a IP at a Data Center

Scaling
scaling

Scaling app servers to nearly unlimited size is easy to explain but really hard in practice. It basically amounts to this:

  1. Balance requests using DNS anycast so you can spread load before it hits your servers
  2. Setup “Head End” machines with as large pipes as possible (40Gbps?) and load balance at the lowest layer you can. Balance at IP level using IPVS and direct server return. A single reasonable machine can handle a 40Gbps pipe….
  3. Setup a ton of HTTP-proxy type load balancers. This includes Nginx, Varnish, Haproxy etc… One of these machines can probably handle 1-5 Gbps of traffic so expect 20 or so behind each layer 3 balancer…
  4. Now for your app servers. Depending on if you’re using a dog slow language or not, you’ll want between 3 and 300 app servers behind each HTTP proxy.

Information source: https://news.ycombinator.com/item?id=13357278

 Share!