How can I create a "fail whale" for my website?

Eric Falsken asked:

I have a website that we load-balance across a few machines. The load-balancer (a Brocade ServerIron ADX) is on the local network. I know it has the capability to configure a “backup” ip address to use as the “real”, but it would need to be on a local network. How can I provide a “fail whale” to my users and a status update message when my entire environment may be offline due to a FW, LB, or multiple-server failure? Since DNS doesn’t have the ability to provide a weighted A-Record backup (like SRV records), what are my options?

Ideally, the solution should be able to step in automatically as soon as my site becomes unreachable and inform my users that our response teams on working on the problem.

My answer:

You get a web designer to produce a document which looks like you want it to look.

Then you configure the load balancer to serve this document for HTTP 500, 502, 503 and 504 errors. Exactly how you do that varies; check your documentation.

Some load balancers are just “dumb” TCP balancers which forward TCP connections and do nothing at layer 7. Others can act as full reverse proxies (e.g. nginx) and these are capable of doing what you need.

From a quick scan of the Brocade ServerIron ADX documentation, it doesn’t seem capable of serving error documents on HTTP requests. While it’s definitely smarter than your typical “dumb” TCP forwarder, it’s probably not going to do what you want here.

A reverse proxy such as nginx would be capable of this, though if you set that up, you may as well just replace the load balancer (since nginx can do HTTP/HTTPS load balancing as well).

View the full question and answer on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.