Fair-proxy balancer patch for nginx - this can keep new connections away from busy Mongrels, but if all (concurrency-disabled) Mongrels in your pool get busy, then you'll still get 502 errors.
We just hit this this morning when we tried to flip on our new and improved HAProxy + Nginx + mongrel clusters architecture. Throughput was fantastic, but I couldn't figure out why I was getting intermittent 502 errors.
Doesn't that swap one relatively annoying problem with a total fail? How is fair proxy balancer (an nginx mod) even useful in the least if when mongrels are all busy, nginx just returns a 502 Bad Gateway that completely hoses the user?
That's actually INFINITELY worse than just making someone wait 500msec longer than they should for a fast request.
I'll post again about our setup when I have all the ducks in order again, but unless something is wrong with my setup, nginx + fair proxy balancer is utterly unusable in production systems.
I'll be switching to serving nginx on the frontend for static content, with a proxy from nginx to my mongrel cluster being passed through HAProxy. HAProxy will then use the maxconn 1 flag to provide the queue-busting load balancing algorithm I wanted in the first place.
Now what to do about load balancing static content? nginx upstream reverse proxy to another cluster of nginx's? *wince*
Edit: The creator of nginx-ey-balancer contacted me and said hey, use this instead. I already switched to using haproxy for mongrel reverse proxy, but wanted to make sure to link to the github project. Looks to be a non-screwed-up version of what the fair proxy balancer was supposed to be. =) Thanks ry!