Why the process-monitoring ruby 'god' broke

We were having all kinds of weird problems with god, and had to resort to weird hacky solutions to try to make it work. Turns out, there's a reason for it. If you try to do all sorts of shell-like actions in the commands for start/stop/restart, god ends up throwing it all in a shell process, e.g. sh -c your_cmd.

But then god gets the pid to the SHELL process and not your_cmd. And when you want to restart, it ends up killing the shell command and leaving you with zombie, orphaned, totally untracked processes that you have to kill -9 manually. Gruesome.

So in short, this stuff you should AVOID:

w.start = "cd #{RAILS_ROOT} && QUEUE=#{name} rake do:mytask"

This is OK:

w.dir = RAILS_ROOT
w.env = {'QUEUE'=> name}
w.start = "rake do:mytask"

And knowing is half the battle!

EDIT: You could also think of using Bluepill, which is God without the insane memory leak (which apparently is fixed in some hacker's dev branch!) It's in use at Serious Business, and they are one hardcore operation. http://github.com/arya/bluepill