Offlining denormalized tasks - gotchas with offline queues and the after_commit callback.

In Rails, I'm not quite sure why nobody really tells you this stuff -- you kind of have to learn on your own.

We're using Workling with Starling. Things are great. (Starling is a perfectly fine queue for low-intensity tasks.) We use it for everything from sending emails to denormalizing our data.

Recently we finally fully upgraded to InnoDB for all tables. Epic win -- no more table locks! However, an unintended consequence happened. There were certain operations (e.g. adding posts to user subscriptions) that were offlined in after_save blocks. But if you look at ActiveRecord's docs, it notes:
Note that this callback is still wrapped in the transaction around save. For example, if you invoke an external indexer at this point it won‘t see the changes in the database.

Well, that's no good. Previously this wasn't an issue since MyISAM was pretty fast about commits. Turns out when using after_save, there was actually a race condition -- Workling would sometimes try to perform a job before the commit was completed, in which case we'd get "ID not found" errors since the record wasn't in there yet.

Enter the Rails after_commit plugin, which was pretty cool. It actually gives you the functionality you want if you're doing offline jobs that require the record to be complete before queueing for more work. However, our after_save callbacks were doing some optimization by checking the model.changed? flag and various related properties given to us by ActiveRecord::Dirty. But in after_commit callbacks, the model.changed? flags are reset to false, because the transaction is done.

The resolution? Move all references to the dirty bits to before_save callbacks that set flags in the instance variable. Then, when after_commit gets called, check those flags and perform the logic you want.

before_update :update_model_changed
after_commit_on_update :update_model

def update_model_changed
  @my_changed_flag = self.changed?

def update_model
  MyWorker.asynch_offline_job(:model_id => if @my_changed_flag

Three User Experience Lessons from the Terrible Garbage Web App Hall of Shame:

The web is incredibly empowering. Perhaps that's why when something is so thoroughly broken, I feel it viscerally in my stomach. Nausea and gnashing of teeth ensue.

That's what happened today when I opened up my browser to look for monthly parking in San Francisco and found IMPark's monthly parking search online. At first glance, everything seems fine.

But peer deeper into this web app and you'll find the soul of a twisted, infuriating, horribly broken waste of electrons.

Lesson 1: Reset state in Javascript on DOM load

Just because you set your form elements to be something in your outgoing HTML doesn't mean that you'll be in the correct state. They're doing an elementary AJAX call (why the hell is this in ajax anyway?) to refresh the cities. But what if I select USA, then refresh? This JS totally assumes that it will always be in the correct state -- but my browser stores form values after reload so as to avoid data loss. Incidentally, this is what happens also if you use the BACK button. And contrary to popular belief, the back button is great. Users love it. I love it.

What ensues isn't pretty...

OK, rookie mistake. But what happens when I select something, anything? I should be able to click to Canada, then back to USA, right?

Nope. Amateur hour. Then click around a bit more and you get...

WHAT THE...  How did we get here? Wow, total error fail. I hate you.

Lesson 2: Let me navigate the way I want to navigate

OK so I've searched for San Francisco. Here it is.

But for some reason, I can't cmd-click to open each of these lots in a new tab. This is a habit I've picked up to avoid the general latency of the web -- why round trip and go back/forth when you can just open a ton of tabs and triage as you see fit there?

Actually, this is a form element, of type submit. Are we in like 1995 again? What the hell is the point of

This is bad also for SEO. OK, amateur hour web dev, what are you thinking? You're costing IMPark money by hiding this info from Google. Way to go. If I search for "Mission Bay Parking" I should get to the page you're hiding behind a form input submit. TWO THUMBS DOWN.

Use a link. A HREF! Don't navigate using Javascript. Don't navigate using a POST from a form input submit. I'm starting to understand why people are so obsessed with REST-- because if you're RESTful, you won't do retarded things like this. Now I understand that the RESTful formalism is like a straightjacket that keeps an insane person from cutting themselves.

Lesson 3: Test your damn app people. Please.

OK, so I'm looking for lots cheaper than $250/month in my area. Yes, that's a parking space, not rent. That's the extortionary fee charged by my current apartment building in South of Market.

Oh, I guess there are none. Oh wait. That's not right.

FAIL!!! In fact pretty much all of the listings have rates that are less than $250/month. First, that confirms SoMA Mission Bay is basically the most god-awful overpriced yuppie-trap in all of San Francisco. Second, that confirms that the creator of is too busy smoking crack to test if their app works AT ALL.

In conclusion

IRONY OF IRONIES: The creators of this godforsaken mess were nominated for the MSDN Code Awards. Microsoft, HOW COULD YOU!?

I feel the KnowledgeTech development team should win the award because the Monthly Parking System project is a true testament to Microsoft technologies and tools and their high-level of integration and cohesiveness.

For Microsoft's sake... I hope not.

Cache Money - The bling bling painless way to do ActiveRecord object and collections caching

Most caching solutions in the Rails world involve something like Cache-Fu: an alternative API to ActiveRecord that explicitly annotates all call sites with cache rules.

  • User.find(1) becomes User.get_cache(1)
  • User.find(:all, ...) becomes User.get_cache("query_name", :ttl => 5.minutes) { User.find(:all, ... )}

I hate this kind of interface, which places the burden on the caller and meekly surrenders any attempt at encapsulation. Your codebase will be littered with haphazard cache rules in your controllers, views, and models.

This looks pretty awesome. Definitely going to investigate this some more.

Because manual cache expiry is not very fun. =)