How to scale your site: Solve only 80% of your problem (Often this is not by choice)

Solve only 80% of a problem. That's usually good enough and you'll not only get done faster, you'll actually have a chance of getting done at all.

This strategy is given by Amix in HOW TWITTER (AND FACEBOOK) SOLVE PROBLEMS PARTIALLY. The idea is solving 100% of a complex problem can be so hard and so expensive that you'll end up wasting all your bullets on a problem that could have been satisfactoraly solved in a much simpler way.

If you solve your scaling issue 100%, then you might not be working on features or other important things enough. In a startup, you just can't freaking afford the time and effort.

ActiveRecord uniqueness checks are 'inherently prone to race conditions"

Using this validation method in conjunction with ActiveRecord::Base#save does not guarantee the absence of duplicate record insertions, because uniqueness checks on the application level are inherently prone to race conditions. For example, suppose that two users try to post a Comment at the same time, and a Comment‘s title must be unique. At the database-level, the actions performed by these users could be interleaved in the following manner:

User 1                 |               User 2
 # User 1 checks whether there's     |
 # already a comment with the title  |
 # 'My Post'. This is not the case.  |
 SELECT * FROM comments              |
 WHERE title = 'My Post'             |
                                     | # User 2 does the same thing and also
                                     | # infers that his title is unique.
                                     | SELECT * FROM comments
                                     | WHERE title = 'My Post'
 # User 1 inserts his comment.       |
 INSERT INTO comments                |
 (title, content) VALUES             |
 ('My Post', 'hi!')                  |
                                     | # User 2 does the same thing.
                                     | INSERT INTO comments
                                     | (title, content) VALUES
                                     | ('My Post', 'hello!')
                                     | # ^^^^^^
                                     | # Boom! We now have a duplicate
                                     | # title!

Something to watch out for as you scale or build out your Rails app. This is the kind of stuff that only bites you when you get bigger.

The simple way to have your gem and eat it too: Symlinks to the rescue (Ruby on Rails Tip!)

Here's how to have your gem and eat it too. It's been quite a pain previously to try and a) modify functionality in a gem and b) also use it in a working dev project directory.

Previously, I've had to resort to git submodules or other horrendous integration points. But now (this is SO simple) I realize all I have to do is:
  1. Fork or check in your gem to github. Fully follow the instructions by github.
  2. Add config.gem 'username-gemname', :source => '' to your environment.rb file.
  3. Now in your local rails project directory, under vendor/gems you can create a symlink from your local dev copy of your gem to username-gemname, e.g. garrytan-ebayapi-0.12.1

Tada! Now your production / staging / other development instances (and your dev buddies) will pick up whatever is checked in as a gem, but you'll be able to mess around with your local copy of the gem and work on it without any hassle.

Don't forget to update your VERSION, generate a new gemspec, and push to github to make sure everyone else picks up your changes when you're done making changes to your gem.

I've forked and added more functionality to the ebayapi gem

If you want to create multi-user eBay apps, you'll need the ebayapi trading API XML gem created by Cody Fauser to let you:

  1. Get a session id (the existing version does not conform to the correct GetSessionID, passing XMLRequesterCredentials node instead of the specified one that passes DevID, AppID and AuthCert.
  2. Pass an RuName for your app from the config

The existing gem doesn't support these, but I've added it today. I'll probably post again soon about how to set up multi-user tokens as mentioned here using the updated ebayapi gem.

Here's the updated ebayapi gem on github. To install it on your machine, you can run:

XML-Mapping gem has really lame permission bug

You may have this error when require ‘xml/mapping’ in your ruby script:

/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require': no such file to load -- xml/mapping (LoadError)

Solution: go to the gem directory of XML-Mapping (mine is /usr/lib/ruby/gems/1.8/gems/xml-mapping-0.8.1/lib/) and chmod all files in this directory to 644.

Who is to point finger at here ? Rubygem or XML-Mapping authors ?

For some reason the default install has them installed with root-only permissions. I love how this is a Priority 3 bug assigned to nobody, and it blocks the use of this gem entirely.

Here's the fix (on OS X anyway)

sudo chmod -R a+r /Library/Ruby/Gems/1.8/gems/xml-mapping-0.8.1

Here's the thread to complain on.