11 distributed key-value stores, as reviewed by last.fm engineers

Here is a list of projects that could potentially replace a group of relational database shards. Some of these are much more than key-value stores, and aren’t suitable for low-latency data serving, but are interesting none-the-less.

Name Language Fault-tolerance Persistence Client Protocol Data model
Project Voldemort Java partitioned, replicated, read-repair Pluggable: BerkleyDB, Mysql Java API Structured / blob / text
Ringo Erlang partitioned, replicated, immutable Custom on-disk (append only log) HTTP blob
Scalaris Erlang partitioned, replicated, paxos In-memory only Erlang, Java, HTTP blob
Kai Erlang partitioned, replicated? On-disk Dets file Memcached blob
Dynomite Erlang partitioned, replicated Pluggable: couch, dets Custom ascii, Thrift blob
MemcacheDB C replication BerkleyDB Memcached blob
ThruDB C++ Replication Pluggable: BerkleyDB, Custom, Mysql, S3 Thrift Document oriented
CouchDB Erlang Replication, partitioning? Custom on-disk HTTP, json Document oriented (json)
Cassandra Java Replication, partitioning Custom on-disk Thrift Bigtable meets Dynamo
HBase Java Replication, partitioning Custom on-disk Custom API, Thrift, Rest Bigtable
Hypertable C++ Replication, partitioning Custom on-disk Thrift, other Bigtable

Awesome review of a bunch of possible Key-value store pairs. Tokyo Tyrant conspicuously missing.