State of current NoSQL databases : A very detailed post about many NoSQL solutions. A lot of work went into this one.
Truth about joins: Google app engine datastore’s limitation of not allowing joins might soon be a thing of the past. Simple joins may now be possible on GAE if you are using Java. Its [...]

Few weeks ago while I was mulling over what kind of service registry/discovery system to use for a scalable application deployment platform, I realized that for mid-size organizations with complex set of services, building one from scratch may be the only option.
I also found out that many AWS/EC2 customers have already been using S3 and [...]

This is a very interesting talk by Jonathan Ellis on database scalability. He designed and implemented multi-petabyte storage for Mozy and is currently the project chair for Apache Cassandra.

What every developer should know about database scalability, PyCon 2010
View more presentations from jbellis.

Scalability is not improving latency, but increasing throughput
But overall performance shouldn’t degrade
Throw [...]

Syslog is a commonly used transport mechanism for system logs. But people sometimes forget it could be used for a lot of other purposes as well.
Take, for example, the interesting challenge of aggregating web server logs from 100 different servers into one server and then figuring out how to merge them. If you have [...]

We discussed Brewer’s Theorm a few days ago and how its challenging to obtain Consistency, Availability and Partition tolerance in any distributed system. We also discussed that many of the distributed datastores allow CAP to be tweaked to attain certain operational goals.
Amazon SimpleDB, which was released as an “Eventually Consistent” datastore,  today launched a [...]

NoSQL solutions have one thing in common. They are generally designed for horizontal scalability. So its no wonder that lot of applications in the “twitter” world have picked NoSQL based datastores for their persistence layer. Here is a collection of these apps from MyNoSQL blog.

Twitter uses Cassandra
MusicTweets used Redis [ Ref ] – The [...]

So there is someone who thinks “eventual consistency is just caching”.  Though I liked the idea of discussing this, I don’t agree with Udi’s views on this.
“Cache” is generally used to store data which is more expensive to obtain from the primary location. For example, caching mysql queries is ideal for queries which could [...]

Some of interesting links for today

A very good post about the need of event driven Cloud API model for monitoring. I think its a matter of time before this happens. Just like feed crawlers are embracing even driven publication notification using protocols like Pubsubhubbub, we need something similar to snmp traps for the cloud notification [...]

If you missed the AWS S3 versioning webcast, I have a copy of the video here. And here are the highlights..

You can enable and disable this at the bucket level
They don’t think there is a performance penalty of turning versioning (but it was kind of obvious S3 would be doing slightly extra work [...]

Large distributed systems run into a problem which smaller systems don’t usually have to worry about. “Brewers CAP Theorem” [ Ref 1] [ Ref 2] [ Ref 3]  defines this problem in a very simple way.
It states, that though its desirable to have Consistency, High-Availability and Partition-tolerance in every system, unfortunately no system can [...]