royans

Few weeks ago while I was mulling over what kind of service registry/discovery system to use for a scalable application deployment platform, I realized that for mid-size organizations with complex set of services, building one from scratch may be the only option. I also found out that many AWS/EC2 customers have already been using S3 and SimpleDB to  publish/discover services. That discussion eventually led me to investigate Cassandra as the service registry datastore in an enterprise network. Here are some of the observations I made as I played with Cassandra for this purpose.  I welcome feedback from readers if you think I’m doing something wrong or if you think I can improve the design further. The biggest issue I noticed with Cassandra was the absence of inverted index which could be worked around as I have blogged here . I later realized there is something called Lucandra   as well which I need to look at, at some point. The keyspace structure I used was very sim

This is a very interesting talk by Jonathan Ellis on database scalability. He designed and implemented multi-petabyte storage for Mozy and is currently the project chair for Apache Cassandra . What every developer should know about database scalability, PyCon 2010 View more presentations from jbellis . Scalability is not improving latency, but increasing throughput But overall performance shouldn’t degrade Throw hardware, not people at the problem Traditional databases use b-tree indexes. But requires the entire index to be in-memory at the same place. Easy bandaid #1– SSD storage is better for b-tree indexes which need to hit disk Easy bandaid #2 – Buy faster server every 2 years. As long as your userbase doesn’t grow faster that Moore’s law Easy bandaid #3 – Use caching to handle hotspots (Distributed) Memcache server failures can change where hashing keys are kept Consistent hashing solves the problem by mapping keys to to