Comments on: Cassandra : inverted index http://www.royans.net/arch/cassandra-inverted-index/ Building reliable, high performance, highly available clusters Mon, 12 Jul 2010 14:35:55 +0000 hourly 1 http://wordpress.org/?v=3.0 By: Royans http://www.royans.net/arch/cassandra-inverted-index/comment-page-1/#comment-697 Royans Sun, 07 Mar 2010 16:57:09 +0000 http://www.royans.net/arch/cassandra-inverted-index/#comment-697 Jeff, The solution I came up with, addresses a particular problem I was having with using cassandra in managing a glorified service state registry which I was building. I wanted Cassandra’s availability and partition-tolerance features and didn’t really care about its scalability because I knew what my dataset size would be. The implementation doesn’t require “read-extend-write”. You can just “write” to the column-family directly because I know what key to add. I also don’t have any use case to to a scan entire index… all I need is last N rows to find something I need from an index. As far as Cassandra is concerned, that table is just a huge ordered list of rows, so I don’t think there should be a performance issue. I think you could make an argument that indexes could be out of sync from data at times since they are not in the same column Family, but again the application I’m building is ok with minor issues like that. My understanding is that what I’m doing is not too far away from how others are using Cassandra. Some sample schema designs from Evan @ twitter shows that they might also use similar way of indexing data using Cassandra. http://blog.evanweaver.com/articles/2009/07/06/up-and-running-with-cassandra/ rkt Jeff,

The solution I came up with, addresses a particular problem I was having with using cassandra in managing a glorified service state registry which I was building. I wanted Cassandra’s availability and partition-tolerance features and didn’t really care about its scalability because I knew what my dataset size would be.

The implementation doesn’t require “read-extend-write”. You can just “write” to the column-family directly because I know what key to add. I also don’t have any use case to to a scan entire index… all I need is last N rows to find something I need from an index. As far as Cassandra is concerned, that table is just a huge ordered list of rows, so I don’t think there should be a performance issue.

I think you could make an argument that indexes could be out of sync from data at times since they are not in the same column Family, but again the application I’m building is ok with minor issues like that.

My understanding is that what I’m doing is not too far away from how others are using Cassandra. Some sample schema designs from Evan @ twitter shows that they might also use similar way of indexing data using Cassandra. http://blog.evanweaver.com/articles/2009/07/06/up-and-running-with-cassandra/

rkt

]]> By: Jeff Darcy http://www.royans.net/arch/cassandra-inverted-index/comment-page-1/#comment-696 Jeff Darcy Sun, 07 Mar 2010 16:32:56 +0000 http://www.royans.net/arch/cassandra-inverted-index/#comment-696 The problem of updating an inverted index is much worse than merely an extra update per new cell. The first extra bit of pain is dealing with concurrent updates; a simple read-extend-write of the index now falls prey to the classic problem of N-1/N concurrent updates being lost. The next bit of pain is dealing with a really big index. If you have a few thousand rows, let alone a billion, updates of a simple index are going to be extremely inefficient. Now you'll have to maintain your index as a B-tree or some such using multiple rows. Now you get to the most painful part of all: concurrent multi-row updates. If you want to support something as simple as SQL's "ORDER BY x LIMIT n" for a potentially large dataset with just about any of the simpler distributed key/value or column stores, it's going to be rather unpleasant. Note that I'm not saying this invalidates the NoSQL approach. I'm pretty well known as a NoSQL *advocate* myself. It's just something people have to be aware of as they're working through their scale/feature/CAP requirements. The problem of updating an inverted index is much worse than merely an extra update per new cell. The first extra bit of pain is dealing with concurrent updates; a simple read-extend-write of the index now falls prey to the classic problem of N-1/N concurrent updates being lost. The next bit of pain is dealing with a really big index. If you have a few thousand rows, let alone a billion, updates of a simple index are going to be extremely inefficient. Now you’ll have to maintain your index as a B-tree or some such using multiple rows. Now you get to the most painful part of all: concurrent multi-row updates. If you want to support something as simple as SQL’s “ORDER BY x LIMIT n” for a potentially large dataset with just about any of the simpler distributed key/value or column stores, it’s going to be rather unpleasant.

Note that I’m not saying this invalidates the NoSQL approach. I’m pretty well known as a NoSQL *advocate* myself. It’s just something people have to be aware of as they’re working through their scale/feature/CAP requirements.

]]> By: Scalability updates for Feb 18, 2010 | Scalable web architectures http://www.royans.net/arch/cassandra-inverted-index/comment-page-1/#comment-448 Scalability updates for Feb 18, 2010 | Scalable web architectures Fri, 19 Feb 2010 05:17:57 +0000 http://www.royans.net/arch/cassandra-inverted-index/#comment-448 [...] backend for Lucene ? This seems to solve the problem of building reverse index on cassandra which I previously blogged [...] [...] backend for Lucene ? This seems to solve the problem of building reverse index on cassandra which I previously blogged [...]

]]> By: Jonathan Ellis http://www.royans.net/arch/cassandra-inverted-index/comment-page-1/#comment-327 Jonathan Ellis Sun, 07 Feb 2010 18:34:52 +0000 http://www.royans.net/arch/cassandra-inverted-index/#comment-327 @Royans Sorry, I read too fast. You're right. :) @Royans Sorry, I read too fast. You’re right. :)

]]> By: Royans http://www.royans.net/arch/cassandra-inverted-index/comment-page-1/#comment-326 Royans Sun, 07 Feb 2010 16:46:19 +0000 http://www.royans.net/arch/cassandra-inverted-index/#comment-326 @jothanan : I assumed that if 1 row of data needs 5 more rows of inserts/updates that would make a total of 6 rows of inserts/updates required per row of data inserted. 10k/6 =~ 1.5K. But I could be wrong, and would appreciate if you could shed a little light on why you think it can still do 10k per second. @jothanan : I assumed that if 1 row of data needs 5 more rows of inserts/updates that would make a total of 6 rows of inserts/updates required per row of data inserted. 10k/6 =~ 1.5K.
But I could be wrong, and would appreciate if you could shed a little light on why you think it can still do 10k per second.

]]> By: Royans http://www.royans.net/arch/cassandra-inverted-index/comment-page-1/#comment-325 Royans Sun, 07 Feb 2010 16:43:11 +0000 http://www.royans.net/arch/cassandra-inverted-index/#comment-325 @adrian : Of course. Cassandra looked like a better fit at this time for me, but I'm sure you can do this on any kind of key-value store. @adrian : Of course. Cassandra looked like a better fit at this time for me, but I’m sure you can do this on any kind of key-value store.

]]> By: Jonathan Ellis http://www.royans.net/arch/cassandra-inverted-index/comment-page-1/#comment-323 Jonathan Ellis Sun, 07 Feb 2010 14:55:01 +0000 http://www.royans.net/arch/cassandra-inverted-index/#comment-323 Correction: the 10k rows/s number _is_ with 5-column rows. Correction: the 10k rows/s number _is_ with 5-column rows.

]]> By: Adrian http://www.royans.net/arch/cassandra-inverted-index/comment-page-1/#comment-320 Adrian Sun, 07 Feb 2010 10:44:09 +0000 http://www.royans.net/arch/cassandra-inverted-index/#comment-320 I'm pretty sure you can also implement inverted indexes with HBase, too. I’m pretty sure you can also implement inverted indexes with HBase, too.

]]>