Comments for Scalable web architectures http://www.royans.net/arch Building reliable, high performance, highly available clusters Sat, 13 Mar 2010 15:42:19 +0000 http://wordpress.org/?v=2.9.2 hourly 1 Comment on The Reddit problem: Learning from mistakes by Scalability links for March 13th 2010 | Scalable web architectures http://www.royans.net/arch/reddit-learning-from-mistakes/comment-page-1/#comment-737 Scalability links for March 13th 2010 | Scalable web architectures Sat, 13 Mar 2010 15:42:19 +0000 http://www.royans.net/arch/reddit-learning-from-mistakes/#comment-737 [...] I suspected, Reddit is now moving to Cassandra – Another [...] [...] I suspected, Reddit is now moving to Cassandra – Another [...]

]]> Comment on Automated, faster, repeatable, scalable deployments by Scalability links for March 13th 2010 | Scalable web architectures http://www.royans.net/arch/automated-faster-repeatable-scalable-deployments/comment-page-1/#comment-736 Scalability links for March 13th 2010 | Scalable web architectures Sat, 13 Mar 2010 15:39:26 +0000 http://www.royans.net/arch/automated-faster-repeatable-scalable-deployments/#comment-736 [...] Automated, faster, repeatable, scalable deployments  [...] [...] Automated, faster, repeatable, scalable deployments  [...]

]]> Comment on Cassandra : inverted index by Royans http://www.royans.net/arch/cassandra-inverted-index/comment-page-1/#comment-697 Royans Sun, 07 Mar 2010 16:57:09 +0000 http://www.royans.net/arch/cassandra-inverted-index/#comment-697 Jeff, The solution I came up with, addresses a particular problem I was having with using cassandra in managing a glorified service state registry which I was building. I wanted Cassandra’s availability and partition-tolerance features and didn’t really care about its scalability because I knew what my dataset size would be. The implementation doesn’t require “read-extend-write”. You can just “write” to the column-family directly because I know what key to add. I also don’t have any use case to to a scan entire index… all I need is last N rows to find something I need from an index. As far as Cassandra is concerned, that table is just a huge ordered list of rows, so I don’t think there should be a performance issue. I think you could make an argument that indexes could be out of sync from data at times since they are not in the same column Family, but again the application I’m building is ok with minor issues like that. My understanding is that what I’m doing is not too far away from how others are using Cassandra. Some sample schema designs from Evan @ twitter shows that they might also use similar way of indexing data using Cassandra. http://blog.evanweaver.com/articles/2009/07/06/up-and-running-with-cassandra/ rkt Jeff,

The solution I came up with, addresses a particular problem I was having with using cassandra in managing a glorified service state registry which I was building. I wanted Cassandra’s availability and partition-tolerance features and didn’t really care about its scalability because I knew what my dataset size would be.

The implementation doesn’t require “read-extend-write”. You can just “write” to the column-family directly because I know what key to add. I also don’t have any use case to to a scan entire index… all I need is last N rows to find something I need from an index. As far as Cassandra is concerned, that table is just a huge ordered list of rows, so I don’t think there should be a performance issue.

I think you could make an argument that indexes could be out of sync from data at times since they are not in the same column Family, but again the application I’m building is ok with minor issues like that.

My understanding is that what I’m doing is not too far away from how others are using Cassandra. Some sample schema designs from Evan @ twitter shows that they might also use similar way of indexing data using Cassandra. http://blog.evanweaver.com/articles/2009/07/06/up-and-running-with-cassandra/

rkt

]]> Comment on Cassandra : inverted index by Jeff Darcy http://www.royans.net/arch/cassandra-inverted-index/comment-page-1/#comment-696 Jeff Darcy Sun, 07 Mar 2010 16:32:56 +0000 http://www.royans.net/arch/cassandra-inverted-index/#comment-696 The problem of updating an inverted index is much worse than merely an extra update per new cell. The first extra bit of pain is dealing with concurrent updates; a simple read-extend-write of the index now falls prey to the classic problem of N-1/N concurrent updates being lost. The next bit of pain is dealing with a really big index. If you have a few thousand rows, let alone a billion, updates of a simple index are going to be extremely inefficient. Now you'll have to maintain your index as a B-tree or some such using multiple rows. Now you get to the most painful part of all: concurrent multi-row updates. If you want to support something as simple as SQL's "ORDER BY x LIMIT n" for a potentially large dataset with just about any of the simpler distributed key/value or column stores, it's going to be rather unpleasant. Note that I'm not saying this invalidates the NoSQL approach. I'm pretty well known as a NoSQL *advocate* myself. It's just something people have to be aware of as they're working through their scale/feature/CAP requirements. The problem of updating an inverted index is much worse than merely an extra update per new cell. The first extra bit of pain is dealing with concurrent updates; a simple read-extend-write of the index now falls prey to the classic problem of N-1/N concurrent updates being lost. The next bit of pain is dealing with a really big index. If you have a few thousand rows, let alone a billion, updates of a simple index are going to be extremely inefficient. Now you’ll have to maintain your index as a B-tree or some such using multiple rows. Now you get to the most painful part of all: concurrent multi-row updates. If you want to support something as simple as SQL’s “ORDER BY x LIMIT n” for a potentially large dataset with just about any of the simpler distributed key/value or column stores, it’s going to be rather unpleasant.

Note that I’m not saying this invalidates the NoSQL approach. I’m pretty well known as a NoSQL *advocate* myself. It’s just something people have to be aware of as they’re working through their scale/feature/CAP requirements.

]]> Comment on The Reddit problem: Learning from mistakes by Royans http://www.royans.net/arch/reddit-learning-from-mistakes/comment-page-1/#comment-548 Royans Fri, 05 Mar 2010 14:59:32 +0000 http://www.royans.net/arch/reddit-learning-from-mistakes/#comment-548 Jereny, Thanks for the comments. I'm not surprised by the plan to move to Cassandra. Best of luck ! Jereny, Thanks for the comments. I’m not surprised by the plan to move to Cassandra. Best of luck !

]]> Comment on The Reddit problem: Learning from mistakes by Jereny Edberg http://www.royans.net/arch/reddit-learning-from-mistakes/comment-page-1/#comment-534 Jereny Edberg Fri, 05 Mar 2010 07:46:43 +0000 http://www.royans.net/arch/reddit-learning-from-mistakes/#comment-534 Hey there. I'm the guy that wrote the post. Your assessment is spot on. We didn't use libketama because it didn't exist at the time (a few years ago) and also because it just wasn't something people thought about. We could switch to it now, but as you said, we are going to drop memcacheDB, so there is no need to "fix" it. Also, the version of the memcached protocol that is the frontend for memcacheDB is old, and doesn't support the binary protocol. Most of the memcached libraries that support ketama require the binary protocol (not all of them). So yeah, it looks like we'll probably go to Cassandra in the very near feature. Hey there. I’m the guy that wrote the post. Your assessment is spot on.

We didn’t use libketama because it didn’t exist at the time (a few years ago) and also because it just wasn’t something people thought about. We could switch to it now, but as you said, we are going to drop memcacheDB, so there is no need to “fix” it. Also, the version of the memcached protocol that is the frontend for memcacheDB is old, and doesn’t support the binary protocol. Most of the memcached libraries that support ketama require the binary protocol (not all of them).

So yeah, it looks like we’ll probably go to Cassandra in the very near feature.

]]> Comment on The Reddit problem: Learning from mistakes by Royans http://www.royans.net/arch/reddit-learning-from-mistakes/comment-page-1/#comment-523 Royans Tue, 02 Mar 2010 18:47:11 +0000 http://www.royans.net/arch/reddit-learning-from-mistakes/#comment-523 Based on what I see, I think Reddit datastore predates libketama, and that they knew about this issue long time ago. BTW Reddit blog has entries from 2005 and looks like libketama was announced in 2007. But thats just a guess and would love for someone to confirm it. Based on what I see, I think Reddit datastore predates libketama, and that they knew about this issue long time ago. BTW Reddit blog has entries from 2005 and looks like libketama was announced in 2007. But thats just a guess and would love for someone to confirm it.

]]> Comment on The Reddit problem: Learning from mistakes by Igor http://www.royans.net/arch/reddit-learning-from-mistakes/comment-page-1/#comment-522 Igor Tue, 02 Mar 2010 18:42:03 +0000 http://www.royans.net/arch/reddit-learning-from-mistakes/#comment-522 I did not quite get Reddit mistake, what stoped them from using libketama as a consistent hashing for MemcacheDB? It's part of almost every memcached client now, so undoubtedly they must have had it. I did not quite get Reddit mistake, what stoped them from using libketama as a consistent hashing for MemcacheDB?
It’s part of almost every memcached client now, so undoubtedly they must have had it.

]]> Comment on Scalable logging using Syslog by Martin Scholl http://www.royans.net/arch/scalable-logging-using-syslog/comment-page-1/#comment-500 Martin Scholl Sat, 27 Feb 2010 07:21:35 +0000 http://www.royans.net/arch/scalable-logging-using-syslog/#comment-500 There is a great syslog replacement: http://www.rsyslog.com/ . rsyslogd is part of most distributed and is open source software. Rsyslog doesn't use UDP so there is no problem of loosing logs. Even more, rsyslog buffers on disk when an upstream rsyslogd is down. All in all, rsyslog is the better syslog. There is a great syslog replacement: http://www.rsyslog.com/ . rsyslogd is part of most distributed and is open source software.

Rsyslog doesn’t use UDP so there is no problem of loosing logs. Even more, rsyslog buffers on disk when an upstream rsyslogd is down. All in all, rsyslog is the better syslog.

]]> Comment on More on Amazon S3 versioning (webinar) by Andy, CloudBerry Lab http://www.royans.net/arch/more-on-amazon-s3-versioning-webinar/comment-page-1/#comment-499 Andy, CloudBerry Lab Sat, 27 Feb 2010 06:43:58 +0000 http://www.royans.net/arch/more-on-amazon-s3-versioning-webinar/#comment-499 I always enjoy learning what other people think about Amazon Web Services and how they use them. Check out my very own tool CloudBerry Explorer that helps manage S3 on Windows . It is a freeware. http://s3.cloudberrylab.com/ New version comes with Versioning support! I always enjoy learning what other people think about Amazon Web Services and how they use them. Check out my very own tool CloudBerry Explorer that helps manage S3 on Windows . It is a freeware. http://s3.cloudberrylab.com/ New version comes with Versioning support!

]]>