Showing posts from January 17, 2010

Architecting for the Cloud: Best practices

Amazon has published another “ Best practices ” document. This one covers the almost the entire collection of services. Its biased towards AWS (obviously), but its still one of the best description summary of the various services amazon offers today. Just the diagram above tells a lot about how the various AWS services interact with each other. Here is another small section from the document. AWS specific tactics to automate your infrastructure Define Auto-scaling groups for different clusters using the Amazon Auto-scaling feature in Amazon EC2. Monitor your system metrics (CPU, Memory, Disk I/O, Network I/O) using Amazon CloudWatch and take appropriate actions (launching new AMIs dynamically using the Auto-scaling service) or send notifications. Store and retrieve machine configuration information dynamically: Utilize Amazon SimpleDB to fetch config data during boot-time of an instance (eg. database connection strings). SimpleDB may also be used to st

Monitoring large-scale application clusters

Most software engineering organizations build applications with some hooks in place to allow functional tests. Some organizations continuously build and test all software automatically at check-in. And then there are those who have learnt from mistakes, and have built a suite of tests which get triggered at startup to look for problems which could indicate a failed initialization. The next step in building a scalable web application, is creating some form of self-monitoring logic (sometimes called a watchdog) which could periodically test itself (or monitor performance statistics) for problems worth escalating to operations team. Arnon has a couple of (1) interesting   (2) posts on the topic which I came across today. He summarized the whole suggestion using 3 acronymns. And I’m going to add one more to make it complete BBIT : Build time Built in Tests. PBIT: Power-on Built in Tests CBIT: Continuous Built in Tests IBIT: Initiated Built in Tests The organizat