Cloud architecture: Notes from an Amazon talk

 

Some notes from a talk I was at. Didn’t get time to write it in detail. But hey, something is better than nothing… right ?

Design for failure

        - handle failure
            - use elastic ip addresses
            - use multiple amazon ec2 availability zones
            - create mutliple database slaves across multiple zones
            - use real-time monitoring (amazon cloudwatch)
            - use amazon EBS for persistent file system
                - snapshot database to s3 (from ebs)
  

Loose coupling sets you free

        - independent components
        - design everything as a blackbox
        - de-coupling for hybrid models
        - loadbalance-clusters
        - use SQS as buffers to queue messages. Allows elasticity
  

Design for dynamism


        - build for changes in infrastructure 
            - Don't assume health of fixed location of components
            - Use designs that are resilient to reboot and re-launch
            - Bootstrap your instances
            - Enable dynamic configuration
                - Enable Self discovery
                    (puttet, chef, ?)
            - Free auto-scaling features (by triggers)
            - Use Elastic loadbalancing on multiple layers
            - Use configurations in SimnpleDB to bootstrap instances

Build security in every layerider encrypted files


        - Physical is free
        - network is easy
            - Can confider app to talk to only web and db layer... etc. Everything can be automated.
        - The rest can be added
            - Create distinct Security Groups for each Amazon EC2 cluster
            - Use group-based rules for controlling access between layers
            - Restrict external access to specific IP ranges
            - Encrypt data "at-rest" in Amazon S3
            - Encrypt data "in-transit" (SSL)
            - Consider encrypted file systems in EC2 for sensitive data

Dont fear constraints

        - More RAM ?
            Distribute load across machines. Shared distributed cache
        - Better IOPS on my database ?
            Multiple read0only / sharding / DB clustering
        - Your server has better config ?
            Implement elasticity
        - Static IP ?
            Boot script for software reconfiguration from SimpleDB

        -
  

Leverage aws storage solutions


        - Amazon S3: for large static objects (whats the maximum size per object ?)
        - Amazon Coudfront: content distribution
        - Amazon SimpleDB: simple data indexing/querying
        - Amazon EC2 local disk drive: transient data
        - Amazon EBS: RDBMS persistent storage + S3 Snapshots

Comments

Nadya said…
I am wondering if you can check out my very own tool CloudBerry Explorer that helps to manage S3 on Windows . It is a freeware. http://cloudberrylab.com/
8:46 PM
Post a Comment

Popular posts from this blog

Chrome Frame - How to add command line parameters

Image
Chrome frame intentionally does stuff without getting in the way of the user. This sometimes makes things harder to debug. For example how can one debug an issue if chrome frame doesn't even launch ? Apparently there is a flag for that. But you have to know how to enable it. Here are the steps. Make sure chrome frame is installed. We can enable startup flags for dumping debug logs using a policy called AdditionalLaunchParameters If this is just for one desktop, I recommend doing a registry edit (it can be pushed via GPO as well) Add a REG_SZ property " AdditionalLaunchParameters " to " SoftwarePoliciesGoogleChromeAdditionalLaunchParameters " with the value "--enable-logging --v=1" (also documented here  and mentioned here )  [ Attachment 1 ] Next kill the IE browser and make sure chrome is also dead by checking taskmgr Restart IE and go to " gcf:about:version " and confirm that the parameters you added show up next to "Command Line:"
Read more

Creating your first chrome app on a Chromebook

Image
Doing development on Chromebook usually means developing online. There are a lot of sites around for that. But with the APIs getting more mature, its just a matter of time before someone builds a kick-ass IDE which runs natively on Chromebooks without network connectivity. One such tool which I've been exploring for a last few weeks is from Google and called " Chrome Dev Editor " If you have never built and run a chrome app or extension, I'll show you how to do this in 3 easy steps.     Step 1 Launch the app and click on the "+" button to add a new project. Pick a project name and select "JavaScript Chrome App" in the drop down menu.       Step 2 Notice how it automatically adds the minimum code required for it to run. This will be the template you are going to work with. You are almost done, but lets review the files in there to understand why they are there.   manifest.json - This is the most important file. Modify the app name, version number
Read more

Brewers CAP Theorem on distributed systems

Image
Large distributed systems run into a problem which smaller systems don’t usually have to worry about. “ Brewers CAP Theorem ” [ Ref 1 ] [ Ref 2 ] [ Ref 3 ]  defines this problem in a very simple way. It states, that though its desirable to have Consistency, High-Availability and Partition-tolerance in every system, unfortunately no system can achieve all three at the same time . C onsistent : A fully Consistent system is one where the system can guarantee that once you store a state (lets say “x=y”) in the system, it will report the same state in every subsequent operation until the state is explicitly changed by something outside the system. [ Example 1 ] A single MySQL database instance is automatically fully consistent since there is only one node keeping the state. [ Example 2 ] If two MySQL servers are involved, and if the system is designed in such a way that all keys starting “a” to “m” is kept on server 1, and keys “n” to “z” are kept on server
Read more