July 11, 2009

Cloud architecture: Notes from an Amazon talk

 

Some notes from a talk I was at. Didn’t get time to write it in detail. But hey, something is better than nothing… right ?

Design for failure

        - handle failure
            - use elastic ip addresses
            - use multiple amazon ec2 availability zones
            - create mutliple database slaves across multiple zones
            - use real-time monitoring (amazon cloudwatch)
            - use amazon EBS for persistent file system
                - snapshot database to s3 (from ebs)
  

Loose coupling sets you free

        - independent components
        - design everything as a blackbox
        - de-coupling for hybrid models
        - loadbalance-clusters
        - use SQS as buffers to queue messages. Allows elasticity
  

Design for dynamism


        - build for changes in infrastructure 
            - Don't assume health of fixed location of components
            - Use designs that are resilient to reboot and re-launch
            - Bootstrap your instances
            - Enable dynamic configuration
                - Enable Self discovery
                    (puttet, chef, ?)
            - Free auto-scaling features (by triggers)
            - Use Elastic loadbalancing on multiple layers
            - Use configurations in SimnpleDB to bootstrap instances

Build security in every layerider encrypted files


        - Physical is free
        - network is easy
            - Can confider app to talk to only web and db layer... etc. Everything can be automated.
        - The rest can be added
            - Create distinct Security Groups for each Amazon EC2 cluster
            - Use group-based rules for controlling access between layers
            - Restrict external access to specific IP ranges
            - Encrypt data "at-rest" in Amazon S3
            - Encrypt data "in-transit" (SSL)
            - Consider encrypted file systems in EC2 for sensitive data

Dont fear constraints

        - More RAM ?
            Distribute load across machines. Shared distributed cache
        - Better IOPS on my database ?
            Multiple read0only / sharding / DB clustering
        - Your server has better config ?
            Implement elasticity
        - Static IP ?
            Boot script for software reconfiguration from SimpleDB

        -
  

Leverage aws storage solutions


        - Amazon S3: for large static objects (whats the maximum size per object ?)
        - Amazon Coudfront: content distribution
        - Amazon SimpleDB: simple data indexing/querying
        - Amazon EC2 local disk drive: transient data
        - Amazon EBS: RDBMS persistent storage + S3 Snapshots

Is Percentage of company Bloggers/Twitter_users inversely proportional to Company size ?

Small organizations often keep a very active online presence . For them, any news is good news. Larger organizations however try to be opposite of that and control information.

What I’ve been trying to understand is how in spite of all that companies like Google and Microsoft still manage to have a huge online presence.

No.Of.TwitterAccounts= (Size.Of.Company)^(1/2)  ?

For example today, Google announced a list of all of its Twitter accounts in one page. 

How do they do it ?

General
twitter.com/Google - our central account
twitter.com/Blogger - for Blogger fans
twitter.com/GoogleCalendar - user tips & updates
twitter.com/GoogleImages - news, tips, tricks on our visual image search
twitter.com/GoogleNews - latest headlines via Google News
twitter.com/GoogleReader - from our feed reader team
twitter.com/iGoogle - news & notes from Google's personalized homepage
twitter.com/GoogleStudents - news of interest to students using Google
twitter.com/YouTube - for YouTube fans
twitter.com/YouTubeES - en Espanol
twitter.com/GoogleAtWork - solutions for IT and workplace productivity

Geo-related
twitter.com/SketchUp - Google SketchUp news
twitter.com/3DWH - SketchUp's 3D Warehouse
twitter.com/Modelyourtown - 3D modeling to build your favorite places
twitter.com/EarthOutreach - Earth & Maps tools for nonprofits & orgs
twitter.com/GoogleMaps - uses, tips, mashups
twitter.com/GoogleSkyMap -Android app for the night sky

Ads-related
twitter.com/AdSense - for online publishers
twitter.com/AdWordsHelper - looking out for AdWords questions and tech issues
twitter.com/AdWordsProSarah - Google Guide for AdWords Help Forum
twitter.com/GoogleAnalytics - insights for website effectiveness
twitter.com/GoogleAdBuilder - re building display ads
twitter.com/GoogleRetail - for retail advertisers
twitter.com/TechnologyUK - for U.K. tech advertisers
twitter.com/InsideAdWordsDE - for German AdWords customers
twitter.com/GoogleAgencyDE - for German ad agencies
twitter.com/AdSensePT - info for Portuguese-language publishers
twitter.com/AdWordsRussia - AdWords news & tips in Russian
twitter.com/DentroDeAdWords - Spanish updates from the Inside

AdWords blog
twitter.com/AdWordsAPI - AdWords API tips

Developer & technical
twitter.com/GoogleResearch - from our research scientists
twitter.com/GoogleWMC - Google Webmaster Central
twitter.com/GoogleCode - latest updates for Google developer products
twitter.com/GoogleData - Data APIs provide a standard protocol for reading and writing web data
twitter.com/app_engine - web apps run on Google infrastructure
twitter.com/DataLiberation - our initiative for complete import/export of all data
twitter.com/GoogleMapsAPI - about using Google Maps embedded in websites
twitter.com/GoogleIO - Google's largest annual developer event

Culture, People
twitter.com/googletalks - notes from our @Google speaker series
twitter.com/googlejobs - the voice of Google recruiters

Country or Region
twitter.com/googledownunder - Google activities in Australia & New Zealand
twitter.com/GoogleDE - Google in Germany
twitter.com/GoogleLatAm - Latin America (en Espanol)
twitter.com/GooglePolicyIt - Notes on Google policy issues in Italy