It has been a while since I got my hands dirty writing code. So here is a tool I cooked up last night.
Share Quickly is a chrome extension which takes a screenshot of the active tab and uploads it to a public website which makes screenshot sharing a one click task. I plan to make these images disappear from cache after some time to save on storage… haven’t figure out how to expose that yet.
The files are hosted on Google’s App engine using cloud storage and the front end for that service is on Google Compute engine. I choose to write this in PHP instead.
Interestingly the backend storage is not limited to images only. You can actually use http://sharequickly.appspot.com/ to save and share arbitrary pieces text. Note that none of this is pretty, so be warned.
Feature requests and general comments are welcome.
Building your first web application on AWS is like shopping for a car at pepboys, part by part. While manuals to build one might be on aisle 5, the experience of having built one already is harder to buy.
Here are some interesting logistical questions, which I donâ€™t think get enough attention, when people discuss issues around building a new AWS based service.
- Picking the right Linux distribution: Switching OS distribution may not be too simple if your applications need custom scripts. Picking and sticking with a single distribution will save a lot of lost time.
- Automated server builds: There are many ways to skin this cat. Chef, Puppet, Cfengine are all good… Whats important is to pick one early in the game.
- Multi-Availability Zone support: Find out if multi availability zone support is important. This can impact over all architecture of the solution and tools used to implement the solution.
- Data consistency requirements: Similar to the Multi-AZ support question, its important to understand the data consistency tolerance of the application before one starts designing the application.
- Datastore: There are different kinds of datastores available as part of AWS itself (SimpleDB, S3 and RDS). If you are planning to keep your options open about moving out of AWS at some point, you should think about picking a datastore which you could move out with you with little effort. There are many NoSQL and RDBMS solutions to choose from.
- Backups: While some think its a waste of time to think about backups too early, I suspect those who donâ€™t will be spending way too much time later. The long term backup strategy is integral part of disaster recovery planning, without which you shouldnâ€™t think of going live.
- Integration with external data sources: If this application is part of a larger cluster of application which is running somewhere else, think about how data would be sent back and forth. There are lots of different options depending on how much data is involved (or how important protection of that data is)
- Monitoring/Alerting: Most standard out of the box monitoring tools canâ€™t handle dynamic infrastructure very well. There are, however, plugins available for many existing monitoring solutions which can handle the dynamic nature of infrastructure. You could also choose to use one of the 3rd party monitoring services if youâ€™d rather pay someone else to do it.
- Security: You should be shocked to see this on #9 on my list. If your service involves user data, or some other kind of intellectual property, build multi-tiered architecture to segment different parts of your application from targeted attacks. Security is also very important while picking the right caching and web server technologies.
- Development: Figure out how developers would use AWS. Would they share the same AWS account, share parts of the infrastructure, share datastore, etc. How would the developer resources be monitored so that unintentional uses of excessive resources could be flagged for alerting.
Are there other subtle issues which I should have listed here ? Let me know.
I had briefly blogged about this presentation before from Velocity 2010. I wish they had released the video for this session. I went through this slide deck again today to see if Paul mentioned any of the problems organization like ours are dealing with in its transition from quarterly releases to weekly/continuous releases.
One of the key observations Paul made during his talk is that most organizations still treat web applications as desktop software and have very strict quality controls which may not be as necessary since releasing changes for web app in a SAAS (Software as a service) is much more cheaper than for releasing patches for traditional desktop software.
Here are some of the other points he made. For really detailed info check out the [slides]
- Deploy frequently, facilitating rapid product iteration
- Avoid large merges and the associated integration testing
- Easily perform A/B testing of functionality
- Run QA and beta testing on production hardware
- Launch big features without worrying about your infrastructure
- Provide all the switches your operations team needs to manage the deployed system
Slides: Always ship Trunk: Managing Change In Complex Websites
Windows Azure is an application platform provided by Microsoft to allow others to run applications on Microsoftâ€™s â€œcloudâ€ infrastructure. Its finally open for business (as of Feb 1, 2010). Below are some links about Azure for those who are still catching up.
Wikipedia: Windows Azure has three core components: Compute, Storage and Fabric. As the names suggest, Compute provides computation environment with Web Role and Worker Role while Storage focuses on providing scalable storage (Blobs, Tables, Queue) for large scale needs.
The hosting environment of Windows Azure is called the Fabric Controller – which pools individual systems into a network that automatically manages resources, load balancing, geo-replication and application lifecycle without requiring the hosted apps to explicitly deal with those requirements. In addition, it also provides other services that most applications require â€” such as the Windows Azure Storage Service that provides applications with the capability to store unstructured data such as binary large objects, queues and non-relational tables. Applications can also use other services that are a part of the Azure Services Platform.
If you donâ€™t like EC2 you have an option to move your app to a new vendor. But if you donâ€™t like GAE (Google app engine) there arenâ€™t any solutions which can replace GAE easily.
AppScale might change that.
AppScale is an open-source implementation of the Google AppEngine (GAE) cloud computing interface from the RACELab at UC Santa Barbara. AppScale enables execution of GAE applications on virtualized cluster systems. In particular, AppScale enables users to execute GAE applications using their own clusters with greater scalability and reliability than the GAE SDK provides. Moreover, AppScale executes automatically and transparently over cloud infrastructures such as the Amazon Web Services (AWS) Elastic Compute Cloud (EC2) and Eucalyptus, the open-source implementation of the AWS interfaces.
The list of supported infrastructures is very impressive. However the key, in my personal opinion, would be stability and compatibility with current GAE APIs.
Learn more about AppScale:
- AppScale Home page
- Google Code page
- Google Group for AppScale
- Demo at Bay area GAE Developers meeting: At Googleplex ( Feb 10, 2010)