Posts

Showing posts from 2011

Clock skew is the new TCP flag

The surprise in my eyes when I read through the papers about how "clock skew" can be used to fingerprint and identify hidden servers, was similar to the surprise I had when I read about nmap and OS fingerprinting the first time ( 2001 ? ). An eye opener in many ways. It reminded me of Dan Brown's book, "Angels and Demons", where they were on a hunt for the hidden "anti-matter particle" container. If the server broadcasting the image was on the internet, they could have flipped airconditioning ( in addition to lights) across the city to detect "clock skew" and narrow down to which part of the city it could have been. http://www.schneier. com/blog/archives/20 11/04/pinpointing_a_ c.html http://www.schneier. com/blog/archives/20 05/03/remote_physica l.html   http://www.caida.org /publications/papers /2005/fingerprinting /KohnoBroidoClaffy05 -devicefingerprintin g.pdf   http://www.hackitoer gosum.org/2010/HES20 10-rlifchitz-Fingerp rinting-hardwa

Humans are the most insecure device attached to computers..

The attack on RSA made me realize that an old joke needs to be updated. The one about "whats the most secure computer" Old Answer: "A computer which is disconnected from the internet, unpowered, buried under 10 feet of concrete." New Answer "A computer which is disconnected from the internet, unpowered, buried under 10 feet of concrete, which no human knows about" Social engineering has always been the week point. But humans have now taken over internet as being the most unreliable piece of equipment connected to the computer.

Scalability and performance - Its all about the customers

Todd pinged me to see how I felt about  antirez's suggession in his post titled " On the webserver scalability and speed are almost the same thing ". While I disagree with parts of the post, I can understand why he believed what he wrote. If someone were to design a state of the art scalable webserver in place of an existing service (lets say apache) which can deliver content in 50 ms, then by my definition the new webserver should continue to serve that content at 50 ms even if the number of requests handled per second by the service increases 100 times. Antirez's argument is somewhat correct that just because you can scale doesn't mean that customers will always forgive you for slower performance. But as it happens I've seen many different solutions in the last decade which ignored that concern and provided  a scalable yet slower service and still succeeded in generating enough revenue to prove that slower speed may not be a show stopper. In my opinion

GPS for tracking smaller objects can now be found in costco ?

Image
Updated: Was fascinated when I found this pallet at costco. Look closely at the words " iGPS "... I assumed that it had GPS based technology embedded in it, but realized its just another RFID enabled pallet.

CR48/ChromeOS - From browser sandbox to browser-in-a-hardware sandbox

I finally got a  Cr48  to play with. After being a linux sysadmin for the better part last decade, I tried to do what every honest sysadmin would try. To root it. I couldn't even get a bash prompt. Ctrl-Alt-T gives a shortcut to something called crosh  which is basically a limited command-set shell. I tried a series of injection based and chrome extension based attacks and was still no where closer to the dream after 2 hours. I further read that if the box ever gets compromised in a bad way, there are ways it can detect it, which will automatically trigger an OS refresh at the next bootup. Thats when I realized that I Cr48 is an an extension to the idea of browser sandbox which makes an attempt to create a secure and stable browser experience. With Cr48, even if your OS is compromised, the detection, refresh and replace is so fast that its almost like a crashed browser tab which is replaced with a fresh new one. A few weeks ago I stopped visiting one of the news websites I loved to

Scalability links for January 19th

Scalability links for January 19th: Optimizing TCP Socket Across Data Centers - Intersting observation of why bandwidth sometime hits a ceiling. Also explains why organizatinos need CDNs and local datacenters closer to the end users. Introducing AWS Elastic Beanstalk - I made a prediction just a few days ago that AWS would do something like this. This is very cool and confirms my observation as to where the industry is heading towards. Paper: Relational Cloud: A Database-as-a-Service for the Cloud - Researcher Develops Password Hacking Software for Wi-Fi Networks Using Amazon Web Services - Scary way of using the cloud. But at the same time it shows that computing power is cheaper than ever. Hadoop I/O: Sequence, Map, Set, Array, BloomMap Files - Skype's acquisition of Qik is now complete - Now who can help me remove Qik app from my Evo ? Its a terrible product for which I have no use... but came bundled with the phone and I'm stuck with it. Gang of Smart

Beanstalk, Gogrid and HBase

Top 3 News items from  scalebig The big news is that Amazon has got into PAAS big time. I predicted it only a couple of days ago ( I said they will launch within next 1 year ). With beanstalk they plan to provide containers into which users can upload code to and let AWS manage rest of the complexities of things around it. They are starting with a tomcat based container for now and have mentioned plans to build other containers. Read more about it at " All things distributed " As weird as would sound, GoGrid is building the private cloud over public infrastructure . They are doing this just to let let CIO claim that they own the servers. This allows CIOs claim to be on two boat at the same time. At some point though CIOs will have to make a call and abandon one. BTW, this is not very different from managed infrastructure, with the exception that there now exists a virtualization toolkit to manage VMs on this managed infrastructure. Hbase 0.90.0 is released . Lots of

Risks of automated stock prediction engines

A celebrety tweeted about a stock tip to his 3 million followers. By the end of the day the stock price jumped over 290%. "You can double your money right now. Just get what you can afford," Jackson tweeted about H&H; Imports, a money-losing venture out of Clearwater, Fla., that owns TV Goods, a marketing firm recently founded by Kevin Harrington. While all the online reports I read assume that his followers fell for it, I am not sold on that. I would like to know if anyone ruled out the root cause as automated systems using twitter data for stock price change prediction. Automated systems using twitter data often use follower count to compute probability of some information being true. One of my news crawlers ( scalebig.com ) actually uses the same twitter feed to rate technical posts on scalability. Knowing how widespread the use of twitter data is in different kind of automations, and because of how some of the way twitter data could behave is unpredictable, I would

How facebook ships code

Stumbled on a fascinating post about how facebook ships code . This level of detail is rare from an organization as big as this.  This is a very long piece... but here are a few lines from it to entice you to click on it. From framethink as of June 2010, the company has nearly 2000 employees, up from roughly 1100 employees 10 months ago.  Nearly doubling staff in under a year! the two largest teams are Engineering and Ops, with roughly 400-500 team members each.  Between the two they make up about 50% of the company. product manager to engineer ratio is roughly 1-to-7 or 1-to-10 all engineers go through 4 to 6 week “Boot Camp” training where they learn the Facebook system by fixing bugs and listening to lectures given by more senior/tenured engineers.  estimate 10% of each boot camp’s trainee class don’t make it and are counseled out of the organization. after boot camp, all engineers get access to live DB (comes with standard lecture about “with great power com

Its Logical - IAAS users will move to PAAS

Sysadmins love infrastructure control, and I have to say that there was a time when root access gave me a high. It wasn’t  until I moved to web operations team (and gave up my root access) that I realized that I was  more productive when I wasn’t dealing with day to day hardware and OS issues. After managing my own EC2/Rackspace instance for my blog for a few years , I came to another realization today that IAAS (infrastructure as a service) might be one of these fads which will give way to PAAS (Platform as a service). WordPress is an excellent blogging platform, and I manage  multiple instances of it  for my blogs (and one for my   wife’s blog ). I chose to run my own wordpress instance because I loved the same control which I used to have when I was a sysadmin. I not only wanted to run my own plugins, configured my own features, play with different kinds of caching features, I also wanted to choose my own linux distribution (Ubuntu ofcourse) and make it work the way I always w

Its logical - IAAS users will move to PAAS

Sysadmins love infrastructure control, and I have to say that there was a time when root access gave me a high. It wasn't  until I moved to web operations team (and gave up my root access) that I realized that I was  more productive when I wasn't dealing with day to day hardware and OS issues. After managing my own EC2/Rackspace instance for my blog for a few years , I came to another realization today that IAAS (infrastructure as a service) might be one of these fads which will give way to PAAS (Platform as a service). Wordpress is an excellent blogging platform, and I manage  multiple instances of it for my blogs (and one for my   wife's blog ). I chose to run my own wordpress instance because I loved the same control which I used to have when I was a sysadmin. I not only wanted to run my own plugins, configured my own features, play with different kinds of caching features, I also wanted to choose my own linux distribution (Ubuntu ofcourse) and make it work the way

Are you ready for IPv6 yet ?

If I say internet is running out of IPs, you might respond with "so whats new?". Whether you like it or not, this time its for real . While IPv4/8 blocks might be gone by the end of this year, it doesn't mean IPv6 trasition needs to happen right away. Fortunately, unlike the Y2K problem, we have a lot of tools and means to make this transition less painful by making it happen over an extended period of time. Most of the larger organizations have been testing IPv6 for years . And thanks to Apple , Microsoft , linux developers and other industry leaders , the latest versions of the most popular operating systems come preconfigured to work with IPv6. Whats missing, unfortunately, is the human element of this transition. Training the core network operators on IPv6 related issues isn't enough. Nor is it enough for all the softwares to support it. Every developer, engineer and users on all the 7 layers of the OSI stack has to understand it well enough to be able to troubl

Splunk : Fastest way to get web operations dashboard running

Image
This is a cross-post from my personal blog . Few weeks ago I asked a question on quora about  log aggregation . I was surprised to find that no opensource solution came close to what I wanted, but I got a lot of suggessions to try out splunk. So I did. What I wanted was an aggregation tool which collects, displays and alerts based on events logged by the various webservers across the network which could be in different datacenters. The organization where I set this up was generating about 300mb of production haproxy logs per day and something around 200mb of non-prod logs. Here is why splunk fit very well in this organization. 1) Log aggregation across multiple servers/datacenters- The organization had already solved this problem by piping haproxy logs using syslog-ng. They used a little bit of filtering to discard logs which are not interesting for splunk. Syslog-ng can be configured to use tcp instead of udp to make log delivery reliable. Splunk is capable of working as remote ag

Scalability links for January 13th

Scalability links for January 13th: Building Mobile Apps With AWS? Submit Them to the Amazon Appstore! - I wonder if this is realted to lab126 project . Big News for Cloud Files Users – Akamai is coming! - This is huge. The onslaught of features from Amazon pretty much forced these two giants to partner up. Akamai and Rackspace have both got something to fear and its the consumers who win at the end. I'm ready for faster services on the cloud. Google Megastore - 3 Billion Writes and 20 Billion Read Transactions Daily - BigQuery, meet Google Spreadsheets - URL Design — Warpspire - Very thoughtful URL designing notes. Extremely helpful if you love building new web-applicaitions. WebServius - Monetization System for Data Vendors - This reminds of of Mashery. But unlike mashery the cost of entry is very very low. Ofcourse Mashery does a lot more than give access to static data, but this is a good starting point for a lot of services which provide static data. Pi

Splunk : Fastest way to get web operations dashboard running

Image
Few weeks ago I asked a question on quora about  log aggregation . I was surprised to find no opensource solution which came close to what I wanted, but I got a lot of suggession from different people to try out splunk. So I did. What I wanted was an aggregation tool which collects, displays and alerts based on events logged by the various webservers across the network which could be in different datacenters. The organization where I set this up was generating about 300mb of production haproxy logs per day and something around 200mb of non-prod logs. Here is why splunk fit very well in this organization. 1) Log aggregation across multiple servers/datacenters- The organization had already solved this problem by piping haproxy logs using syslog-ng. They used a little bit of filtering to discard logs which are not interesting for splunk. Syslog-ng can be configured to use tcp instead of udp to make log delivery reliable. Splunk is capable of working as remote agents as well... but sending