July 30, 2006

Hybrid drives

Hybrid cars solved the problems associated with electric cars and fuel guzzling engines. By bringing both of the technologies together, Hybrid cars can function on gasoline and still save costs by switching to an electric engine when possible.

A similar problem in computing industry is forcing storage manufacturers to work on a new kind of hybrid storage device called a Hybrid Drive. This device is a result of combination of the technologies behind regular disk based drive and the faster USB drives on your keychains. This combination provides it with high speed data access and cheap-per-byte pricing in the same storage device.

This concept isn't new, and if you have worked with storage devices you will remember that most high end RAID devices already have an internal cache which does something similar. Infact most Operating systems, including Windows, Linux has Solaris have builtin file cache too. But most of these devices don't use non-volatile Solid state (flash) which forces the cache to be destroyed everytime the Operating system is restarted. Solid state cache within the Harddrives can not only survive reboots (if non-volatile memory is used), it can also reduce the dependency on third-party caching software and hardwares which can introduce its own set of problems.

One thing to note is that though overall i/o speed will improve, the Solid state storage within HDDs will probably never completely replace in-memory(RAM) cache.

Though the technology behind this has existed for a while with a few very expensive implementations, its not until now, due to dropping solid state prices, that we might have a real chance at seeing this in action inside our home computers.
References

July 28, 2006

Sysadmin Day

Pat yourselves on your back for fixing all those servers,
- doing backup,recovery and user creation.
Pat yourselves for saying no to root and yes to sudo,
- for writing ACLs and scripting voodoo...

Pat again for waking at 2am
- just to put your cellphone on charge.
..for dealing with people
- who wanted everything a day past

Pat again for reading 650 mails a day.
- for blocking SYNFIN floods on ur network
..for carrying those secure-ids
- even while you are not at work.

When you are done patting... please stop by a bar
- pick your pagers and throw away..
'cause you all need a break once in a while
- atleast on the feaking System Admin Day !!

July 24, 2006

Nutch Distributed file system

Nutch is a very interesting java based crawler and search engine based off the lucene project. The part which captivated me, however, was this component called Distributed File system which was built to support the Nutch's quest for all the pages on internet.

July 19, 2006

Over 250 Google Wi-Fi Access points in mountain view

Google's plan for giving out wifi access to everyone in mountain view is old news. It just started rolling out. Here is a map of all the Access points in mountain view plotted on the map. Based on my initial analysis it has about 269 Access points all over Mountain View.

Google Sitemaps and DMOZ inaccuracies

If you run a website, you might have heard of Google Sitemaps and DMOZ already. What you didn't know probably is that Google Sitemaps can now learn from DMOZ if your site is listed on DMOZ.

The problem Google and other search engine face is that though they can crawl your site, they don't really know how to describe your site to an search engine user. Apart from looking at your description Meta tag they also look at various other sources of information including DMOZ database to find the best way to describe your site. Though in most cases databases like DMOZ can acurately describe  the website, its not always the case, and letting search bots like that of google know that using  a meta tag can be very helpful.

July 16, 2006

Plotting Hosts/IP Addresses on the google map

I have setup a new IP Address mapping tool on huntip today which allows anyone to plot multiple ip addresses on the map. Here is the the quick API for this map

API

  • Method: POST/GET

  • Parameters: ips ( comma delimited list of IP addresses or hostnames. For example 10.10.10.1,10.10.10.2,10.10.10.3)

  • Parametsrs: ips ( You can add a comment for each IP by using : as delimiter . For example www.hotmail.com:hotmail server,www.google.com: Google servers,202.54.15.1:VLSNL server in india)

  • Parameters: showinput (1= default, 0=dont show input box, 2 = don't show menus either)

  • Restrictions: Maximum of 100 IP addresses at any given time.


Notes

  • Accuracy: The version of MaxMind database I'm using gives accuracy of around 20 miles


Examples

July 09, 2006

Internet Health monitoring Reports

I was looking for worldwide internet health statistics and found some interesting links.

General Connectivity Reports

BGP and DNS Reports


Where is my root dns server ?

I'm sure you have heard that there are 13 root servers in the world. This cache file (root hint) provided by internic/IANA http://www.internic.net/zones/named.root should confirm that. So how does these 13 servers brave a DDOS attack.

Aparently 6 of the 13 root servers are mirrored using Anycast routing to loadbalance between multiple servers. The F Root server itself has about 37 mirrors in the world. Anycast routing is implemented using BGP by simultaneously announcing the same destination IP range from many different places on the internet. So even though an IP might be registered for a location here in US, if someone announces that a route to the same IP block in Tokyo, hosts in or around that country will try to pick the cheapest route to get to a DNS server. DDOS attacks against root dns servers have happened in the past, and will continue to happen in future. Anycast routing is probably why these "13" DNS servers are still alive today.

The next question some might ask is why we can't have more than 13 IP addresses for root servers... or why can't we just have a large root hint (cache). The answer is simple. For DNS to work using UDP protocol (which is stateless) there is a recommended upper limit on the size of a DNS packet (512 bytes). TCP/IP, which is much more expensive because of its overhead, is the recommended protocol for queries/replies beyond that packet size. The root server administrators understand this very well (who else will know better) and decided to restrict the total number of servers to 13 which can easily be embedded as a list of IPs inside a 512 byte UDP packet if required.

Here is a map of the 13 registered root servers on the global map. A complete list of root servers are listed at http://www.root-servers.org/.

July 07, 2006

How many root dns server do we have ?

Haven't you heard that we have 13 root dns servers in the world ? This map on huntip.com was created based on the root file hint provided by internic/IANA http://www.iana.org/popular.htm, http://www.internic.net/zones/named.root which listed the 13 IP addresses. The part which I later found out is that 6 of these IP addresses use Anycast addressing (different from multicast, broadcast and unicast).


Anycast routing is implemented using BGP by simultaneously announcing the same destination IP range from many different places on the internet. So even though an IP might be registered for a location here in US, if someone announces that a route to the same IP block in Tokyo, hosts in or around that country will try to pick the cheapest route to get to a DNS server. The F Root server itself has about 37 mirrors in the world. So, we are very well protected against the DOS attack.

Some might ask why we can't have more than 13 IP addresses for root servers. For DNS to work using UDP protocol (which is stateless) there is a recommended upperlimit on the size of a DNS packet (512 bytes). TCP/IP is the recommended protocol for queries/replies beyond that packet size. The root server administrators understand this very well (who else will know better) and decided to restrict the total number of servers to 13 which can easily be embedded as a list of IPs inside a 512 byte UDP packet if required.

A complete list of root servers are listed at http://www.root-servers.org/. The graphical coordinates for non-anycast IP addresses are accurate to within 50 miles of the actual server.

July 04, 2006

HuntIP.com goes live

Hunt IP is a collection of systems admin tools and links to looks which can help in investigating network, dns and Email problems.
HuntIP.com