Nutch Distributed file system
Nutch is a very interesting java based crawler and search engine based off the lucene project . The part which captivated me, however, was this component called Distributed File system which was built to support the Nutch's quest for all the pages on internet.