Scalable logging using Syslog
Syslog is a commonly used transport mechanism for system logs. But people sometimes forget it could be used for a lot of other purposes as well. Take, for example, the interesting challenge of aggregating web server logs from 100 different servers into one server and then figuring out how to merge them. If you have built your own tool to do this, you would have figured out by now how expensive it is to poll all the servers and how out-of-date these logs could get by the time you process it. If you are not inserting them into some kind of datastore which sorts the rows by timestamp, you now also have to take up the challenge of building merge-sort script. There is nothing which stops applications from using syslog as well. If your apps are in Java, you should try out Syslog appender for log4j [ Ref 1 ] [ Ref 2 ]. Not only do you get central logging, you also get get to see real-time “tail -f†of events as they happen in a merged file. If there are issues anywhere in your netwo