Doing geolocation lookups in command line
Did you know that it's possible to do your own geoip lookups from the linux command line?
You need to install the geoip-bin package in Ubuntu/Debian's APT system:
sudo apt-get install geoip-bin
Then after which, lookups can be done as simply as:
$ geoiplookup 8.8.8.8 GeoIP Country Edition: US, United States
Note that the lookups are based on the GeoLite Country database. For more detailed geoip lookups you will need to buy the better databases.
Converting IDNs in Ubuntu
With the start of Internationalized domain names (IDNs) it sparked my interest since it requires conversion to punycode in order to continue working with existing DNS systems/applications, which work with ASCII.
Taking a search through Ubuntu's APT system, to see whether any IDN related tools are available...
$ apt-cache search punycode libidn11 - GNU Libidn library, implementation of IETF IDN specifications libidn11-dev - Development files for GNU Libidn, an IDN library idn - Command line and Emacs interface to GNU Libidn libidn11-java - Java port of the GNU Libidn library, an IDN implementation libidna-punycode-perl - encodes Unicode string in Punycode
There's the idn package! Which allows encoding of IDNs in punycode in the command line...
Doing an install...
$ sudo apt-get install idn -y
And trying it out!
$ idn правительство.рф libidn 1.15 Copyright 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Simon Josefsson. GNU Libidn comes with NO WARRANTY, to the extent permitted by law. You may redistribute copies of GNU Libidn under the terms of the GNU Lesser General Public License. For more information about these matters, see the file named COPYING.LIB. xn--80aealotwbjpid2k.xn--p1ai
And resolving the domain...
$ nslookup xn--80aealotwbjpid2k.xn--p1ai Non-authoritative answer: Name: xn--80aealotwbjpid2k.xn--p1ai Address: 95.173.135.62
Note that resolving the domain directly results in rubbish!
$ nslookup правительство.рф Non-authoritative answer: Name: \208\191\209\128\208\176\208\178\208\184\209\130\208\181\208\187\209\140\209\129\209\130\208\178\208\190.\209\128\209\132 Address: 67.215.65.132
So, basically from this we understand that applications will need to use the punycode encoded version of the IDN, NOT the original IDN, when resolving. And there're tools out there already can do that for us.
Since Ubuntu has these packages, Debian would also have the corresponding packages available too.
OS Profiling
Trying out p0f along with Splunk..
p0f allows you to determine the OS of the remote machine based on the TCP fields characteristics. It can also tell whether the machine is behind a firewall, what kind of internet connection it is running from...pretty useful for information junkies like me 
Here's what I did:
./p0f -t -u MyUseridHere -i eth0 'src not MyIPAddressHere' | tee -a p0f.log
Runs p0f, logging with actual timestamps (-t), chroot and setuid to MyUserIdHere (-u), listening on eth0 (-i), and filtering out packets for connections initiated from my machine itself (since I'm not interested in profiling my own machine).
tee is a (really nifty!) linux command. What it does is to "split" the input (stdin) to two parts: stdout and the file specified. The -a option tells it to append to the file instead of overwriting it.
Using this, p0f outputs logs like this one:
<Sat Jul 3 07:03:56 2010> 175.40.12.47:1095 - Windows 2000 SP2+, XP SP1+ (seldom 98)
-> 74.207.229.183:80 (distance 12, link: sometimes DSL (2))
One of the Splunk queries that I poked around with:
| file /path/to/p0f.log | rex field=_raw "> (?<srcip>[^:]+):(?<srcport>[^ ]+) - (?<srcos>.+?) \(" | rex field=_raw "-> (?<dstip>[^:]+):(?<dstport>[^ ]+) " | regex srcos!="UNKNOWN" | top limit=0 srcos
This query extracts out the source and destination IP and port, and the source OS. Then after filtering out the OS tagged with UNKNOWN, the remaining entries are ranked...
The resulting chart, of not much real interest by itself, just shows that other than that the connections are predominantly from linux machines (hurhur), and there's a connection from a really old Netware machine (5 was released in Oct 1998!).
