Geolocation lookups in Linux (/Ubuntu)
Have written a short post on this before, but it seems that I've only scratched the surface 
For Ubuntu/Debian users, the APT package to install would be:
$ sudo apt-get install geoip-bin
MaxMind has (free and commercial) databases that can be queried using these command line tools in Linux. Installing the geoip-bin package installs the free version of the country database, but you don't need to stop there!
By default, the free IP-Country database is situated at /usr/share/GeoIP/GeoIP.dat. Do note that the APT package for it is NOT updated automatically, so you will need to update it yourself.
Grabbing hold of the other two free databases (they're updated monthly I think) and placing them the shared folder. IP-ASN is a nice way to quickly determine the ownership of an IP address, which you can follow up with actually looking through the WHOIS info should that be too generic. IP-City info comes with geolocation (lat-long coordinates!) info, which is very nice for plotting IP address lists on nice maps for analysis, or for the less technically inclined (or your bosses 
).
$ ls /usr/share/GeoIP/ GeoIPASNum.dat GeoIP.dat GeoLiteCity.dat
It appears that GeoIP and GeoIPASNum are queried automatically by default
$ geoiplookup 8.8.8.8 GeoIP Country Edition: US, United States GeoIP ASNum Edition: AS15169 Google Inc.
Now let's try querying for basic location information:
$ geoiplookup 8.8.8.8 -f /usr/share/GeoIP/GeoLiteCity.dat GeoIP City Edition, Rev 1: US, N/A, N/A, N/A, 38.000000, -97.000000, 0, 0
What are the MaxMind database versions currently "installed"?
$ geoiplookup 8.8.8.8 -v GeoIP Country Edition: GEO-106FREE 20120403 Build 1 Copyright (c) 2012 MaxMind Inc All Rights Reserved GeoIP ASNum Edition: GEO-117 20120402 Build 1 Copyright (c) 2012 MaxMind Inc All Rights Reserved $ geoiplookup 8.8.8.8 -f /usr/share/GeoIP/GeoLiteCity.dat -v GeoIP City Edition, Rev 1: GEO-533LITE 20120403 Build 1 Copyright (c) 2012 MaxMind Inc All Rights Reserved
If you want more verbose reporting (shows the IP address block that matched the query):
$ geoiplookup 8.8.8.8 -i GeoIP Country Edition: US, United States ipaddr: 8.8.8.8 range_by_ip: 8.7.211.0 - 8.14.223.255 network: 8.8.0.0 - 8.11.255.255 ::14 ipnum: 134744072 range_by_num: 134730496 - 135192575 network num: 134742016 - 135004159 ::14 GeoIP ASNum Edition: AS15169 Google Inc. ipaddr: 8.8.8.8 range_by_ip: 8.8.8.0 - 8.8.8.255 network: 8.8.8.0 - 8.8.8.255 ::24 ipnum: 134744072 range_by_num: 134744064 - 134744319 network num: 134744064 - 134744319 ::24 $ geoiplookup 8.8.8.8 -f /usr/share/GeoIP/GeoLiteCity.dat -i GeoIP City Edition, Rev 1: US, N/A, N/A, N/A, 38.000000, -97.000000, 0, 0 ipaddr: 8.8.8.8 range_by_ip: 8.7.228.0 - 8.8.37.255 network: 8.8.0.0 - 8.8.31.255 ::19 ipnum: 134744072 range_by_num: 134734848 - 134751743 network num: 134742016 - 134750207 ::19
Cooking all of this with a little CLI script-fu for mass lookups!
$ output=outputfile.csv; echo "ip,country" > $output; for i in $( cat /path/to/list-of-ips.txt ); do echo "$i,\"$( geoiplookup -f /usr/share/GeoIP/GeoIP.dat $i | cut -d' ' -f4-99 )\"" >> $output; done
HTH, and have fun!
Linux Login Detection Redux
Have almost forgotten how fun it is to mess around with a Linux server. Building another Linux server did indeed bring back some memories
This is another scratchpad post: little to no explanation/breakdown on the script involved (unless there's the "impetus" to elaborate in future). Feel free to ask/discuss in the comments section below though.
Any user who logs in should trigger the sending of the notification email from the server immediately, and if it wasn't an expected login, well at least you'd know it's time to trigger some incident response processes.
As an improved version of the old post on the same topic, this script similarly is to be appended to /etc/profile or the relevant ~/.bash_profile per user.
echo -e "$(hostname) shell access\n$(date)\n$(who)\n\
$(for i in $(who|cut -d"(" -f2|cut -d")" -f1|cut -d":" -f1|sort -u);
do echo -e "==========\nwhois $i"; whois $i;
echo -e "\n=====\nreverse $i"; dig -x $i;
done;)" | \
mail -s "$(hostname) alert: shell access from \
$(who|cut -d"(" -f2|cut -d")" -f1|cut -d":" -f1|tr "\n" " ")" \
'youremail@domain.com'
Changes namely are the adding of whois and reverse IP (DNS PTR) lookups for all IP addresses currently logged on via SSH, and also the use of the more readable $() Bash command substitution expansion rather than the backtick (`).
You will need to have installed the mailutils package (apt-get install mailutils), and probably a MTA like postfix or exim too.
HTH.
Edit 30 Apr 2012: small bug fix in the sequence to extract all IPs from the who command output.
Dynamic conversion of epoch timestamps in logs
In the course of your logs or text processing, you may come across certain timestamps in epoch format. Whilst there's always online resources to assist with the conversion of such timestamps, it may not be the best way if you need to keep the timestamp "secret" during then, or if you have many timestamps to convert going by the thousands, millions, etc.
Whilst there's always free tools like Splunk which is available for free to the masses (and yes, it does automatically convert epoch timestamps for you), there's always our "humble" awk. 