Visualizing Data (using Processing)
Visualizing Data, by Ben Fry (O'Reilly)
[Amazon] [Google Books] [O'Reilly] [Google Search]
One of my interests has always been in data visualization (makes data more understandable, and is one step towards easier interaction with it). Chanced upon this book at the library today, certainly one thing I'd like to look into in more detail at a later point in time.
Why this book caught my interest was the fact that there was another book on such a topic in itself. Other than Applied Security Visualization by Raffael Marty, I've yet to chance upon anything else.
A quick browse of the book showed that it's very possible to use Processing (yet another good reason to take up this book: simple programming!) to implement many of the data visualization concepts. Though many people would say that this is "raw" and "slow" as compared to having a tool to do this simply and quickly, I'd say that doing it this way would certainly give the user a great understanding of the data visualization process itself. Furthermore, who's to say that Processing's not the tool itself! 
Also, the author has helpfully made the source code examples available online at his blog too.
Will keep this book in mind to look at later. Have other books to go through first... :}
Profiling client internet connections
Some more fun with p0f and Splunk...Now with profiling of client internet connections!
Setup of the p0f and logging is the same as in the OS Profiling post.
The Splunk search string has been extended to extract the source's internet link as a field too (go for the portion in bold for the field extracting commands):
| file /home/path/to/p0f.log | rex field=_raw "> (?<srcip>[^:]+):(?<srcport>[^ ]+) – (?<srcos>.+?) \(" | rex field=_raw "-> (?<dstip>[^:]+):(?<dstport>[^ ]+) " | rex field=_raw "link: (?<srclink>.*)\)$" | regex srclink!="(unspecified|unknown)" | top limit=0 srclink
The fields that I extract with this:
- srcip -> source IP
- srcport -> source TCP port
- srcos -> source's OS (woot!)
- dstip -> destination IP (which is my machine's)
- dstport -> the destination port which the TCP connection was initiated to
- srclink -> the internet link of the source machine
After filtering out the "unspecified" and "unknown" links, the list of the detected links are as follows:
"ethernet/modem" points to mostly cable connections. There're some interesting entries in the list though, like vtun, pppoe, Google/AOL, IPv6/IPIP (early adopters? haha). Don't have any idea on what's IPSec/GRE, or vLAN here in this context though.
Just for the heck of it, here's the chart for this table, generated from the reports link in Splunk.
I like the charts, because they allow some interaction with the charts for simple datasets, but I digress
Visualizing sshd brute-force attempts (part 2)
It's always better to Read The Fine Manual (or run perl afterglow.pl -h for the more updated helpfile)...though it's not really that well documented Afterglow allows for two column inputs, rather than us having to do weird tricks to make them 3-column.
(Note to self: get the raw data with fields in the order that you want where possible/faster, rather than pumping it through sed. Makes for good practice though.)
Using the csv file containing userids (visualized in yellow) and IPs (visualized in green) over the past few months from Splunk, here're the results of some of the experiments.
Oh, for the Windows users, you can use type instead of cat 
First test using GraphViz's neato to layout:
perl afterglow.pl -b 1 -i <infile> -c color.properties -t | neato -Tgif -o output.gif
Huge, but better visualized with -e 5 option (Resulting image for that is too huge to upload though ). Note the single IP in the middle (the yellow explosion) that had been trying a LOT of userids to date.
Second test using fdp:
perl afterglow.pl -b 1 -i <infile> -c color.properties -t | fdp -Tgif -o output.gif
fdp doesn't seem to be well suited for this
Third test using sfdp:
No command here, you should have noticed the pattern from the first two...
_even_ less suited for this type of data...
Last test using twopi:
According to the GraphViz site, twopi's more suited for visualizing stuff like telecommunications flows.
twopi