Dynamic conversion of epoch timestamps in logs
In the course of your logs or text processing, you may come across certain timestamps in epoch format. Whilst there's always online resources to assist with the conversion of such timestamps, it may not be the best way if you need to keep the timestamp "secret" during then, or if you have many timestamps to convert going by the thousands, millions, etc.
Whilst there's always free tools like Splunk which is available for free to the masses (and yes, it does automatically convert epoch timestamps for you), there's always our "humble" awk.
The linux awk command has the ability to invoke other commands as part of its computation. The date command can be used to convert epoch times to local times. Putting both together would allow us to do just what we need here!
First some examples with the date command:
$ date -d @1280921130.313 Wed Aug 4 19:25:30 SGT 2010
Or should we want to get the dates only:
$ date -d @1280921130.313 +%D 08/04/10
Now, making use of awk to convert only one epoch timestamp:
$ echo -n "1280921130.313" | \
awk '{"date -d @"$1" +%D" | getline myvariable; print myvariable}'
08/04/10
The important part to note is that we must enclose the "external" command in quotes (we use the unquoted $1 variable to pass the epoch timestamp from awk), and that we pipe the output of that command to the getline directive in awk. getline by itself would replace the $0 variable in awk when referencing it subsequently, whereas specifying a variable ("myvariable" in this example) would keep the $0 variable as it is, allowing you to use the variable to reference the output of the external command.
Final example showing how logs preprocessing using these commands might look like:
$ cat sample.log 1280921130.313 logentry1 1280921131.313 logentry2 1280921132.313 logentry3 1280921133.313 logentry4$ cat sample.log | \ awk '{"date -d @"$1 | getline myvariable2; print myvariable2 "\t" $0}' Wed Aug 4 19:25:30 SGT 2010 1280921130.313 logentry1 Wed Aug 4 19:25:31 SGT 2010 1280921131.313 logentry2 Wed Aug 4 19:25:32 SGT 2010 1280921132.313 logentry3 Wed Aug 4 19:25:33 SGT 2010 1280921133.313 logentry4
Have fun!