How to tail a log with live server clock

At work I have to watch my server’s error log. Good old tail -F /path/to/log runs over SSH all day, every day. Each entry is timestamped. Sometimes those timestamps give me trouble. The age of a timestamp is found by subtracting it from the current time. If I need to know whether the last error occurred 1 minute ago or 61 minutes ago I have to contend with:

  • timestamps in UTC vs. local clock in CST
  • timestamps in 24h vs. local clock in 12h
  • terminal window on second screen vs. clock on main screen
  • daylight savings vs. non-crazy time
  • being at home vs. traveling in other time zones

Finding the age of a timestamp should be simpler than that! So I put a real-time UTC clock in the most convenient place I could find: on the blank line where the next log entry will be written. It refreshes itself every second. Log entries overwrite it and then it reappears again on the new blank line.

The clock on the next line refreshes itself.
The clock on the next line refreshes itself.

Now I can find the age of the last entry without thinking about time zones or daylight savings, without turning my head to check the clock, and without even holding any numbers in my head. Eyeball math. Success!

ssh $1 $(cat <<COMMAND
    while true; do
        echo -ne \`date +"[%d-%b-%Y %H:%M:%S UTC]"\` "\r";
        sleep 1;
    done &
    tail -Fn`tput lines` $2

If you save this script as st in your path and chmod +x it you can invoke it with st /var/log/file (provided you can ssh and read the file).

I used a few small tricks to compose this solution.

  • the while loop runs in the background (done &)
  • echo -n prints its arguments without a line feed
  • echo -e interprets escape sequences, converting "\r" into a carriage return
  • bonus: `tput lines` passes the local terminal height to tail
  • the final echo gives your prompt a fresh line when you exit with ^C

After SSH connects, the loop starts sending [timestamp]\r to the terminal every second. The carriage return moves the insertion point back to the start of the line without moving it down to the next line. That explains the location of the highlighted character in the image above.

Then tail fills your terminal window with log entries and continues printing new entries as they appear, overwriting the clock. No space wasted, no information lost.


  • log entries must be timestamped and must end in a newline
  • date format is up to you
  • n`tput lines` is optional
  • the clock might draw over log data

That last point is theoretical. It hasn’t been an issue in the two months since I started using this. I also tried to force the clock to overwrite the tail output under abnormal conditions. The only way I could do it was by manually appending to the file without a trailing newline. I guess it depends on the size of the log entries and how they are written. Either way, it’s only terminal output. The log file itself is unmodified.

The script I posted above isn’t exactly what I have been using. It was modified for clarity and generality. I added positional parameters and $(cat <<heredoc) to get clean newlines without having to escape too much. Since I only use this utility to tail one file on one host, the original looks more like this one-liner:

ssh 'while true; do echo -ne `date +"[%d-%b-%Y %H:%M:%S UTC]"`"\r"; sleep 1; done & tail -Fn'`tput lines`' /var/log/file'; echo

Automatically open remote files in local emacs

I prefer to edit text locally in emacs. Most of the files I edit reside on remote servers so I use TRAMP to open remote files locally. What kills me is using emacs remotely via terminal when a shell command invokes $EDITOR (e.g. svn commit). With my new setup, the default editor on the remote machine is my local emacs. I love this.

First, I configure SSH to forward a remote port to my machine. This means that whenever the remote machine tries to connect to itself on that port (localhost:9999) it actually connects to port 9999 on my local OSX machine. I like to keep these details in my ssh_config file (my local ~/.ssh/config):

Host des
User wpdev
ControlMaster auto
ControlPath ~/.ssh/des.sock
RemoteForward 9999 localhost:9999

(I use abbreviated hostnames to save keystrokes. There is a matching entry in my hosts file.)

Second, I configure my local emacs to start the server and copy the server file to the remote host. The server file tells emacsclient how to connect to the server. Adding this to emacs-startup-hook adds a few seconds to my emacs startup time but I rarely start emacs more than once in a day so that’s fine. This is in my local ~/.emacs:

(setq server-use-tcp t
      server-port    9999)
(defun server-start-and-copy ()
  (copy-file "~/.emacs.d/server/server" "/des:.emacs.d/server/server" t))
(add-hook 'emacs-startup-hook 'server-start-and-copy)

Third, I create a bash script on the remote host which calls emacsclient with the necessary TRAMP path prefixed to its arguments. (If you try running emacsclient remotely without the TRAMP path you’ll get an empty emacs buffer.) Here is the script I put in remote ~/bin/ec and then chmod +x:


for p in "$@"; do
    if [ "$p" == "-n" ]; then
        params+=( "$p" )
    elif [ "${p:0:1}" == "+" ]; then
        params+=( "$p" )
        params+=( "/ssh:des:"$(readlink -f $p) )
emacsclient "${params[@]}"

Finally, I set up $EDITOR on the remote machine. I also add my bin directory to $PATH so I can invoke ec. This is in my remote ~/.bashrc:

export PATH=~/bin:$PATH
export EDITOR=~/bin/ec

That’s it! More elegant solutions are possible but my new tool is sufficiently sharp and I have work to do!

Python: ISO-8601 year/week from year/month/day

I needed to convert dates to ISO-8601 calendar week and year in a bash script. This is what I came up with:

#!/usr/bin/env python

import optparse
import datetime

parser = optparse.OptionParser()
_, args = parser.parse_args()

y, w, _ = datetime.datetime.strptime(args[0], "%Y/%m/%d").isocalendar()
print "%d/%02d" % (y, w)

It takes input from the command line as YYYY/MM/DD and outputs YYYY/WW.

Copyright Automattic 2012. Public domain where recognized; otherwise Apache 2.0.

svn merge -c

How many times have you frantically googled “svn merge revert” to remember how to undo a recent commit? If you answered “too many” then you were probably confused by the syntax of svn merge -rARG1:ARG2. After this post you’ll never have that problem again because you can easily memorize svn merge -c-ARG instead.

# to undo changeset 1234 in the working copy:
svn merge -c-1234 files

The c is for “change” and the - means “in reverse”. When you read that command to yourself, say “svn merge change in reverse 1234”. That is sensible enough that you can remember it forever, even in an emergency. Right?

To apply several changes at once, use several -c- options:

# to undo two changesets at once:
svn merge -c-1234 -c-1220 files

To re-apply the changes in your working copy for debugging, use -c:

# to re-apply the changes you undid:
svn merge -c1220 -c1234 files

May this knowledge shave precious seconds off your next svn emergency!

Programming is not Algebra

A friend discovered a small interest in learning to program computers. She tried an online course but she didn’t get far before losing interest. She may have had other reasons for giving it up but by watching her and asking questions I did locate one significant hurdle: she was trying to read code through the lens of algebra.

I have seen this before. A student encounters a word, symbol, or pattern they recognize from some other context. Believing they have sufficient information to understand it in the new context, they push ahead. Soon their mental model fills with contradictions and the entire subject becomes intractable. This state of affairs causes people to quit their studies.

I’ve been there. Fortunately I learned the solution:

  1. Find what word or symbol was misunderstood.
  2. Learn the correct meaning.
  3. Re-study the material from the first occurrence.

The hardest part is completing the first step. This is especially true of students who already know a great deal and who are confident in their knowledge. Preconceptions hide themselves well but they surrender easily once found. Most people will be eager to complete the remaining steps after they have identified a real misunderstanding.

In this article I will expose some common preconceptions which can cause students of programming to quit. Teachers of beginner programming classes and writers of books would do well to treat these handicaps before setting students to code.
Continue reading Programming is not Algebra

Atomically update serialized PHP arrays in MySQL

Okay, okay, it’s hard to find a use case for this when it’s so obvious that the correct way to handle one-to-many is with JOIN. But if you’re already committed to your schema and you decide you need to append serialized PHP data to a row atomically, you can cons serialized values with this query:

  serialized = "i:1;"
    serialized = CONCAT(

After you have performed this three times with the serialized values 1, 2, and 3, the row contains this:


After unserializing, deconstruct it with this function:

function decons($list) {
    $res = array();
    while ( $list != array() ) {
        if ( $list[0] === 'cons' ) {
            array_unshift( $res, $list[1] );
            $list = $list[2];
        } else {
            array_unshift( $res, $list );
    return $res;

The result:

array(1, 2, 3)

I haven’t actually used it (probably never will) but you are welcome to try this at home!

Proving that this is stupid is left as an exercise for the reader.