Sniffing HTTP

Recently I spent a bit of time studying the effects of HTTP headers on different browsers. There was this issue with IE6 caching things too aggressively… but I digress. I crafted this command line for the command line version of Ethereal (WireShark). It continuously dumps HTTP request headers, response headers, and text responses. There is a 30-line limit on all three. Here is it, mainly for my memory but maybe someone else will benefit:

tethereal -i en1 -f 'host' -R 'http' -S -V -l | \
awk '/^[HL]/ {p=30} /^[^ HL]/ {p=0} /^ / {--p} {if (p>0) print}'

Replace en1 with the network adapter you are using (ifconfig). Replace with the IP of the destination machine. I used the awk command as a state machine to filter out unwanted output from tethereal and to impose the 30-line limit. The output looks like this:

Hypertext Transfer Protocol
    GET /style.css HTTP/1.1\r\n
        Request Method: GET
        Request URI: /style.css
        Request Version: HTTP/1.1
    User-Agent: Mozilla/5.0 [...] Firefox/3.0\r\n
    Accept: text/css,*/*;q=0.1\r\n
    Accept-Language: en-us,en;q=0.5\r\n
    Accept-Encoding: gzip,deflate\r\n
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n
    Keep-Alive: 300\r\n
    Connection: keep-alive\r\n
    Cookie: wp_test=WP+Cookie+check\r\n

Hypertext Transfer Protocol
    HTTP/1.1 200 OK\r\n
        Request Version: HTTP/1.1
        Response Code: 200
    Date: Thu, 10 Jul 2008 20:37:45 GMT\r\n
    Server: LiteSpeed\r\n
    Accept-Ranges: bytes\r\n
    Connection: Keep-Alive\r\n
    Keep-Alive: timeout=5, max=100\r\n
    Cache-Control: max-age=604800\r\n
    Expires: Thu, 17 Jul 2008 20:37:45 GMT\r\n
    ETag: "461d-47e542a4-0"\r\n
    Last-Modified: Sat, 22 Mar 2008 17:32:20 GMT\r\n
    Content-Type: text/css\r\n
    Content-Length: 2400\r\n
    Content-Encoding: gzip\r\n
    Vary: Accept-Encoding\r\n
    Content-encoded entity body (gzip): 2400 bytes -> 17949 bytes
Line-based text data: text/css
    \tTheme Name: Example
    \tTheme URL:

Published by

Andy Skelton

CTO @ Lunar Wireless

6 thoughts on “Sniffing HTTP”

  1. That is such a great idea I can’t believe I didn’t think of that! I guess most people don’t have access to tethereal or packet-capturing apps, especially on shared hosts like Very cool scripting there… what are you doing with the data? Something for bat-cache?

  2. For what its worth ethereal is no longer the name of the sniffer… its called wireshark now.

    This is what caused Constantinos Kouloumbris the trouble. You need to install the wireshark package for your distro. Then you can use the “tshark” command with exactly the same syntax written above.

    Lovely snippet. Helped me a bunch!


  3. Thanks, I found this was very useful. Allowed me to determine with my own eyes that when I use HTTP basic auth over http the username/password are sent in clear text but when I use it over https they don’t show up.

  4. Here is it, mainly for my memory but maybe someone else will benefit

    How small the web is. This seriously helped me out today. Thanks! Just note that you now have to use “tshark” instead of “tethereal”

Comments are closed.