Sniffing HTTP

Recently I spent a bit of time studying the effects of HTTP headers on different browsers. There was this issue with IE6 caching things too aggressively… but I digress. I crafted this command line for the command line version of Ethereal (WireShark). It continuously dumps HTTP request headers, response headers, and text responses. There is a 30-line limit on all three. Here is it, mainly for my memory but maybe someone else will benefit:

tethereal -i en1 -f 'host 1.2.3.4' -R 'http' -S -V -l | \
awk '/^[HL]/ {p=30} /^[^ HL]/ {p=0} /^ / {--p} {if (p>0) print}'

Replace en1 with the network adapter you are using (ifconfig). Replace 1.2.3.4 with the IP of the destination machine. I used the awk command as a state machine to filter out unwanted output from tethereal and to impose the 30-line limit. The output looks like this:

Hypertext Transfer Protocol
    GET /style.css HTTP/1.1\r\n
        Request Method: GET
        Request URI: /style.css
        Request Version: HTTP/1.1
    Host: example.wordpress.com\r\n
    User-Agent: Mozilla/5.0 [...] Firefox/3.0\r\n
    Accept: text/css,*/*;q=0.1\r\n
    Accept-Language: en-us,en;q=0.5\r\n
    Accept-Encoding: gzip,deflate\r\n
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n
    Keep-Alive: 300\r\n
    Connection: keep-alive\r\n
    Referer: http://example.com/\r\n
    Cookie: wp_test=WP+Cookie+check\r\n
    \r\n

Hypertext Transfer Protocol
    HTTP/1.1 200 OK\r\n
        Request Version: HTTP/1.1
        Response Code: 200
    Date: Thu, 10 Jul 2008 20:37:45 GMT\r\n
    Server: LiteSpeed\r\n
    Accept-Ranges: bytes\r\n
    Connection: Keep-Alive\r\n
    Keep-Alive: timeout=5, max=100\r\n
    Cache-Control: max-age=604800\r\n
    Expires: Thu, 17 Jul 2008 20:37:45 GMT\r\n
    ETag: "461d-47e542a4-0"\r\n
    Last-Modified: Sat, 22 Mar 2008 17:32:20 GMT\r\n
    Content-Type: text/css\r\n
    Content-Length: 2400\r\n
    Content-Encoding: gzip\r\n
    Vary: Accept-Encoding\r\n
    \r\n
    Content-encoded entity body (gzip): 2400 bytes -> 17949 bytes
Line-based text data: text/css
    /*
    \tTheme Name: Example
    \tTheme URL: http://wordpress.com
    */
    [...]
Leave a comment

6 Comments

  1. That is such a great idea I can’t believe I didn’t think of that! I guess most people don’t have access to tethereal or packet-capturing apps, especially on shared hosts like wordpress.com.. Very cool scripting there… what are you doing with the data? Something for bat-cache?

    Reply
  2. Very nice idea, but i am having problems installing tethereal. It says that it needs tshark but it won’t install it. I am using debian lenny.

    Reply
  3. For what its worth ethereal is no longer the name of the sniffer… its called wireshark now.

    This is what caused Constantinos Kouloumbris the trouble. You need to install the wireshark package for your distro. Then you can use the “tshark” command with exactly the same syntax written above.

    Lovely snippet. Helped me a bunch!

    Thanks,
    -FT

    Reply
  4. Bob

     /  October 1, 2009

    Thanks, I found this was very useful. Allowed me to determine with my own eyes that when I use HTTP basic auth over http the username/password are sent in clear text but when I use it over https they don’t show up.

    Reply
  5. Here is it, mainly for my memory but maybe someone else will benefit

    How small the web is. This seriously helped me out today. Thanks! Just note that you now have to use “tshark” instead of “tethereal”

    Reply
  1. Sniffing http headers with Wireshark | Eriky.com

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,667 other followers

%d bloggers like this: