cowboy me, 2.0: jose nazario beauty and the street

graphical and passive browsing history

one of the many things i've wanted my browser to do is to build a tree or directed graph based model of my browsing history. since i have yet to see a widget do this, i whipped up a small tool that does this: http graph. all of this data can be gathered passively, and it builds the graph based solely on the HTTP request header sent by the client:
GET /favicon.ico HTTP/1.1
User-Agent: Mozilla/5.0 (X11; U; OpenBSD i386; en-US; rv:1.5a) 
            Gecko/20031030 Mozilla Firebird/0.6.1
Accept: image/png,image/jpeg,image/gif;q=0.2;q=0.1
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7;q=0.7
Keep-Alive: 300
Connection: keep-alive
from this you can get the URL (Host + GET string) and the referrer, meaning you can infer with high confidence how the user surfed there. this still leaves you with some islands, but you can now graphically see website structures and information.

the graph output format is in dot/neato format (via graphviz). while not always perfect, it's standard, decent and should work for most people. it writes the output to a file, /tmp/ a Makefile line like this to produce a PS file output would look like this:
        neato -Tps -o -Nstyle=filled -Nshape=box \
          -Ncolor=grey -Elen=3 -Ew=2
those weights and lengths balance between graph size and readability.

http-graph uses pynids, which is obviously built on top of python and libnids.



| archives

Last modified: Friday, Oct 01, 2004 @ 06:44am
Weblog Commenting and Trackback by

Your Ad Here

copyright © 2002-2015 jose nazario, all rights reserved.