"the world's cheapest IPS" -- a friend
"cool, ngrep but better ;)" -- another friend
flowgrep is a basic IDS/IPS tool written in python as a way to help you
investigate and manage your network. it works by sniffing traffic,
reassembling TCP streams, and IP and UDP fragments into single packets, and
allowing you to "grep" through their payloads using regular expressions.
the quality of the regular expression engine is similar to Perl's. think
of it as a marriage of tcpflow, tcpkill, and ngrep.
flowgrep's usage is quite simple:
$ flowgrep.py -h
./flowgrep: TCP stream/UDP/IP payload 'grep' utility
Usage: ./flowgrep OPTIONS [FILTER]
where OPTIONS are any of the following:
-a [pattern] match any stream with pattern
-c [pattern] match client stream with pattern
-D [num] distance score for libdistance-based match
-d [device] input device
-E [name] string distance algorithm to use
(one of: levenshtein, damerau, hamming, jaccard)
-e [string] string to compare against for distance-based matches
-F [file] obtain server patterns from file, one per line
-f [file] obtain client patterns from file, one per line
-i case insensitive match
-k kill matched stream (TCP only)
-l [dir] log matched flows relative to dir (default: .)
-r [file] input file (in pcap(3) format)
-s [pattern] match server stream with pattern
-u [username] run as username (default: nobody)
-V print version information and exit
-v select non-matching input
-x print logged filenames (for use with xargs(1))
[FILTER] pcap(3) filter expression
UDP and IP payloads will test any pattern (no stream to test).
you can specify any string or regular expression to look for and optionally
set the stream to log or be killed (-l or -k, respectively). note that the
kill option only works on TCP streams. you can look for expressions in
the client or the server (or both) streams. note that for UDP and IP, both
expressions are tested since no "stream" is established. you need
root permissions to use this effectively since you are sniffing the
logged streams are written out as two streams for TCP streams or a single
file for UDP and IP payloads that match. the payload for any matched stream
is written out. files are named 'time-source-sport-dest-dport-proto' in the
local directory. examples:
-rw-r--r-- 1 nobody jose 40 Oct 20 21:09 1098320995-192.168.10.84-22-192.168.10.190-28318-tcp
-rw-r--r-- 1 nobody jose 7 Oct 20 21:09 1098320995-192.168.10.190-28318-192.168.10.84-22-tcp
note that the files may contain binary data, or malicous data, so handle
them with care. non-UDP and non-TCP payloads are not written using a
port number, and the protocol is "protoN", where "N" is the number.
USES OF FLOWGREP
you can do measurements of particular traffic, you can build a very cheap
IPS device (ie for mail-based worm infestations), disrupt spammers, or you
can just sniff on your coworkers.
- be a jerk and kill everyone's web sessions, flowgrep -s "HTTP/1.1 200
- be a manager and detect and stop SSH on unauthorized ports, flowgrep
-i -a "ssh-" -k tcp not port 22
- kill HTTP tunnels by stopping non-HTTP traffic on port 80,
flowgrep -v -i -c "^GET|^POST|^PUT|^HEAD" -k tcp port 80.
"By now everyone should appreciate just how powerful and useful Flowgrep
can be." Richard Bejtlich on his blog on 28 Jan, 2005 (see below).
jose nazario <firstname.lastname@example.org>
- 20 october, 2004. version 0.5
- 20 october, 2004. version 0.6
added a configurable log directory, timestamps to filenames, and compiled
regexp statements (to improve performance).
- 21 october, 2004. version 0.6.1
fixed a UDP callback typo
- 22 october, 2004. version 0.6.2
typo in getopt
- 27 october, 2004. version 0.7
added Makefile, setup.py. installation is now a snap ... changed how -v
operates, now it acts a lot more like grep (ie negate the match). added a
manpage. fixed matching the end or beginning of line anchors in regexps.
- 1 november, 2004. version 0.7.1
add support for specific devices (flowgrep -d) and input files (-r; broken
due to pynids 0.1 limitation, patch sent to author). install as flowgrep,
- 4 november, 2004. version 0.7.2
use the right method to call nids_killtcp() from pynids ...
- 7 november, 2004. version 0.7.3
add -F/-f to load patterns from a file (like grep does), add -V (like grep),
add -u (like snort), allow multiple -a/-c/-s declarations, add -x (for
combinations with xargs).
- 12 may, 2005. version 0.8a
typo fix in manpage (thank you laurent)
- 10 sept, 2005. version 0.9: flowgrep 0.9
add preliminary libdistance support, requested by l. sheng at hitb. fix
a bug in matching TCP streams with -v.
REQUIREMENTS AND INSTALLATION
- implement -A, -B, -C
- find, fix bugs
THINGS THAT REFERENCE FLOWGREP