Collaborative Network Forensics

August 23, 2009 by kowsik

If you’ve dealt with really large packet captures, you’ve probably tried to break things apart into smaller chunks just so you can figure out what’s actually in there. There are lots of command line tools out there that already do this. So it started out as an experiment to see if there’s a better, interactive, visual way to explore large pcaps and rapidly hone in on what you are looking for. With the recent release of large datasets from ITOC the need for this just became a whole lot more critical.

As a week long project, we took all these pcaps and the ones from the Shmoo group, indexed them and put them up at pcapr. We used Wireshark, CouchDB and jQuery (remember my last blog on JS3?) as the underlying technologies to pull this off. We indexed 15.0 GBytes of pcaps with a total of 26.3 million packets which makes this the largest collection of indexed pcaps online. With full-text search, rapid extraction of packet slices, index preview and instantaneous access to any packet in the dataset, this should hopefully make large pcap forensics a painless process.

When you are exploring packets on this large scale, it’s always nice to have other users mark certain packets that are interesting, add insightful (or not) comments on packets so the rest of the community can benefit from those. To make this a fun process, we’ve thrown in HN/Twitter style one-liners that you can attach to packets and saved searches. We call this Collaborative Network Forensics, ‘cos that’s what it really is.

So head on over to pcapr and let us know what you think! While you are at it, maybe you can find out who actually captured the flag?

Categories: Announcements, CouchDB, jQuery, pcapr
403 Comments
Bookmark and Share