We launched pcapr over a year ago now with just a few of us working part time to build and manage the site. pcapr is powered by CouchDB, a NoSQL database written in Erlang with JavaScript as the primary query language. Frankly, this has been a disaster. We are planning on rebuilding the site with Java, Hibernate and MySQL for a number of reasons.
Archive for pcapr
Network forensics in IRB: xtractr Ruby gem
What started off as a way to fully unit test xtractr, turned out to be a Gem, literally. First xtractr, then nuggets and now a gem. You follow? Seriously though, we are happy to announce a Ruby gem for xtractr which takes all the goodness of Ruby and interacts RESTfully with xtractr for oh-so-fun packet mining and troubleshooting all from within IRB.
Using Map/Reduce for Network Forensics and Troubleshooting
We launched xtractr earlier this week for network forensics, troubleshooting and handling support escalations involving large packet captures. Just so you know xtractr is a 4-tier app (more on that below) that combines the best of Web 2.0 with looking at packets in new light. Looking beyond the “unleash the power of packets” message, I wanted to write about what’s under the hood a little bit and how we are using CouchDB-style of Map/Reduce for uncovering all sorts of information inside large packet captures.
Evolution of Testing
So I was up early this morning and counting packets didn’t help. I was thinking of what we do here at Mu and how testing requirements have dramatically changed over the last few years. This blog is an ode (well it doesn’t rhyme) to the most awesomest testing product created by the Mu team.
Announcing xtractr – unleash the power of packets
At Mu, we deal with pcaps every day. We love Wireshark. We decode packets, work with protocols, auto generate test cases (functional to fuzz) from pcaps by analyzing the contents and just have incredible amounts of fun solving major problems for our customers. Yet when it comes to replicating field issues, most of our customers struggle with large pcaps and try to get a bird’s eye view of what’s in it to pinpoint the conversation or packet that triggered a bug. This takes hours if not days. With Mu Studio, it’s super easy to load a multi-protocol transaction and use it as the basis for testing – from functional to fuzz. But how do you find the suspicious transaction or conversation from the large pcap before you can test?
7 things you didn’t know about pcapr
As we approach the 1 year anniversary of pcapr, we were looking back to see how it has evolved. As a company that tests pretty much everything under the sun that has an IP stack, we deal with pcaps for all kinds of protocols. These pcaps were being littered around in public shares, wiki attachments, emails, internal mailing lists and blogs. Turns out we were not the only ones. The broader community and our customers were having similar problems. So it really started out as a way to organize a large collection of pcaps for us and the broader community. Hence the r in pcapr, which stands for repository. But thanks to the community feedback and contribution, pcapr has become a whole lot more than just a repository.
Categories: pcapr
Multi-dimensional data visualization
Way back in grad school, I was working on a project involving Auralization. The key idea was that your ear can process multi-dimensional data (pitch, volume, instruments, silence, tempo, etc) way better than your eyes can (try closing your eyes and listening to a Bach Fugue). So back then, we tried to take these types of data (stocks, sales reports, expenses, etc) and created MIDI files out of it to understand trends. Ever since I saw the Hans Rosling’s TED Talk I’ve wondered the applicability of this type of visualization on something other than economics.
Collaborative Network Forensics
If you’ve dealt with really large packet captures, you’ve probably tried to break things apart into smaller chunks just so you can figure out what’s actually in there. There are lots of command line tools out there that already do this. So it started out as an experiment to see if there’s a better, interactive, visual way to explore large pcaps and rapidly hone in on what you are looking for. With the recent release of large datasets from ITOC the need for this just became a whole lot more critical.
Firefox, Googlecode and Mime Types
So we moved a bunch of the css and images (scripts coming soon) to google code and noticed something. pcapr now directly mashes up images and css from svn/trunk on google code to your browser. But…
Categories: pcapr
Protocol coverage metrics
If all you have is a pcap with some protocol packets in it, how would you know how much of the actual protocol specification (the possible set of fields that the packets could carry) is being covered? This is a useful metric to have when writing a dissector or IPS/DPI signatures. This is much in the spirit of code coverage.
Categories: pcapr
