Friday, November 22, 2013

An Interview with Bro(s)

In an effort to get additional perspectives on Bro, I recently contacted the contributors to Chris Sanders's and Jason Smith's upcoming book Applied Network Security Monitoring (NSM). Chris, Jason, Liam Randall, and David Bianco (who wrote the bulk of the book's material on Bro) were all kind enough to providing further insight into the Bro platform by answering a set of questions I proposed to them. 


I was first introduced to Bro as "the IDS you probably haven't heard of". How did you first learn about Bro?

Chris: I've been a long time Snort user and really believe in the power of signature-based IDS when used properly, but I also know that the future of NSM (or the now, some might say) is anomaly-based detection. I knew of Bro for quite a while as one of the better ways to perform anomaly-based detection for quite some time, but it wasn't until the past couple of years that I felt Bro was really beginning to gain the maturity that made it feasible for use in modern networks at scale. Once I really dug into Bro and was shown a few things by David Bianco and Liam Randall I really started to see the power of Bro and was blown away.

David: I used to work for Jefferson Lab, a US Department of Energy facility. Bro, of course, came out of UC Berkeley & Lawrence Berkeley National Lab, the latter of which is a DOE facility. Bro has been popular at DOE for quite a long time. My first real encounter with Bro was as a guest member of a DOE Red Team assigned to do a assessment at a sister site. It was pretty effective at detecting typical Red Team internal recon behavior. I didn't start actually using Bro myself until 2007, though. I've used in on and off sense then, and it's been a pretty valuable tool in my kit.

Jason: I learned about Bro several years back. Some guys much smarter than myself were in talks about creating a correlation engine of sorts. In reference to network logging, they had in mind what they wanted to look at, and in the end one of the gentlemen in that group mentioned that what they want was already being created. I'm not sure what happened with their correlation engine, but Bro has come a long way since then.

Liam:  I came to Bro out of desperation. Working in IR and auditing I was constantly finding fault in other open source and commercial tools. They had weak or no IPv6 support, they didn't work well with complex network provisioning, and they were just not designed to deal with the advanced threats I kept encountering in the field.


What advice would you give to those just starting out with Bro (besides reading the Bro chapter in Applied NSM)?

Chris: I think the best advice that I can give is the same advice I'd give anybody wanting to learn something new, and that is to just play around with it. It is so powerful and flexible and you can literally do almost anything with it. One good task might be to take a Snort rule that you understand, and try to achieve the same type of detection with Bro. This will help you learn the syntax and how to interact with Bro data types and generate the appropriate logs.

David: Learn all the flags to the "grep" command, because you'll be using it a lot to go through code samples! Seriously, the best advice is to take it slow and don't get discouraged. Bro is not a product, per se, so you don't just install, configure and turn it on. Come up with a few simple network oriented tasks you want to solve (not necessarily something that will detect Evil!) and use those to learn the language and the platform. After that, spend some time thinking about where you really need Bro. It can do almost anything, though it's up to you to use it wisely!

Jason: Start with the basic logging that Bro provides. Once you've managed to set up logging in your environment, set an analysis goal and approach it methodically (with Bro scripts or otherwise). With tools like Bro, it seems that the more you learn, the more you realize you can do. Suddenly, the one obtainable goal that you had set is now 10 obtainable goals, thanks to the knowledge you've obtained through approaching the original goal.

Liam: The best way to get into Bro is to use it. Start by replaying traffic samples through Bro and looking at the output. Looking at anomalous traffic will surprise you. Many common attacks and abuse just stand out like a sore thumb in Bro logs. Run Bro on your home network. It's informative seeing the output of traffic you are already familiar with on a smaller scale. You'll be surprised at what the devices on your home network are actually doing. Believe me, I put my TV on its own VLAN years ago!


What is the one thing Bro does that makes each of you say "Wow"?

Chris: The thing that makes me say WOW the most is that there isn't much that Bro can't do. I think the most useful feature for me is that every entry in every log file has a unique identifier that can be used to identify other aspects of communication between hosts in other log files. This takes a lot of the leg work out of correlation. I'm also generally amazed at how you can take every output from Bro and feed it back in as an input to refine detection or analysis.

David: I think that's the wrong question to ask. It's a lot like saying "What does Python do that makes you say 'Wow'?" Bro can do pretty much anything that involves network traffic (and even things that don't, so that's a "wow!" for you). The thing about Bro that really amazes me is the team behind it. They produce this fantastic platform, with tons of built-in features and frameworks to enable the community to do things even the Bro team hasn't yet thought of, and they give it all away for free! Wow!

Jason: Bro is one of very few tools that can make me feel like a beginner every time I use it, and I mean that in the best way. Calling it a "tool" isn't really fair, as it is more of a "capability". It is a wide open means of getting what you want from your data, which results in learning new ways to use Bro every time. Again, when I approach one goal with Bro, I generally end up with 10 more ideas.

Liam:  Bro continues to wow me on a regular basis. As Bro's tool chains and frameworks have evolved we keep discovering more and more use cases. Here's one that still can't believe: I had worked with a global retailer (a name brand everyone knows) to deploy Bro on their network. I get a call from their IR staff that they thought Bro had misfired with a "'credit card detected" alert. I thought that was unusual as Bro runs the Luhn checks on candidate hits and so I requested a pcap. What I found will continue to be passed along as Bro lore. A credit card processor had faxed over an 18 page credit card dispute form that included notes from a customer where the customer had printed their actual credit card number on paper. When it hit the fax server it was OCR'd and forwarded to Exchange for delivery. When Bro saw it on the wire, Bro popped the vlan, saw smtp, extracted the message, saw a .tif attachment, pulled the credit card number out of the OCR text in the TIF, verified it with Luhn, and then fired an alert. I shit you not. Part of the magic for this one was decent OCR, but still, I have broments like that all the time.


Do you use Bro at work, or if you're more comfortable, home, and how?

Chris: I use Bro at home on my network and lab, but I also deploy it to my clients in some instances. I try to use Bro in a lot of environments for my clients for specific tasks. For instance, deploying Bro to monitor a sensitive network segment with static IP addresses where all assets should be known. Using the same script that is discussed in the book in Chapter 10, Bro becomes a "dark net" detector, which will alert when communication is seen with any IP address that isn't on an approved list, existing in a dark part of the network.

David: I've used Bro at work for a number of things, not all of them detection oriented. I do a lot of packet analysis, and Bro is becoming my tool for quickly ripping apart PCAPs and extracting all kinds of useful info. I've also used it for a while inside malware sandboxes to try to detect/decode known families of malware, or even to just apply the Intel framework to see if the samples coming in are hitting any of the intel we already know about.

Jason: I use Bro as much as I am able to. My first and foremost use of Bro is just the basic out of the box logging. It logs everything I want out of the box, and on top of that it allows me to perform custom logging. On top of that, I use Bro for a lot of my detection experimentation. With Bro, I generally seem to start with narrowing down a custom logging format, and the next thing you know, it is more of a finely tuned detection mechanism. That feels good.

Liam: I use bro everywhere. I run Bro at home, it's my go to tool for IR, and professionally I deploy Bro for a living. As a team we are constantly discovering new use cases for the Bro Platform. I actually spend a lot of my time helping customers customize Bro for their specific use cases. It's exciting.


What are some commercial tools Bro could replace or compliment?

Chris: I am generally of the belief that most commercial NSM and detection tools on the market aren't entirely useful. They are basically "network antivirus" and try to automate detection and analysis. While you can automate detection to some extent, you can't automate analysis. The human has to be in the loop. I also think that most commercial tools abstract you from the raw data too much. These are reasons we only cover free/open source tools in Applied NSM. As for drawing comparisons between Bro and something else... I don't think there is anything commercially available that is like Bro.

David: Since it's really just a powerful programming language for network traffic, Bro really does have the potential to replace almost any network-oriented security tool out there. That said, I may be a little unusual in the Bro community, because I really also like traditional NIDS platforms like Snort. I think there's a place for both Bro and Snort (or similar IDS), and while one *could* implement traditional IDS signatures in Bro, it usually doesn't make as much sense because Snort is so good at that already. There's plenty of room in that space for both to co-exist, and I think they really complement each other when used appropriately.

Jason: Well, when you have Bro fans regularly give demos to vendors about what they should be doing instead of the other way around, you know that there seems to be a shift in things. A ton of organizations implement a ridiculous array of logging devices and detection mechanisms to perform tasks that are trivial for Bro. Without being too specific, it is more responsible to have centralized network intelligence, and Bro has that covered. Previously, many organizations had a huge reason to turn down Bro, and that was that it didn't meet the compliance check box for "professional support", but with groups like Broala, that is no longer a barrier for entry. I see a future with Bro replacing many current logging platforms, and down the road, complimenting other detection platforms.

Liam:  Bro both replaces and compliments entire suites of tools. Fundamentally it is a different approach to streams. A good example, people frequently ask, "Bro or Snort?". That's the wrong question. The tools approach problems with entirely different functions. Ultimately I believe in defense-in-depth, and that means layers.


What are some key takeaways you hope analysts will get from the Bro chapters?

Chris: The power and flexibility of Bro are unlimited. Applied NSM doesn't aim to be a thorough text on everything you can do with Bro, but rather, it contains some basic to advanced Bro concepts that are taught via example. We teach the reader how to build a dark net detector, how to extract specific files based upon MIME type, how to customize logging output, and more. With each practical example the reader should learn more concepts that can be applied to other scenarios. After all, this is APPLIED NSM, and we want the reader to be able to apply the knowledge. Between the concepts we present, the official Bro documentation, and scripts available from the Bro community, I think that leaves a lot that the reader can really work it.

David: The key points probably are the following: Bro is a platform, not an IDS, and getting started with Bro isn't so hard if you don't try to learn everything at once. I didn't have space to cover even 10% of what you really should know about Bro, though even what I have in the examples is pretty useful for solving real problems. Really, Bro needs it's own book, both for programming and for using it "cookbook" style.

Jason: It is an excellent tutorial for those getting started and it will no doubt teach the regular users a thing or two. I wish this material had existed when I first started with Bro!

Liam:  The biggest take away for people just getting started with Bro is this: Bro IDS is just the first great application written in the Bro Programming Language. There are hundreds of use cases for the Bro Platform.


Is there anything else you want people to know about Applied Network Security Monitoring or Bro?

Chris: I'm an analyst first and foremost. That goes for my co-author Jason Smith and my Contributing authors David Bianco and Liam Randall as well. This is really a book for analysts, by analysts. I've always wanted a book that I could give an entry to mid level analyst that would allow them to get up to speed on necessary topics for the job, or to help them further refine their methods and procedures. No book like that has ever existed, so that's why I decided to write Applied NSM. While it doesn't cover every topic thoroughly, I believe that it provides the necessary jump start for the individual who wants to be an NSM analyst. A lot of the concepts I've written into this book are new, and a lot of them are known techniques that have just never been codified and written down. I hope that this will be THE book for new NSM analyst, and that even the NSM veterans can gain something from it.

David: It's coming out on December 15th, and would look *great* under some security pro's tree! Seriously, I just want to thank Chris Sanders and Jason Smith for letting me help in my small way.

Jason: Applied Network Security Monitoring attempts to approach NSM in a way that will ensure that when you or your organization set an implementation goal, ANSM provides solid direction so you don't feel stranded. However, ANSM doesn't stop there. ANSM provides solid methods for collection, detection, and analysis, as well as an immense amount of reference material that should be at every analyst's desk. Regarding Bro, everyone is talking about Bro, but it is time to start implementing Bro. ANSM will get you started.

Liam: Working with Chris, Jason and David on the book was a real pleasure. It's great to work side by side with such talented people!


So there you have it - not only is Bro flexible enough to be used in a myriad of situations, it's also a continually-evolving platform that can only get better with time. To learn more about the book discussed, be sure to check out Applied Network Security Monitoring, available on December 15. Thank you to Chris, Jason, Liam, and David for taking time to participate in this interview.