Tuesday, September 1, 2015

Bro Plugins Part II: Redis

Plugins Part II
Plugins in Bro have been around for a while, however they are starting to move outside of the "core" and are now beginning to resemble Apache modules. I think this is wise for a few reasons. One, as Bro becomes popular certain features need to stay in the core and certain features need to definitely not be in the core. This is the same reason why numpy or pandas libraries aren't included in Python by default. They offer fantastic solutions to specific problems but not everyone needs or wants them. Two, as more people start to use Bro and want to customize it an issue around trusted extensions and capabilities arises. Moving to a plugin architecture will push the burden of vetting code to the Bro operator and away from the Bro development team. Third, an external plugin architecture provides a nice landscape for a package manager, which has been on the Bro todo list for a little while.
   
To test out the Redis plugin, simple clone the Bro repository recursively as typically done, configure and build Bro, change to aux/plugins, and run "make build-redis". Lastly, you'll need to set your BRO_PLUGIN_PATH environment variable the location of aux/plugins. To compile the plugin you'll need a few development header files installed but if all was successful you should see a dynamically loaded Redis plugin when you run:

bro -N 2>&1 | less -S
All the configurable options for the plugin are located in aux/plugins/redis/scripts/init.bro, which is bootstrapped by aux/plugins/redis/scripts/__load__.bro as one would expect.

If you then start the Redis server on the same host you are running Bro on and invoke Bro with the script aux/plugins/redis/scripts/Bro/Redis/logs-to-redis.bro you should eventually see protocol logs which would have been sent to Redis dumped to stdout. Currently, I could only get the plugin to execute the Redis PING command for each log entry Bro was ready to write.

It will be interesting...
It will be interesting for reading/writing to/from memory instead of disk. Imagine using Redis as a LRU cache to store things like HTTP user-agents seen egressing a network or creating a hot (Redis) and cold (ElasticSearch) storage system for complex hash maps of connection tuples and service types. Check out this sort of old article outlining how HipChat did it.

It will be interesting to use for logging. Redis is often used as a broker back end in scalable applications and I think having logs written to it is a big first step in building larger scale systems which act on network activity. Other processes and systems may be able to ingest Bro logs much quicker than if reading from flat files or having to hook into Bro specific client libraries.

It will be interesting for the Input framework. Having Redis as a back end could also allow easier integration between third party intelligence providers. For example, external Python scripts could be used to parse network indicators from those flashy [adjective] [noun] titled threat reports vendors publish and push those threat indicators into Redis for Bro to ingest and notify against.

In Conclusion    
It's good to see Bro playing nice with other open source project. The JSON log writer was a huge boon for those indexing logs with ElasticSearch and Mongo. Plugins of these types will assist in more wide spread use and adoption of Bro outside of research labs.