21 Mar 2016

Adding two OS Projects together

Testing ElastAlert

For a project we’ve been working on, we had to find an alerting solution that uses ElasticSearch as source. Watcher - the paid enterprise software product sold by Elasticsearch was not an option. So we researched a bit more and found ElastAlert brought to you by the guys from Yelp. Now we had figure out, if it would do what we would need and if it would really work for us.
So how could we setup a test mule? The answer to this question is always Docker thses days and here I remembered a presentation that mentioned something Jerome Petazzoni was working on as part of an Docker orchestration workshop. Using this Docker compose file you can bring up a full ELK with just one command (after you configured docker/docker-compose and probably virtualbox correctly). The nice part with this configuration is that it will generate some logs right from the get-go, as it installs the logstash heartbeat plugin that periodically sends log data to ElasticSearch.
Initially, I started to install ElastAlert with all its dependencies - and there are a lot - in the ElasticSearch Docker container. But then, while searching solutions to all the issues I encountered I found another container that had everything already installed here. There is probably a few more out there, but the one I found first…
So I added this container to Jerome’s docker-compose fil, adjusted it a bit for my purposes and vioala my test mule was ready. Right away I could test - somewhat limited - frequency alerts.
Next I went back to Jerome’s presentation where he demonstrates gelf docker log driver by generating a Hello World log message. One can easily use that to create blacklist alerts. spike alerts can also be simulates, you just have to hit return plenty of times.
Hope this blog entry inspires others.

Til next time,
E.E at 18:15