Centralise Your Logging with ELK

Kibana

3 years using elasticsearch, I only know 1 purpose for that apps..for indexing my content for superfast searching, until yesterday. 

I managed a couple of dedicated server and vps on various country and it kinda pain in my ass to ssh into everybox and check each one of them, after a brief research..I found out that there's an outsource solution for that, using Elasticsearch, Logstash and Kibana.

Elasticsearch is for indexing your logs, Logstash will get and process your logs and pipe it to Kibana for visualizing in better manner.

You know from something like this :

to this :

 

Yes, you can get some of ithose info from other analytics software like webstats, google analytics etc, but with ELK, you can do much more than webserver stats.

for example, I can do statistic from my Fail2ban or auth.log to see what most used password they tried to bruteforce my box, or what time, which country most of them active doing it.

 

 

You can also all your server stats using Topbeat plugin.

 

For people who do statistic, which also parts of my job.. I usually use ipython, pandas and numpy combos. But now I also can do it with ELK for some moderate statistic query.. Just load it on Elasticsearch and do those queries, charts etc on Kibana.

 

For more info and tutorial for how to setup ELK, you can visit Elastic website. Have fun!

Share this Post:

Related Posts:

Comments