elasticsearch, logstash and kibana configuration for pi-hole visualiziation
elk-hole provides the relevant files and configuration to easily visualize pi-holes/dnsmasq statistics via the popular elasticstack.
For official installation guides of the elk stack - refer to [Elastic](https://www.elastic.co/ for details)
For a quick setup, check out easyELK
Elk-hole provides the relevant files and configuration for sending the pi-hole logs via filebeat directly to logstash/elasticsearch. We will then visualize the logs in kibana with a custom dashboard.
The result will look like this:
20-dns-syslog.conf
to /etc/logstash/conf.d/
and /patterns
to /etc/logstash/
to your logstash system.Your files should be like this:
/etc/logstash/conf.d/20-dns-syslog.conf
/etc/logstash/patterns/dns
If you have other files in this folder make sure to properly edit the input/output/filter sections to avoid matching our filebeat dns logs in these files which may be processed earlier. For testing purposes you can name your conf files like so:
/conf.d/20-dns-syslog.conf
/conf.d/30-other1.conf
/conf.d/40-other2.conf
This makes sure that /conf.d/20-dns-syslog.conf
is processed at the beginning.
3.a Using vim or nano, open/edit 20-dns-syslog.conf
. You may want to scroll down to the date section and change timezone
to match your local time.
3.b Head to the output section and set ELASTICSEARCHHOST:PORT
to match your environment. If elasticsearch is running on the same system as logstash, then 127.0.0.1:9200
should work.
systemctl restart logstash.service
filebeat.yml
to your /etc/filebeat/
and copy 99-pihole-log-facility.conf
to /etc/dnsmasq.d/
hosts:
line and enter the IP address of the logstash system LOGSTASH IP:5141
systemctl restart filebeat.service
pihole restartdns
service filebeat status
The output should show a couple key message. Active: active (running) & Connection establishedsudo filebeat test output
should show:Logstash: <Logstash IP>:5141...
Connection..
Parse hosts... OK
Dns lookup... OK
Addresses: <Logstash IP>
Dial up... OK
TLS... WARN secure connection disabled
Talk to server... OK
The following steps on the Kibana Host will not work correctly if sending data to logstash is not successfull!
http://Kibana IP:5601
logstash-syslog-dns*
- It should find one index@timestamp
json
(or ndjson
if you are using a recent version of elk) folder and import the following files depending on your software version (1.3.1 or 7.x)elk-hole - vis.json
elk-hole - vis_enhanced.json
elk-hole - vis_enhanced_fix.json
elk-hole - dash.json
elk-hole - dash_enhanced.json
Note: When you import these files, you could possibly see a message "Index Pattern Conflicts". This is ok. Below that message you may see one or two rows of data. On each row click on the drop down menu and select "logstash-syslog-dns*"
logstash-syslog-dns-index.template_ELK7.x.json
logstash-syslog-dns-index.template_ELK7.x.json
into kibanas dev tools console{
"acknowledged" : true
}
systemctl restart logstash.service
systemctl restart elasticsearch.service
systemctl restart kibana.service
Important: Please also re-index after setting everything up and also refresh kibanas index field list
You should then be able to see your new dashboard and visualizations.