Pfelk Docker Versions Save

Incorporated default security (elastic) into the pfelk repo. Added more steps and inhibited script from doing a complete installation but it's a simple solution to getting started with OPNsense & pfSense remote logging.

Data Streams, native ILM support and various tidying (more efficient logging)

Various updates and tweaks. This release was to capture the past several months of revisions. Additionally, the file structure was amended to allow for a more seamless install (docker/host). The pipelines.yml file points to the new conf file location (/etc/pfelk/conf.d) and those wishing to add multiple pipelines (e.g. Wazuh etc..) can now amend the pipelines.yml for additionally pipelines while utilizing the default conf.d folder (doesn't conflict with pfelk).

v6.1 2020/12/10 -LOGSTASH

• conf files - Made various changes for ECS conformity - Prevented default logstash template from being installed (eliminated initial setup issues) manage_template => false - Enabled ECS compatibility (v1) - Update GROK pattern aligning log output with ECS v1.7.0 - Most fields are now compliant - Fields with pf parent are not ECS supported but renamed within GROK pattern for better organization - Squid and Snort parent fields removed to align with ECS - Enriched tcp.options field parsing out values in an array vs single string - Parsed DHCP logs for independent indexing - Removed or amended 'host' field to comply with ECS

-ELASTICSEARCH

• templates - Migrated to new index templates - Legacy templates are depreciated and likely removed with pending v8 release (Elastic) - ECS compliant template utilized/implemented - Created ILM - Roll over at 5G or 7-days - Still needs refining - Suricata template built based off of https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-suricata.html - The following alias fields were ommited - fileinfo.filename - fileinfo.size - dest_port - src_port - proto - src_ip - dest_ip - http_status - http.http_user_agent - http.http_refer - http.url - http.hostname - http.length - http.http_method - timestamp - alert.severity - alert.action - flow.bytes_toclient - flow.start - flow.pkts_toclient - flow.bytes_toserver - flow.pkts_toserver - app_proto

                  - Haproxy template was refined based off of https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-haproxy.html
                    - Still needs testing and finalization (note: grok pattern was primary utilized to amend fields)
                    - The following fields were ommited
                      - time_request <-- needs to be amended to align with haproxy module
                      - time_backend_response <-- needs to be amended to align with haproxy module
                      - http_status_code <-- Alias 
  

-KIBANA

• Visualizations - Updated and aligned with templates
• Dashboards - Updated and aligned with updates

v6.0 2020/10/18 -LOGSTASH

conf files - Removed host filtering (mitigate issues with logs traversing via routers/containers)
- Added observer fields for enhanced filtering for multiple firewall setups
grok pattern - Updated to conform to Elastic Common Schema (ECS) and aligned with pfsense Raw Filter Format

-ELASTICSEARCH

templates - Added index settings and mappings
- Templates are dependent upon underlying templates
-KIBANA
Visualizations - Updated and aligned with templates
Dashboards - Custom index pattern ID for each major template

Updated with latest configuration files.

• Refined configuration files
• Merged Suricata, Snort and Squid within 10-apps.conf
• Added haproxy.json and pfelk.json tempaltes

Updated with latest configuration files.

• Supporting Squid
• Supporting HAProxy
• Enhanced Unbound
• Rebuilt Dashboards
• Reconfigured Configuration Files For Future Enrichment
• Versioning skipped to match pfELK and pfELK Docker

Working stable version of pfelk running in a docker.