Go Dns Collector Versions Save

Ingesting, pipelining, and enhancing your DNS logs with usage indicators, security analysis, and additional metadata.

v0.44.0-beta2

2 weeks ago

What's Changed

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces some minor improvements:

Support properly IP fragment with sniffers mode
Code refactoring in collectors package
Complete redesign of the configuration checker

What's Changed

feat: support IP defrag on afpacket sniffer by @dmachard in https://github.com/dmachard/go-dnscollector/pull/691
feat: redesign config checker to support freeform keys by @dmachard in https://github.com/dmachard/go-dnscollector/pull/678
feat: add Scalyr defaults by @pieterlexis-tomtom in https://github.com/dmachard/go-dnscollector/pull/677
feat: major code refacto in collectors package by @dmachard in https://github.com/dmachard/go-dnscollector/pull/667
fix: quick fix to avoid panic with fragmented packet by @dmachard in https://github.com/dmachard/go-dnscollector/pull/684
chore(deps): bump github.com/klauspost/compress from 1.17.7 to 1.17.8 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/680

Bug reporters

@top-secrett #683
@ pieterlexis-tomtom #676

Full Changelog: https://github.com/dmachard/go-dnscollector/compare/v0.43.0...v0.44.0-beta2

v0.43.0

1 month ago

What's Changed

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces some great features

Compression support on DNSTap collector and logger
Custom Flat-JSON relabeling
New ClickHouse logger thank to @zunnurainbadar

What's Changed

feat: transform: normalize lowercase RR's by @dmachard in https://github.com/dmachard/go-dnscollector/pull/665
feat: build with golang 1.22 and 1.20 support removed by @dmachard in https://github.com/dmachard/go-dnscollector/pull/660
feat: added Clickhouse Logger by @zunnurainbadar in https://github.com/dmachard/go-dnscollector/pull/349
feat: add compression support on DNSTap client and server by @dmachard in https://github.com/dmachard/go-dnscollector/pull/538
feat: new relabeling transform by @dmachard in https://github.com/dmachard/go-dnscollector/pull/645
fix: relabel-config error in loki client by @dmachard in https://github.com/dmachard/go-dnscollector/pull/664
chore(docs): add more docs for the atags transformer by @dmachard in https://github.com/dmachard/go-dnscollector/pull/656 https://github.com/dmachard/go-dnscollector/pull/659
chore(deps): bump golang.org/x/sys from 0.17.0 to 0.18.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/651
chore(deps): bump golang.org/x/net from 0.21.0 to 0.22.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/652
chore(deps): bump Loki to v2.9.6
chore: code factory on transformers by @dmachard in https://github.com/dmachard/go-dnscollector/pull/666

New Contributors

@zunnurainbadar made their first contribution in https://github.com/dmachard/go-dnscollector/pull/349

Bug reporters

@rajmohanram #643

Full Changelog: https://github.com/dmachard/go-dnscollector/compare/v0.42.0...v0.43.0

v0.43.0-beta2

1 month ago

What's Changed

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces some great features

Compression support on DNSTap collector and logger
Custom Flat-JSON relabeling
New ClickHouse logger thank to @zunnurainbadar

What's Changed

feat: transform: normalize lowercase RR's by @dmachard in https://github.com/dmachard/go-dnscollector/pull/665
feat: build with golang 1.22 and 1.20 support removed by @dmachard in https://github.com/dmachard/go-dnscollector/pull/660
feat: added Clickhouse Logger by @zunnurainbadar in https://github.com/dmachard/go-dnscollector/pull/349
feat: add compression support on DNSTap client and server by @dmachard in https://github.com/dmachard/go-dnscollector/pull/538
feat: new relabeling transform by @dmachard in https://github.com/dmachard/go-dnscollector/pull/645
fix: relabel-config error in loki client by @dmachard in https://github.com/dmachard/go-dnscollector/pull/664
chore(docs): add more docs for the atags transformer by @dmachard in https://github.com/dmachard/go-dnscollector/pull/656 https://github.com/dmachard/go-dnscollector/pull/659
chore(deps): bump golang.org/x/sys from 0.17.0 to 0.18.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/651
chore(deps): bump golang.org/x/net from 0.21.0 to 0.22.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/652
chore(deps): bump Loki to v2.9.6
chore: code factory on transformers by @dmachard in https://github.com/dmachard/go-dnscollector/pull/666

New Contributors

@zunnurainbadar made their first contribution in https://github.com/dmachard/go-dnscollector/pull/349

Bug reporters

@rajmohanram #643

Full Changelog: https://github.com/dmachard/go-dnscollector/compare/v0.42.0...v0.43.0-beta2

v0.42.0

1 month ago

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces some important bug fixes (memory leak) and minor features

Fix memory leak with ElasticSearch logger, thank to @misaki-kawakami to report that.
Optimizations to reduce CPU usage, more particularly for flat-json
Add support for query-zone field on DNStap collector

What's Changed

feat: decode dns class on queries and replies by @dmachard in https://github.com/dmachard/go-dnscollector/pull/635
feat: more docs for kafka and fluentd loggers, change channel buffer to 4096 by @dmachard in https://github.com/dmachard/go-dnscollector/pull/637 https://github.com/dmachard/go-dnscollector/pull/639
feat: normalize transformer: add icann managed flag in dnsmessage by @dmachard in https://github.com/dmachard/go-dnscollector/pull/640
feat: refactor to optimize dnsmessage model by @dmachard in https://github.com/dmachard/go-dnscollector/pull/632
feat: dnstap collector: decode query zone field by @dmachard in https://github.com/dmachard/go-dnscollector/pull/621
feat: dnstap collector: ignore empty raw dns payload by @dmachard in https://github.com/dmachard/go-dnscollector/pull/622
fix: elasticsearch: release buffer properly to avoid memory leaks by @dmachard in https://github.com/dmachard/go-dnscollector/pull/630 https://github.com/dmachard/go-dnscollector/pull/644
fix: build to support 32b platform by @dmachard in https://github.com/dmachard/go-dnscollector/pull/620
chore(deps): bump github.com/klauspost/compress from 1.17.6 to 1.17.7 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/625
chore(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/633
chore(deps): bump golang.org/x/sys from 0.16.0 to 0.17.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/606
chore(deps): bump golang.org/x/net from 0.20.0 to 0.21.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/607
chore(deps): bump github.com/prometheus/client_model from 0.5.0 to 0.6.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/616
chore(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/647
chore(deps): bump github.com/prometheus/client_golang from 1.18.0 to 1.19.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/648
chore(ci): add timeout in github actions tests by @dmachard in https://github.com/dmachard/go-dnscollector/pull/623
chore(ci): add bench-go github actions by @dmachard in https://github.com/dmachard/go-dnscollector/pull/638
chore(ci): bump softprops/action-gh-release from 1 to 2 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/646

Bug reporters

@mikeS7
@misaki-kawakami

Full Changelog: https://github.com/dmachard/go-dnscollector/compare/v0.41.0...v0.42.0

v0.42.0-alpha1

2 months ago

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces some important bug fixes (memory leaks) and minor features

query-zone field support on DNStap collector

What's Changed

feat: dnstap collector: decode query zone field by @dmachard in https://github.com/dmachard/go-dnscollector/pull/621
feat: dnstap collector: ignore empty raw dns payload by @dmachard in https://github.com/dmachard/go-dnscollector/pull/622
fix: build to support 32b platform by @dmachard in https://github.com/dmachard/go-dnscollector/pull/620
chore(deps): bump golang.org/x/sys from 0.16.0 to 0.17.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/606
chore(deps): bump golang.org/x/net from 0.20.0 to 0.21.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/607
chore(deps): bump github.com/prometheus/client_model from 0.5.0 to 0.6.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/616

Bug reporters

@mikeS7

Full Changelog: https://github.com/dmachard/go-dnscollector/compare/v0.41.0...v0.42.0-alpha1

v0.41.0

2 months ago

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces some new features and bug fixes.

RPZ policy fields support on DNStap/PowerDNS protobuf collectors
A new peer-name text-format directive and JSON field, thanks to @peterand-pa for idea
Support the DoH3 HTTP field added with the release of DNSdist 1.9

What's Changed

feat: add doh3 http version field support and test for DoQ by @dmachard in https://github.com/dmachard/go-dnscollector/pull/610
feat: add peer name text-format directive by @dmachard in https://github.com/dmachard/go-dnscollector/pull/608
feat: add configuration examples for pipeline mode by @dmachard in https://github.com/dmachard/go-dnscollector/pull/599
feat: add DNStap policy support by @dmachard in https://github.com/dmachard/go-dnscollector/pull/593 #530
feat: add RPZ policy details with powerdns protobuf by @dmachard in https://github.com/dmachard/go-dnscollector/pull/590 #572
feat: expose dns id in json model by @dmachard in https://github.com/dmachard/go-dnscollector/pull/586
feat: detect and exit on error with invalid text directives by @dmachard in https://github.com/dmachard/go-dnscollector/pull/576
fix: fluentd: remove msgpack lib and duplicates entries #611 by @dmachard in https://github.com/dmachard/go-dnscollector/pull/612
fix: add missing rd and cd flags in json model by @dmachard in https://github.com/dmachard/go-dnscollector/pull/589
fix: dnstap collector: stuck on stop can be possible in specific situation by @dmachard in https://github.com/dmachard/go-dnscollector/pull/597
fix: syslog logger - fix protocol wrong type for unix socket by @dmachard in https://github.com/dmachard/go-dnscollector/pull/598
chore(docs): add documentation to use docker-compose by @dmachard in https://github.com/dmachard/go-dnscollector/pull/563
chore(docs): update makefile and docs by @dmachard in https://github.com/dmachard/go-dnscollector/pull/604
chore(deps): bump golangci/golangci-lint-action from 3 to 4 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/603
chore(deps): bump alpine from 3.19.0 to 3.19.1 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/577
chore(deps): bump github.com/google/uuid from 1.5.0 to 1.6.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/579
chore(deps): bump github.com/klauspost/compress from 1.17.4 to 1.17.6 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/596
chore(deps): bump github.com/dmachard/go-topmap from 0.5.0 to 1.0.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/580
chore(deps): bump github.com/dmachard/go-powerdns-protobuf from 1.0.0 to 1.1.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/578

Full Changelog: https://github.com/dmachard/go-dnscollector/compare/v0.40.2...v0.41.0

Bug reporters

@johnhtodd
@peterand-pa
@Jon1760

v0.41.0-beta1

3 months ago

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release introduces the support of DNSTAP and PowerDNS fields policy and some bug fixes.

What's Changed

feat: add configuration examples for pipeline mode by @dmachard in https://github.com/dmachard/go-dnscollector/pull/599
feat: add DNStap policy support by @dmachard in https://github.com/dmachard/go-dnscollector/pull/593 #530
feat: add RPZ policy details with powerdns protobuf by @dmachard in https://github.com/dmachard/go-dnscollector/pull/590 #572
feat: expose dns id in json model by @dmachard in https://github.com/dmachard/go-dnscollector/pull/586
feat: detect and exit on error with invalid text directives by @dmachard in https://github.com/dmachard/go-dnscollector/pull/576
fix: add missing rd and cd flags in json model by @dmachard in https://github.com/dmachard/go-dnscollector/pull/589
fix: dnstap collector: stuck on stop can be possible in specific situation by @dmachard in https://github.com/dmachard/go-dnscollector/pull/597
fix: syslog logger - fix protocol wrong type for unix socket by @dmachard in https://github.com/dmachard/go-dnscollector/pull/598
chore(docs): add documentation to use docker-compose by @dmachard in https://github.com/dmachard/go-dnscollector/pull/563
chore(deps): bump golangci/golangci-lint-action from 3 to 4 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/603
chore(deps): bump alpine from 3.19.0 to 3.19.1 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/577
chore(deps): bump github.com/google/uuid from 1.5.0 to 1.6.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/579
chore(deps): bump github.com/klauspost/compress from 1.17.4 to 1.17.6 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/596
chore(deps): bump github.com/dmachard/go-topmap from 0.5.0 to 1.0.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/580
chore(deps): bump github.com/dmachard/go-powerdns-protobuf from 1.0.0 to 1.1.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/578

Full Changelog: https://github.com/dmachard/go-dnscollector/compare/v0.40.2...v0.41.0-beta1

Bug reporters

@johnhtodd
@peterand-pa

v0.40.2

3 months ago

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

What's Changed

fix(major): patch to avoid deadlock situations in multiplexer mode - regression by @dmachard in https://github.com/dmachard/go-dnscollector/pull/569
fix(minor): detect invalid logger type in configuration file by @dmachard in https://github.com/dmachard/go-dnscollector/pull/573
fix(minor): add tests to check and avoid deadlocks with pipeline mode by @dmachard in https://github.com/dmachard/go-dnscollector/pull/571
chore(deps): bump github.com/segmentio/kafka-go from 0.4.46 to 0.4.47 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/517
chore(deps): bump github.com/google/uuid from 1.4.0 to 1.5.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/518
chore(deps): bump golang.org/x/crypto from 0.16.0 to 0.17.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/519
chore(deps): bump google.golang.org/protobuf from 1.31.0 to 1.32.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/524
chore(deps): clean security code scanning and some dependencies by @dmachard in https://github.com/dmachard/go-dnscollector/pull/554
chore(deps): bump Loki to v2.9.3 by @dmachard in https://github.com/dmachard/go-dnscollector/pull/555 https://github.com/dmachard/go-dnscollector/pull/556
chore(deps): bump golang.org/x/sys from 0.15.0 to 0.16.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/559
chore(deps): bump github.com/dmachard/go-framestream from 0.7.0 to 0.10.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/560
chore(deps): bump github.com/miekg/dns from 1.1.57 to 1.1.58 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/561
chore(deps): bump github.com/prometheus/common from 0.45.0 to 0.46.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/558

Full Changelog: https://github.com/dmachard/go-dnscollector/compare/v0.40.0...v0.40.2

v0.40.0

3 months ago

If you find the project helpful, please consider supporting the project via monthly donation via GitHub Sponsors or simply add a star to this project.

Highlights

This release is major and introduces some important breaking changes and bug fixes.

A new codebase to support the future pipelining mode, adding more flexibility to handle your incoming logs.
New Prometheus metrics has been added to fix a potential unlimited memory usage, which is not acceptable as resources are not limitless.
The possibility to extend ougoing DNStap streams with metadata such as geoip, filtering, ... It applied to have a DNScollector in remote side to decode these extended DNStap messages
Invalid configurations are now properly detected.

What's Changed

feat (#516): dnstap logger/collector - support extended field by @dmachard in https://github.com/dmachard/go-dnscollector/pull/548
feat: dnstap logger: support of DNSTap extra field by @dmachard in https://github.com/dmachard/go-dnscollector/pull/547
feat: experimental support for pipelining by @dmachard in https://github.com/dmachard/go-dnscollector/pull/503
feat (#473): configurable CIDR mask and hash for user-privacy transformer by @dmachard in https://github.com/dmachard/go-dnscollector/pull/528
feat (#522): more checks to detect invalid configurations by @dmachard in https://github.com/dmachard/go-dnscollector/pull/523
feat: rename dnstap logger to dnstapclient in YAML config (breaking changes) by @dmachard in https://github.com/dmachard/go-dnscollector/pull/526
feat: avoid unlimited mem usage with prometheus logger (breaking changes in metrics) by @dmachard in https://github.com/dmachard/go-dnscollector/pull/529
feat: update grafana dashboard according to PR#529 by @dmachard in https://github.com/dmachard/go-dnscollector/pull/542
fix: kafka, redis, fluentd and tcpclient loggers: no flush occurred on specific conditions with connection attempts by @dmachard in https://github.com/dmachard/go-dnscollector/pull/552
fix: syslog logger: fix crash when remote server refused connections by @dmachard in https://github.com/dmachard/go-dnscollector/pull/551
fix: bad yaml key for dnstap-relay collector by @dmachard in https://github.com/dmachard/go-dnscollector/pull/546
fix (#539): missing latency when quiet-text is enabled by @dmachard in https://github.com/dmachard/go-dnscollector/pull/540
fix: typo error - text format directive reducer-occurences renamed to reducer-occurrences
chore(deps): bump golang.org/x/sys from 0.14.1-.... to 0.15.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/486
chore(deps): bump github.com/segmentio/kafka-go from 0.4.45 to 0.4.46 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/494
chore(deps): bump golang.org/x/net from 0.18.0 to 0.19.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/495
chore(deps): bump github.com/klauspost/compress from 1.17.3 to 1.17.4 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/496
chore(deps): bump alpine from 3.18.5 to 3.19.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/505
chore(deps): bump golang from 1.21.4-alpine3.17 to 1.21.5-alpine3.17 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/506
chore(ci): update testing-dnstap.yml with unbound 1.19 by @dmachard in https://github.com/dmachard/go-dnscollector/pull/543
chore(docs): docs updated by @dmachard in https://github.com/dmachard/go-dnscollector/pull/541

Full Changelog: https://github.com/dmachard/go-dnscollector/compare/v0.39.0...v0.40.0

Bug reporters

@top-secrett
@johnhtodd

v0.40.0-alpha3

4 months ago

What's Changed

feat (#473): configurable CIDR mask and hash for user-privacy transformer by @dmachard in https://github.com/dmachard/go-dnscollector/pull/528
feat (#522): more checks to detect invalid configurations by @dmachard in https://github.com/dmachard/go-dnscollector/pull/523
feat: rename dnstap logger to dnstapclient in YAML config (breaking changes) by @dmachard in https://github.com/dmachard/go-dnscollector/pull/526
feat/fix: avoid unlimited memory usage with prometheus logger (breaking changes) by @dmachard in https://github.com/dmachard/go-dnscollector/pull/529
chore(deps): bump golang.org/x/sys from 0.14.1-.... to 0.15.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/486
chore(deps): bump github.com/segmentio/kafka-go from 0.4.45 to 0.4.46 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/494
chore(deps): bump golang.org/x/net from 0.18.0 to 0.19.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/495
chore(deps): bump github.com/klauspost/compress from 1.17.3 to 1.17.4 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/496
chore(deps): bump alpine from 3.18.5 to 3.19.0 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/505
chore(deps): bump golang from 1.21.4-alpine3.17 to 1.21.5-alpine3.17 by @dependabot in https://github.com/dmachard/go-dnscollector/pull/506

Full Changelog: https://github.com/dmachard/go-dnscollector/compare/v0.39.0...v0.40.0-alpha3