Logback JSON encoder and appenders
Special thanks to three new contributors for this release!
writeObject of JsonGenerator passed to setJsonGeneratorDecorator by @rdesgroppes in https://github.com/logfellow/logstash-logback-encoder/pull/968
Full Changelog: https://github.com/logfellow/logstash-logback-encoder/compare/logstash-logback-encoder-7.3...logstash-logback-encoder-7.4
This new version brings support for the new features introduced in Logback 1.3 while remaining compatible with both Logback 1.2 (can be used with both versions).
Support for Timestamp with nanosecond precision (logback 1.3+) by @brenuart in https://github.com/logfellow/logstash-logback-encoder/pull/833
TimestampJsonProvider has been upgraded to take advantage of the nanosecond precision introduced in Logback 1.3 when recording event timestamp.
When upgrading to Logback 1.3, people using one of the standard java.time.DateTimeFormatter formats like [ISO_OFFSET_DATE_TIME] (see documentation for more information) will have their timestamp rendered with nanoseconds instead of millis. If this is not desired you should instead configure the explicit format you need instead of making reference to one of the standard formats provided by java.time.DateTimeFormatter.
Add support for Event#getSequenceNumber (logback 1.3+) by @brenuart in https://github.com/logfellow/logstash-logback-encoder/pull/843
Starting from version 1.3 Logback can assign a unique sequence number to each event as long as the LoggerContext is configured with a SequenceNumberGenerator (see Logback). This number can later be retrieved from the event itself by calling the newly introduced getSequenceNumber() method.
The SequenceJsonProvider has been upgraded to leverage this new feature. The actual behaviour depends on which version of Logback is found on the class path at runtime:
If this behaviour is not desired, you have the choice to set your own custom strategy (see the documentation for more information).
ShortenedThrowableConverter: <exclude> and <exclusions> are mutually exclusive (#876) by @brenuart in https://github.com/logfellow/logstash-logback-encoder/pull/881
Add ability to customize the PrettyPrinter by @philsttr in https://github.com/logfellow/logstash-logback-encoder/pull/926
Add support for multiple Markers attached to a single LoggingEvent by @brenuart in https://github.com/logfellow/logstash-logback-encoder/pull/840
848 ipv6 support in DestinationParser by @brenuart in https://github.com/logfellow/logstash-logback-encoder/pull/849
Implement initialSendDelay feature by @brenuart in https://github.com/logfellow/logstash-logback-encoder/pull/860 It may happen you want to add an extra delay before sending the first events after the connection is established. This may come in handy in situations where the appender connects to an intermediate proxy that needs some time to establish a connection to the final destination. Without this extra delay, the first events written by the appender immediately after the connection to the proxy is created may be lost if the proxy ultimately fails to connect to the final destination.
To enable this feature, set the new initialSendDelay property to the desired value. The default value is 0 which means "no delay", that is start flushing events immediately after the connection is established. See Initial Send Delay for more information.
859 LogstashEncoder should throw IllegalArgumentException instead of logging an error status when attempting to use
867 shortenedthrowableconverter truncate stacktrace based on regex pattern by @brenuart in https://github.com/logfellow/logstash-logback-encoder/pull/874 The ShortenedThrowableConverter now allows you to shorten the stacktrace by cutting it just after frames matching a regular expression. See Truncate after Regex in the documentation for more details about this new feature.
Add a way to escape the comma in comma separated property values by @brenuart in https://github.com/logfellow/logstash-logback-encoder/pull/882
ShortenedThrowableConverter: add option to set my own custom abbreviator by @brenuart in https://github.com/logfellow/logstash-logback-encoder/pull/893
Build consumer (flatten) POM by @brenuart in https://github.com/logfellow/logstash-logback-encoder/pull/898
Full Changelog: https://github.com/logfellow/logstash-logback-encoder/compare/logstash-logback-encoder-7.2...logstash-logback-encoder-7.3
#807 Allow inline generation of stacktrace (@MarneusCalgarXP)
The ShortenedThrowableConverter now allows you to specify the line separator to use instead of a new line by default. This is particularly useful to format a stacktrace on a single line when using rsyslog or fluentd when you don't want to use json format.
For example:
<conversionRule conversionWord="stack"
converterClass="net.logstash.logback.stacktrace.ShortenedThrowableConverter" />
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>... %stack{full,40,500,rootFirst,inline} ...</pattern>
</encoder>
</appender>
| Usage | Dependency | Old Version | New Version |
|---|---|---|---|
| Runtime | jackson | 2.13.2.20220328 | 2.13.3 |
Version 7.1.1 is a minor release with bug fixes only.
Z[UTC] time zone indication is missing when using ISO_ZONED_DATE_TIME or ISO_DATE_TIME formats (@brenuart)#747/#748 Logger name shortening down to the class name (length set to zero). (@brenuart)
When using a LogstashEncoder, the <shortenedLoggerNameLength> configuration property can be used to shorten the logger name as described in here. This feature is supposed to behave the same as the standard Logback length conversion word. However, although the logger name was properly shortened when the desired length was greater than zero, it did not produce the expected result when the length was set to 0. In fact, foo.bar.baz.MyClass was shortened to f.b.b.MyClass instead of MyClass.
#765 PatternJsonProvider: omitEmptyFields doesn't work with %property{key}. (@brenuart)
The Logback %property{key} conversion specifier returns null when the property does not exists. This value is added as-is to the String evaluation of the pattern which ultimately gives the string "null" (in letters) that won't be filtered out by the omitEmptyField feature.
To solve this issue, the PatternJsonProvider now uses a custom %property{key} converter that returns an empty string instead of null when the property does not exist. It also allows you to specify a default value to use when the property does not exist. People wanting to revert to the old behaviour can explicitly set the default value to "null" like this %property{key:-null}.
#751 Fixed wrong link in README.md for #event-specific-custom-fields (@mightymoogle)
#711 Fast Timestamp Format (@brenuart)
The *FormattedTimestampJsonProvider are now faster than ever thanks to the introduction of a new FastISOTimestampFormat.
This new formatter leverages the fact that in a logging scenario timestamps are mostly formatted in chronological order. Instead of computing every date/time fields like the standard DateTimeFormatter does each time it is invoked, the fast formatter remembers the previous formatted value and replaces the second and millis parts when the next timestamp to format is within the same minute. This strategy is 7x faster and produces 10x less garbage compared to the DateTimeFormatter.
The "fast" formatter is used only when the provider is configured with one of the following standard ISO formats (obtained from the Java DateTimeFormatter). A standard DateTimeFormatter is used for the other standard formats or if a custom pattern is configured (even if the pattern matches one of the supported ISO formats).
#772 PatternJsonProvider: support for default value in %property{key} (@brenuart)
The %property{key} conversion specifier used by the PatternJsonProvider has been enhanced to add support for a default value to use when the property does not exist. The default value is optional and can be specified using the :- operator as in Bash shell. For example, assuming the property "foo" is not defined, %property{foo:-bar} will return bar.
If no optional value is declared, the converter returns an empty string instead of null as does the original Logback version.
Check the relevant section of the documentation for more information.
#743 Add new JsonProviders for exception messages (@twz123)
#773/#774 PatternJsonProvider: provide a "#asNullIfEmpty{}" operation (@brenuart)
#719/#720 getFieldValue() should return fieldValue without transforming it into a String (@brenuart)
#728 Custom JsonGenerator with optimized writeObject() for simple object types (@brenuart)
| Usage | Dependency | Old Version | New Version |
|---|---|---|---|
| Runtime | jackson | 2.13.0 | 2.13.2.20220328 |
| Runtime | logback | 1.2.7 | 1.2.11 |
| Compile-time | checkstyle | 9.1 | 9.2.1 |
| Compile-time | animal-sniffer-maven-plugin | 1.20 | 1.21 |
| Compile-time | maven-bundle-plugin | 5.1.2 | 5.1.4 |
| Compile-time | maven-clean-plugin | 3.1.0 | 3.2.0 |
| Compile-time | maven-compiler-plugin | 3.8.1 | 3.10.0 |
| Compile-time | maven-jar-plugin | 3.2.0 | 3.2.2 |
| Compile-time | maven-javadoc-plugin | 3.3.1 | 3.3.2 |
| Compile-time | maven-shade-plugin | 3.2.4 | 3.3.0 |
| Compile-time | nexus-staging-maven-plugin | 1.6.8 | 1.6.12 |
| Test-time | assertj | 3.21.0 | 3.22.0 |
| Test-time | awaitility | 4.1.1 | 4.2.0 |
| Test-time | junit | 5.8.1 | 5.8.2 |
| Test-time | mockito | 4.0.0 | 4.4.0 |
Version 7.0.1 is a minor release with bug fixes only.
7.0 caused #asJson() and #tryJson()to throw a NullPointerException when the returned JSON object contains a nested object or array with null values. This caused the corresponding field to be ignored from the final output. (@brenuart)Version 7.0 is a major release with new enhancements, bugfixes, and several backwards incompatibilities mentioned in the sections below.
I'd also like to introduce @brenuart as a new maintainer! logstash-logback-encoder now has a bus factor of 2. Welcome Bertrand!
We've moved the logstash-logback-encoder repository to the new logfellow GitHub organization. See #700 for more info.
AccessCompositeJsonProvider:
threadName: Name of the thread from which the event was logged (@brenuart).sequence: Output an incrementing sequence number for every log event (@brenuart).uuid: Outputs random UUID as field value. Handy to provide unique identifier for log lines (@brenuart).In addition, a longtime bug in logback (LOGBACK-1326) that affected logstash-logback-encoder (#232) has been fixed in logback 1.2.5!
AsyncDisruptorAppender base class does not expose any ExecutorService anymore. Sub-classes should create their own executor if needed, potentially reusing the exposed ThreadFactory.AsyncDisruptorAppender in SINGLE mode is now deprecated and will be removed in future versions. This mode provides some performance improvements when only one thread is ever appending to the appender but may lead to unpredictable behaviour if not the case.queueSize property to set the size of the ring buffer of the TCP async appender is now deprecated. Use ringBufferSize instead.secondaryConnectionTTL directly on the AbstractLogstashTcpSocketAppender is now deprecated. You should instead set the property on the connection strategy itself. Example: <appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
<connectionStrategy>
<preferPrimary>
<secondaryConnectionTTL>5 minutes</secondaryConnectionTTL>
</preferPrimary>
</connectionStrategy>
</appender>
net.logstash.logback.composite.loggingevent.SequenceJsonProvider is deprecated in favour of net.logstash.logback.composite.SequenceJsonProvider.net.logstash.logback.composite.loggingevent.UuidProvider is deprecated in favour of net.logstash.logback.composite.UuidJsonProvider.AppenderListener and TcpAppenderListener interfaces are now deprecated in favour of the default methods provided by the interfaces themselves.3.6.1 to 3.8.3.| Usage | Dependency | Old Version | New Version |
|---|---|---|---|
| Runtime | logback | 1.2.3 | 1.2.7 |
| Runtime | jackson | 2.12.0 | 2.13.0 |
| Runtime (shaded) | disruptor | 3.4.2 | 3.4.4 |
| Runtime (shaded) | commons-lang3 | 3.11 | removed |
| Compile-time | animal-sniffer-maven-plugin | 1.19 | 1.20 |
| Compile-time | checkstyle | n/a | 9.1 |
| Compile-time | license-maven-plugin | 1.9.0 | 4.1 |
| Compile-time | maven-bundle-plugin | 5.1.1 | 5.1.2 |
| Compile-time | maven-checkstyle-plugin | n/a | 3.1.2 |
| Compile-time | maven-clean-plugin | n/a | 3.1.0 |
| Compile-time | maven-gpg-plugin | 1.6 | 3.0.1 |
| Compile-time | maven-javadoc-plugin | 3.2.0 | 3.3.1 |
| Test-time | awaitility | n/a | 4.1.1 |
| Test-time | assertj | 3.18.1 | 3.21.0 |
| Test-time | junit | 5.7.0 | 5.8.1 |
| Test-time | mockito | 3.6.28 | 4.0.0 |
| Test-time | servlet-api | 2.5 | 4.0.1 |
^) and end-of-line markers ($)AbstractLogstashTcpSocketAppender#setWriteTimeout(Duration) does not accept null values anymore. Use a positive value to enable the write timeout feature. A timeout of zero disables it.\n) instead of being platform dependent. To revert to previous behaviour, set the property keepAliveMessage to SYSTE%.LogstashMarker used to be equal if they had the same name and their references (child Markers) were equal. This means that adding a child marker to an existing one will break the equal and hashCode method and may lead to unpredicted behaviour when used in a Set or used as a key in a Map for example. To avoid those potential issues, the equals and hashCode methods have been changed to test equality on the Marker's name only - just like what SLF4J BasicMarker does. To summarise, LogstashMarker now behave exactly the same as SLF4J BasicMarker: two markers are equal if they have the same name.LogstashBasicMarker has been reduced to package protected. People using this class as a base class for their own SLF4J Marker implementation should implement their own.| Usage | Dependency | Old Version | New Version |
|---|---|---|---|
| Runtime | jackson | 2.11.3 | 2.12.0 |
| Test-time | junit | 4.13.1 | 5.7.0 |
| Usage | Dependency | Old Version | New Version |
|---|---|---|---|
| Runtime | jackson | 2.11.0 | 2.11.3 |
| Runtime (shaded) | commons-lang3 | 3.10 | 3.11 |
| Compile-time | animal-sniffer-maven-plugin | 1.18 | 1.19 |
| Compile-time | maven-bundle-plugin | 4.2.1 | 5.1.1 |
| Compile-time | maven-shade-plugin | 3.2.3 | 3.2.4 |
| Test-time | junit | 4.13 | 4.13.1 |
| Test-time | assertj | 3.16.1 | 3.18.1 |
| Test-time | mockito | 3.3.3 | 3.6.28 |
Charset.forName("UTF-8") with StandardCharsets.UTF_8 (@ArthurGazizov)defer support for StructuredArguments (@philsttr)| Usage | Dependency | Old Version | New Version |
|---|---|---|---|
| Runtime | jackson | 2.10.1 | 2.11.0 |
| Runtime | uuid-generator | 3.2.0 | 4.0.1 |
| Runtime (shaded) | commons-lang3 | 3.9 | 3.10 |
| Compile-time | maven-source-plugin | 3.2.0 | 3.2.1 |
| Compile-time | maven-javadoc-plugin | 3.1.1 | 3.2.0 |
| Compile-time | maven-shade-plugin | 3.2.1 | 3.2.3 |
| Test-time | junit | 4.12 | 4.13 |
| Test-time | assertj | 3.14.0 | 3.16.1 |
| Test-time | mockito | 3.2.0 | 3.3.3 |