What's Changed

• test: use T.TempDir to create temporary test directory by @Juneezee in https://github.com/talariadb/talaria/pull/101
• Add cocureency ingestion per threads setting by @atlas-comstock in https://github.com/talariadb/talaria/pull/100
• Update badgerdb and ristretto v0.2.0 to fix bugs on arm by @atlas-comstock in https://github.com/talariadb/talaria/pull/98
• Add config block cache size for badgerDB by @atlas-comstock in https://github.com/talariadb/talaria/pull/102
• fix issues with pr #86 by @jeffreylean in https://github.com/talariadb/talaria/pull/106
• Refactor/streaming v2 by @jeffreylean in https://github.com/talariadb/talaria/pull/107
• add tables field to ingest grpc contract by @jeffreylean in https://github.com/talariadb/talaria/pull/108
• Upgrade github CI go version to 1.17 by @atlas-comstock in https://github.com/talariadb/talaria/pull/111
• Update GitHub action go version by @atlas-comstock in https://github.com/talariadb/talaria/pull/112
• Fix graceful shutdown by @atlas-comstock in https://github.com/talariadb/talaria/pull/110
• implement instrumentation in bigquery by @jeffreylean in https://github.com/talariadb/talaria/pull/117
• lint the code base on https://staticcheck.io/docs/checks/ by @atlas-comstock in https://github.com/talariadb/talaria/pull/119
• fix streaming inside ingestWithTable method by @jeffreylean in https://github.com/talariadb/talaria/pull/121
• add grpc call message size configuration by @jeffreylean in https://github.com/talariadb/talaria/pull/123
• Feature/nats ingress by @jeffreylean in https://github.com/talariadb/talaria/pull/122
• Spark connector by @tardunge in https://github.com/talariadb/talaria/pull/115
• fix nats conf tag casing by @jeffreylean in https://github.com/talariadb/talaria/pull/124
• Bump github.com/aws/aws-sdk-go from 1.30.25 to 1.33.0 by @dependabot in https://github.com/talariadb/talaria/pull/125
• Hotfix/slow consumer handling by @jeffreylean in https://github.com/talariadb/talaria/pull/127
• ingest add metrics ingested rows by @atlas-comstock in https://github.com/talariadb/talaria/pull/129
• Feat/nats multi table support by @jeffreylean in https://github.com/talariadb/talaria/pull/133

New Contributors

• @Juneezee made their first contribution in https://github.com/talariadb/talaria/pull/101
• @jeffreylean made their first contribution in https://github.com/talariadb/talaria/pull/106
• @dependabot made their first contribution in https://github.com/talariadb/talaria/pull/125

Full Changelog: https://github.com/talariadb/talaria/compare/v1.6.0...v1.6.1

What's Changed

• Refactor Sink Writer to accept blocks inplace of bytes by @tardunge in https://github.com/kelindar/talaria/pull/86

Full Changelog: https://github.com/kelindar/talaria/compare/v1.5.1...v1.6.0

What's Changed

• Fix issue #88: Use env to set Azure auth if existed by @atlas-comstock in https://github.com/kelindar/talaria/pull/91

Full Changelog: https://github.com/kelindar/talaria/compare/v1.5.0...v1.6.0

What's Changed

• adding go plugin as a new method in Computed by @atlas-comstock in https://github.com/kelindar/talaria/pull/87

New Contributors

• @atlas-comstock made their first contribution in https://github.com/kelindar/talaria/pull/87

Full Changelog: https://github.com/kelindar/talaria/compare/v1.4.0...v1.5.0

Changelog

• Added support for managed identities on Azure and improved monitoring https://github.com/kelindar/talaria/pull/85

This release fixes the protobuf errors that occur with the new gRPC.

Changelog

• Gogo protobuf is incompatible with grpc-go API/v2.
• Replaced proto-gen-gogo-slick with proto-gen-go-grpc and proto-gen-go-vtproto plugins.
• Modified python code generation script.

This release upgrades to badger/v3 which has a lot of memory-related changes under the hood. See https://github.com/dgraph-io/badger/blob/master/CHANGELOG.md

Changelog

• (Breaking) Rebasing on badger/v3 which makes data incompatible https://github.com/kelindar/talaria/pull/78

This releases introduces fixes of parquet ingestion and expands the number of keys that can be added as partition keys.

Changelog

• Expand the range of data types that can be used for hashBy column https://github.com/kelindar/talaria/pull/75
• Fix parquet ingestion for missing columns https://github.com/kelindar/talaria/pull/77

This release includes azure weighted storage when using multi-account (https://github.com/kelindar/talaria/pull/70) and parquet encoder (egress) https://github.com/kelindar/talaria/pull/67

Changelog

• Added support for writing to Azure with weighted choice
• Added support for parquet encoding when writing data out of Talaria during compaction
• Added an entry point to allow loading env variable and potentially and generic init procedure within the container. I believe this is useful because if you are using something like Vault to store your secrets, the secrets are mounted as files within the container. With this entry point it will be able to load those secrets.
• Updated Docker build pattern, I noticed there was the attempt to have a "build" pattern however it was commented out. I believe having the Dockerfile in this way is better because people can test easily locally to see if the image builds correctly.

This release contains major throughput improvements to Azure writer and Talaria now writes to all sinks in parallel. See #66 for more details. Special thanks to @qiaowei-g and @ocassetti for this PR.

Changelog

• Add Azure Multi Storage Account Writer, use Azure AD credential for authentication.
• Change multi.writer to call writers in parallel.
• Add Datadog metric for writer latency and error stats.