Questdb Versions Save

An open source time-series database for fast ingest and SQL queries

7.4.2

1 week ago

Choice is essential. But isn't it nice when a clear, happy path emerges?

To that tune, we're pleased to announce QuestDB 7.4.2, with the usual mix of features, fixes and improvements.

In focus this week is the refinement of our updated ingestion clients. Built atop the InfluxDB Line Protocol (ILP) and leveraging HTTP instead of TCP, our clients and core database now offer a primary ingestion path that is robust, responsive and fast. They're available in popular languages, including Rust, Python, Java, Go, C & C++, and more.

It is quicker, cleaner and clearer to both get started and refine.

New Features 🐣

Fresh goods.

  • Added bind variable support for long256 and IPv4 operators, milliseconds and microseconds support to datediff and dateadd.

  • Enabled FILL(LINEAR) to support ALIGN TO CALENDAR.

Performance 🚀

Always a category unto its own.

  • ILP: Optimized data ingestion for tables with a large number of columns.

  • Core: Improved hash table lookup for small string keys and optimized memory usage to reduce OS kills.

  • SQL: Faster SQL COPY and REST API CSV import, optimized binary search corner cases, and fixed the stuck parallel GROUP BY queries via better query timeout handling.

Bug Fixes 🥾

Squished, squashed & kiboshed.

  • ILP: Addressed 'maximum buffer size exceeded' in ILP client and made config adjustments to disable individual flush types and support auto_flush_bytes. also corrected client processing of data received before TLS close_notification.

  • SQL: Fixed incorrect results from lower, upper, substring functions when executed in parallel, erroneous SAMPLE BY behavior mixing asterisks, errors in CASE statements when ELSE branch is missing, incorrect results in GROUP BY queries, and SAMPLE BY queries with functions in the SELECT clause.

    Also fixed a bug for negative upper bound in BETWEEN operator, issues with null string insertions as designated timestamps, and inaccurate double comparisons. Finally, resolved stuck parallel group by queries due to incorrect timeout handling.

  • Core: Addressed a rare segmentation fault during o3 commit of a non-WAL table, potential error rebuilding nascent data index, and incorrect partition size after an update.

New Contributors 🎉

@hoangsvit and @sivukhin made their first contributions. Thanks, both of you. 🙏

Pull requests

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/7.4.0...7.4.2

7.4.1

1 week ago

What's Changed

We discovered SQL regression in 7.4.1. This regression does not effect any other version. This release is superceeded by 7.4.2

Full Changelog: https://github.com/questdb/questdb/compare/7.4.0...7.4.1

7.4.0

1 month ago

🌸 QuestDB 7.4 Release Notes

Ahh, Spring! Sunshine, flowers, all that nice stuff.

As the wheels of the seasons turn, so to the wheels of the applications we're all busy building.

Hopefully good things bloom for us all.

QuestDB is pleased to release version 7.4!

As you've come to expect, this release further improves core database performance. Also as per usual, several bugs and rough edges have been addressed. In important addition, there are note-able breaking changes that will provide a smoother overall experience and set us up well for future changes. Please address them prior to upgrade, and reach out to us if you run into any issues.

⚠️ New Features with breaking changes

  • SAMPLE BY queries now use ALIGN TO CALENDAR by default. As such, if you're using SAMPLE BY and expecting ALIGN TO FIRST OBSERVATION, you'll need to update your queries to specify that syntax.

  • Java client: The ILP Sender used in our first-party Java client now requires an explicit selection of Transport, (HTTP or TCP) for enhanced reliability and clarity in data ingestion workflows. You will need to pass this value forward. See the PR for more information. If you do not use the Java client, but use another language client, see your respective language client repository and update if needed.

  • CREATE TABLE AS and INSERT INTO SELECT statements now operate non-atomically and are batched by default. If you require an atomic table, you can now use the very powerful sounding: CREATE ATOMIC TABLE. For more information, checkout the docs.

🚀 Performance Improvements

This release improves parallel GROUP BY query performance and also makes SAMPLE BY queries more efficient. In addition, we've reduced thread-local allocator overhead in single-threaded GROUP BY & SAMPLE BY queries. The first/last aggregate functions for string columns have also been optimized.

🐛 Bug Fixes

  • Resolved issues leading to segmentation faults and incorrect results in certain SQL queries involving GROUP BY and SAMPLE BY clauses.
  • Addressed a bug that could cause a table to be omitted from snapshots after significant DDL changes.
  • Fixed HTTP request handling to prevent rejections when content-length is set to 0.
  • Solved potential TLS handshake stalls on MacOS.
  • Corrected the behavior of altering tables to add indexes that could inadvertently remove deduplication keys.
  • Eliminated a crash during ingestion and various other stability issues.

What's Changed

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/7.3.10...7.4.0

7.3.10

2 months ago

Fresh from QuestDB 🐲

The 2024 Lunar New Year marks the Year of the Dragon. 🐉 When you think of the mighty dragon, what comes to mind? For us, well, we're a little different. Dragons evoke strength, wisdom, and - naturally - wild performance (we're performance obsessed).

Within that theme, the 7.3.10 release offers a 5x-10x speed-up in ASOF and LT JOIN queries, as well as a number of optimizations in parallel filters, GROUP BY, and within the JIT compiler. Further, UUID and LONG256 columns are now supported by parallel GROUP BY for efficient use of all hardware resources. For workloads that feature small, but frequent transactions you'll also see improved performance.

Familiar with InfluxDB? Interested in using InfluxDB Line Protocol (ILP), but with something more robust under-the-hood? This release provides 5x faster ingestion performance when multiple connections are in use, further increasing overall throughput strength compared to InfluxDB, especially in data with high cardinality. Whether you're looking to leverage ILP for high performance streaming, or upgrading an existing InfluxDB workload over to QuestDB, this update makes it easier.

Read more about "drop-in" InfluxDB migration in our recent blog.

Alongside the usual battery of bug fixes, this release also includes a new HyperLogLog-based approx_count_distinct() SQL function that is 3x-5x faster and more memory efficient than the existing count_distinct() function for high cardinality data sets. And finally, Write-Ahead Log (WAL) enabled tables are now the default table type. This means that the benefits of WAL & WAL-required features like deduplication are available to each newly created table.

Whether you're new to QuestDB, working in prod, or migrating existing time series workloads, this release has something for you.

Performance improvements

New features

Fixes and other improvements

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/7.3.9...7.3.10

7.3.8b

4 months ago

Welcome back from the holidays! (If you had one...)

In the latest QuestDB release, we've focused on enhanced stability and introduced two innovative features, currently in beta. Notably, we've improved GROUP BY performance tenfold through parallel SQL executions and introduced InfluxDB Line Protocol (ILP) support over HTTP, which is compatible with existing InfluxDB drivers. Those familiar with ILP will find a clean entryway into QuestDB.

Thanks to @ssharaev and @pompeiifreckles for their first contributions - we appreciate you.

Dive in to explore the release in detail. 🤿

What's changed

Please note that the following features are in beta - please let us know if you run into any issues.

  • 10x GROUP BY performance improvement via parallel SQL executions. Should you encounter any problems with this implementation, you can safely disabled [cairo.sql.parallel.groupby.enabled=false] it via configuration without downgrading the database instance.
  • InfluxDB Line Protocol (ILP) support over HTTP. This implementation supports existing InfluxDB drivers and eases the transition into QuestDB for existing ILP users. The new client has control over transaction boundaries and will offer helpful feedback on transaction activity. We're also pleased to offer no noticeable performance impact in ingestions performance compared to our TCP implementation.

New features

Fixes and improvements

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/7.3.7...7.3.8b

7.3.9

4 months ago

Welcome back from the holidays! (If you had one...)

In the latest QuestDB release, we've focused on enhanced stability and introduced two innovative features. Notably, we've improved GROUP BY performance tenfold through parallel SQL executions and introduced InfluxDB Line Protocol (ILP) support over HTTP, which is compatible with existing InfluxDB drivers. Those familiar with ILP will find a clean entryway into QuestDB.

Thanks to @ssharaev and @pompeiifreckles for their first contributions - we appreciate you.

Dive in to explore the release in detail. 🤿

What's changed

  • 10x GROUP BY performance improvement via parallel SQL executions. Should you encounter any problems with this implementation, you can safely disabled [cairo.sql.parallel.groupby.enabled=false] it via configuration without downgrading the database instance.
  • InfluxDB Line Protocol (ILP) support over HTTP. This implementation supports existing InfluxDB drivers and eases the transition into QuestDB for existing ILP users. The new client has control over transaction boundaries and will offer helpful feedback on transaction activity. We're also pleased to offer no noticeable performance impact in ingestions performance compared to our TCP implementation.

New features

Fixes and improvements

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/7.3.7...7.3.8

7.3.7

4 months ago

What's Changed

The hits keep comin'. Newly released QuestDB 7.3.7 adds supporting functions to pair with our recently released window functions. It also offers brings further improvements to SQL and ingestion performance - this has been a pattern of late!

Additional key features include:

  • A 5x speed up of WAL Apply for small transactions
  • New built-in functions for covariance, correlation, cumulative sum, and first value window
  • Enhanced compatibility with Grafana

Thanks to @AjCH1 and @ksankeerth for their first contributions!

New features

Fixes and improvements

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/7.3.5...7.3.7

7.3.5

5 months ago

What's Changed

Following up our recent 7.3.4 release which provided Window Functions, Web Console improvements, and more, we're pleased to announce another significant release. Headlining 7.3.5 are the following key additions:

  • Percentile function: Find an approximate percentile for a given series with approx_percentile(DD).
  • Optimized queries: QuestDB has long stood out for its high performance ingestion. Not be left behind, queries also keep getting faster. In benchmarks, recent performance optimizations have shown 50% improvement to several query types.

New features

Fixes and improvements

7.3.4

5 months ago

What's Changed

Better. Faster. Stronger. And even more good looking. 😎

Highlighting our latest release:

  • Window functions. These flexible functions provide moving averages, running totals, or Top-N results within a group, and more via a "window", defined by an OFFSET against a RANGE or ROWS. Unlike aggregate functions, window functions do not cause rows to become grouped into a single output row — the rows retain their separate identities. Additional perks? Performance and an easing of query complexity.

  • More functions. We've added:first(boolean), last(boolean), first(string), last(string), mathematical exponent exp(D) and first_not_null(), last_not_null(), which operate on most nullable data types, except IPv4 and GeoHash.

  • Performance. Our goal is top performance. LIKE/ILIKE queries now perform over 80% faster.

  • Web Console Refresh. The best tools provide the help that you need without getting in your way. We've tuned the Web Console UI to put what you need closer to where you need it. Less used UI elements have had their significance reduced and overall navigation has been streamlined. On top of that, a discrete news feed will send you all the latest, so that you don't miss a beat.

Before:

Screenshot 2023-11-07 at 1 00 40 PM

After:

web-console-after

New features

UI

Improvements and bug fixes

Documentation

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/7.3.3...7.3.4

7.3.3

6 months ago

Overview

Thanks for all who contributed to our most recent release.

This release provides a set of stability fixes and usability improvements.

We are excited to highlight the following areas:

  • Improved operations: A trio of features including improved snapshots to replace existing backup functionality, connection count gauge metrics and a new mkdir.mode for detached partitions make QuestDB easier to operate.

  • InfluxDB Line Protocol improvements: The InfluxDB Line Protocol is commonly used for rapid ingest. We’ve made it better, by introducing more flexible and less error prone timestamps. In addition, it also authenticates faster and disconnects on error when using WAL tables.

  • New aggregate functions: Thanks to community member @charlespnh QuestDB now offers variance(), var_samp() and var_pop(), stddev(), stddev_pop() functions.

What is new

CORE

SQL

ILP

What is fixed

CORE

SQL

ILP

HTTP

PGWire

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/7.3.2...7.3.3