An open source time-series database for fast ingest and SQL queries
Choice is essential. But isn't it nice when a clear, happy path emerges?
To that tune, we're pleased to announce QuestDB 7.4.2, with the usual mix of features, fixes and improvements.
In focus this week is the refinement of our updated ingestion clients. Built atop the InfluxDB Line Protocol (ILP) and leveraging HTTP instead of TCP, our clients and core database now offer a primary ingestion path that is robust, responsive and fast. They're available in popular languages, including Rust, Python, Java, Go, C & C++, and more.
It is quicker, cleaner and clearer to both get started and refine.
Fresh goods.
Added bind variable support for long256 and IPv4 operators, milliseconds and microseconds support to datediff and dateadd.
Enabled FILL(LINEAR) to support ALIGN TO CALENDAR.
Always a category unto its own.
ILP: Optimized data ingestion for tables with a large number of columns.
Core: Improved hash table lookup for small string keys and optimized memory usage to reduce OS kills.
SQL: Faster SQL COPY and REST API CSV import, optimized binary search corner cases, and fixed the stuck parallel GROUP BY queries via better query timeout handling.
Squished, squashed & kiboshed.
ILP: Addressed 'maximum buffer size exceeded' in ILP client and made config adjustments to disable individual flush types and support auto_flush_bytes. also corrected client processing of data received before TLS close_notification.
SQL: Fixed incorrect results from lower, upper, substring functions when executed in parallel, erroneous SAMPLE BY behavior mixing asterisks, errors in CASE statements when ELSE branch is missing, incorrect results in GROUP BY queries, and SAMPLE BY queries with functions in the SELECT clause.
Also fixed a bug for negative upper bound in BETWEEN operator, issues with null string insertions as designated timestamps, and inaccurate double comparisons. Finally, resolved stuck parallel group by queries due to incorrect timeout handling.
Core: Addressed a rare segmentation fault during o3 commit of a non-WAL table, potential error rebuilding nascent data index, and incorrect partition size after an update.
@hoangsvit and @sivukhin made their first contributions. Thanks, both of you. 🙏
Full Changelog: https://github.com/questdb/questdb/compare/7.4.0...7.4.2
We discovered SQL regression in 7.4.1. This regression does not effect any other version. This release is superceeded by 7.4.2
Full Changelog: https://github.com/questdb/questdb/compare/7.4.0...7.4.1
Ahh, Spring! Sunshine, flowers, all that nice stuff.
As the wheels of the seasons turn, so to the wheels of the applications we're all busy building.
Hopefully good things bloom for us all.
QuestDB is pleased to release version 7.4!
As you've come to expect, this release further improves core database performance. Also as per usual, several bugs and rough edges have been addressed. In important addition, there are note-able breaking changes that will provide a smoother overall experience and set us up well for future changes. Please address them prior to upgrade, and reach out to us if you run into any issues.
SAMPLE BY
queries now use ALIGN TO CALENDAR
by default. As such, if you're using SAMPLE BY
and expecting ALIGN TO FIRST OBSERVATION
, you'll need to update your queries to specify that syntax.
Java client: The ILP Sender used in our first-party Java client now requires an explicit selection of Transport, (HTTP or TCP) for enhanced reliability and clarity in data ingestion workflows. You will need to pass this value forward. See the PR for more information. If you do not use the Java client, but use another language client, see your respective language client repository and update if needed.
CREATE TABLE AS
and INSERT INTO SELECT
statements now operate non-atomically and are batched by default. If you require an atomic table, you can now use the very powerful sounding: CREATE ATOMIC TABLE
. For more information, checkout the docs.
This release improves parallel GROUP BY
query performance and also makes SAMPLE BY
queries more efficient. In addition, we've reduced thread-local allocator overhead in single-threaded GROUP BY
& SAMPLE BY
queries. The first/last aggregate functions for string columns have also been optimized.
sample by
implementation by @bluestreak01 in https://github.com/questdb/questdb/pull/4259
Full Changelog: https://github.com/questdb/questdb/compare/7.3.10...7.4.0
The 2024 Lunar New Year marks the Year of the Dragon. 🐉 When you think of the mighty dragon, what comes to mind? For us, well, we're a little different. Dragons evoke strength, wisdom, and - naturally - wild performance (we're performance obsessed).
Within that theme, the 7.3.10 release offers a 5x-10x speed-up in ASOF
and LT JOIN
queries, as well as a number of optimizations in parallel filters, GROUP BY
, and within the JIT compiler. Further, UUID
and LONG256
columns are now supported by parallel GROUP BY
for efficient use of all hardware resources. For workloads that feature small, but frequent transactions you'll also see improved performance.
Familiar with InfluxDB? Interested in using InfluxDB Line Protocol (ILP), but with something more robust under-the-hood? This release provides 5x faster ingestion performance when multiple connections are in use, further increasing overall throughput strength compared to InfluxDB, especially in data with high cardinality. Whether you're looking to leverage ILP for high performance streaming, or upgrading an existing InfluxDB workload over to QuestDB, this update makes it easier.
Read more about "drop-in" InfluxDB migration in our recent blog.
Alongside the usual battery of bug fixes, this release also includes a new HyperLogLog-based approx_count_distinct()
SQL function that is 3x-5x faster and more memory efficient than the existing count_distinct()
function for high cardinality data sets. And finally, Write-Ahead Log (WAL) enabled tables are now the default table type. This means that the benefits of WAL & WAL-required features like deduplication are available to each newly created table.
Whether you're new to QuestDB, working in prod, or migrating existing time series workloads, this release has something for you.
Full Changelog: https://github.com/questdb/questdb/compare/7.3.9...7.3.10
Welcome back from the holidays! (If you had one...)
In the latest QuestDB release, we've focused on enhanced stability and introduced two innovative features, currently in beta. Notably, we've improved GROUP BY
performance tenfold through parallel SQL executions and introduced InfluxDB Line Protocol (ILP) support over HTTP, which is compatible with existing InfluxDB drivers. Those familiar with ILP will find a clean entryway into QuestDB.
Thanks to @ssharaev and @pompeiifreckles for their first contributions - we appreciate you.
Dive in to explore the release in detail. 🤿
Please note that the following features are in
beta
- please let us know if you run into any issues.
GROUP BY
performance improvement via parallel SQL executions. Should you encounter any problems with this implementation, you can safely disabled [cairo.sql.parallel.groupby.enabled=false]
it via configuration without downgrading the database instance.nullif(double, double)
SQL function by @ssharaev in https://github.com/questdb/questdb/pull/4063
truncate
& table corruption after restore from snapshot by @bluestreak01 in https://github.com/questdb/questdb/pull/4095
Full Changelog: https://github.com/questdb/questdb/compare/7.3.7...7.3.8b
Welcome back from the holidays! (If you had one...)
In the latest QuestDB release, we've focused on enhanced stability and introduced two innovative features. Notably, we've improved GROUP BY
performance tenfold through parallel SQL executions and introduced InfluxDB Line Protocol (ILP) support over HTTP, which is compatible with existing InfluxDB drivers. Those familiar with ILP will find a clean entryway into QuestDB.
Thanks to @ssharaev and @pompeiifreckles for their first contributions - we appreciate you.
Dive in to explore the release in detail. 🤿
GROUP BY
performance improvement via parallel SQL executions. Should you encounter any problems with this implementation, you can safely disabled [cairo.sql.parallel.groupby.enabled=false]
it via configuration without downgrading the database instance.nullif(double, double)
SQL function by @ssharaev in https://github.com/questdb/questdb/pull/4063
truncate
& table corruption after restore from snapshot by @bluestreak01 in https://github.com/questdb/questdb/pull/4095
Full Changelog: https://github.com/questdb/questdb/compare/7.3.7...7.3.8
The hits keep comin'. Newly released QuestDB 7.3.7 adds supporting functions to pair with our recently released window functions. It also offers brings further improvements to SQL and ingestion performance - this has been a pattern of late!
Additional key features include:
Thanks to @AjCH1 and @ksankeerth for their first contributions!
information_schema.columns()
function to provide list of columns across all tables in the database by @marregui in https://github.com/questdb/questdb/pull/3975
Full Changelog: https://github.com/questdb/questdb/compare/7.3.5...7.3.7
Following up our recent 7.3.4
release which provided Window Functions, Web Console improvements, and more, we're pleased to announce another significant release. Headlining 7.3.5
are the following key additions:
approx_percentile(DD)
.show server_version
by @nitram509 in https://github.com/questdb/questdb/pull/3901
show parameters
SQL by @bluestreak01 in https://github.com/questdb/questdb/pull/3972
Better. Faster. Stronger. And even more good looking. 😎
Highlighting our latest release:
Window functions. These flexible functions provide moving averages, running totals, or Top-N results within a group, and more via a "window", defined by an OFFSET
against a RANGE
or ROWS
. Unlike aggregate functions, window functions do not cause rows to become grouped into a single output row — the rows retain their separate identities. Additional perks? Performance and an easing of query complexity.
More functions. We've added:first(boolean)
, last(boolean)
, first(string)
, last(string)
, mathematical exponent exp(D)
and first_not_null()
, last_not_null()
, which operate on most nullable data types, except IPv4 and GeoHash.
Performance. Our goal is top performance. LIKE
/ILIKE
queries now perform over 80% faster.
Web Console Refresh. The best tools provide the help that you need without getting in your way. We've tuned the Web Console UI to put what you need closer to where you need it. Less used UI elements have had their significance reduced and overall navigation has been streamlined. On top of that, a discrete news feed will send you all the latest, so that you don't miss a beat.
Before:
After:
short
columns by @bluestreak01 (#3924)name
to table_name
in cursor function tables by @marregui in https://github.com/questdb/questdb/pull/3857
Full Changelog: https://github.com/questdb/questdb/compare/7.3.3...7.3.4
Thanks for all who contributed to our most recent release.
This release provides a set of stability fixes and usability improvements.
We are excited to highlight the following areas:
Improved operations: A trio of features including improved snapshots to replace existing backup functionality, connection count gauge metrics and a new mkdir.mode
for detached partitions make QuestDB easier to operate.
InfluxDB Line Protocol improvements: The InfluxDB Line Protocol is commonly used for rapid ingest. We’ve made it better, by introducing more flexible and less error prone timestamps. In addition, it also authenticates faster and disconnects on error when using WAL tables.
New aggregate functions: Thanks to community member @charlespnh QuestDB now offers variance()
, var_samp()
and var_pop()
, stddev()
, stddev_pop()
functions.
mkdir.mode
permission config for detached partitions by @debanjanc01 in https://github.com/questdb/questdb/pull/3818
Full Changelog: https://github.com/questdb/questdb/compare/7.3.2...7.3.3