An open source time-series database for fast ingest and SQL queries
This is the Beta release of the Write Ahead Log (WAL) table storage model #2951.
We are always listening to our users, and took to heart your problem of ingestion slowdown with heavy out-of-order data flow. This release provides a comprehensive solution to that.
The reasons why you should switch to WAL are:
The WAL storage model is inactive by default. To create a WAL-based table, the following syntax should be used (note WAL
at the end):
create table ABC(x int, t timestamp) timestamp(t) partition by day WAL;
You can also convert your existing table to WAL (followed by an instance restart). The conversion touches table configuration (not data!) and is fully reversible.
To convert an existing table to WAL:
alter table ABC set type WAL;
To convert a WAL table to non-WAL:
alter table ABC set type bypass WAL;
Full Changelog: https://github.com/questdb/questdb/compare/6.7...7.0.0
As of this release, it is strongly discouraged to modify or delete table directories in the db
directory or delete table directories while the database instance is still running. It is still safe to do those activities while database is shut down. (https://github.com/questdb/questdb/pull/2752)
If multiple database instances share the same db
directory, you will now have to declare all but one of them as read-only instances via their configuration. (https://github.com/questdb/questdb/pull/2752)
sin
, cos
, tan
, cot
, asin
, acos
, atan
, atan2
, toRadians
, fromRadians
, pi
by @marregui in https://github.com/questdb/questdb/pull/2890
explain
SQL statement by @bziobrowski in https://github.com/questdb/questdb/pull/2680
group by
, sample by
and other queries by @puzpuzpuz in https://github.com/questdb/questdb/pull/2903
UUID
column type by @jerrinot in https://github.com/questdb/questdb/pull/2769
long256
comparisons on different columns of the same table @jerrinot https://github.com/questdb/questdb/pull/2924
Full Changelog: https://github.com/questdb/questdb/compare/6.6.1...6.7
This is a follow up release for the 6.6. It fixes a serious data consistency issue, introduced specifically by 6.6.
Unless you're upgrading from versions before 6.6, you will be affected by a minor breaking change: commitLag
keyword in SQLs is no longer supported in this release. We recommend you remove these keywords and "lag" values and let the system handle things automatically.
Full Changelog: https://github.com/questdb/questdb/compare/6.6...6.6.1
We introduce an automated system, which removes the concept of commit lag
thus simplifying capacity planning. Developers don't have to compromise between high ingestion speed and data visibility. Data visibility is instant while ingestion performance is always at its peak.
Additionally, there is a number of important bug fixes that improve system performance and stability.
commitLag
keyword in SQLs is no longer supported in this release. We recommend you remove these keywords and "lag" values and let the system handle things automatically.
ltrim()
, rtrim()
and trim()
functions by @klurpicolo in https://github.com/questdb/questdb/pull/2613
LATEST ON
with multiple symbol columns by @puzpuzpuz in https://github.com/questdb/questdb/pull/2783
LogFactory
instances on start-up by @puzpuzpuz in https://github.com/questdb/questdb/pull/2756
long256
could not be parsed by @vgrichina in https://github.com/questdb/questdb/pull/2762
could not mmap
error by @glasstiger in https://github.com/questdb/questdb/pull/2775
text_loader.json
configuration file by @puzpuzpuz in https://github.com/questdb/questdb/pull/2750
WhereClauseParser
when using epoch by @stephenspol in https://github.com/questdb/questdb/pull/2744
We want to thank our new contributors:
Full Changelog: https://github.com/questdb/questdb/compare/6.5.5...6.6
This release features an improved migration experience when importing CSV files from PostgreSQL and stability improvements for the Core and SQL engine.
U+
- six-digit microsecond format component by @bluestreak01 in https://github.com/questdb/questdb/pull/2688
N+
- nine-digit nanosecond format component by @bluestreak01 in https://github.com/questdb/questdb/pull/2709
insert into table with ...
syntax by @bluestreak01 in https://github.com/questdb/questdb/pull/2724
joins
by @bluestreak01 in https://github.com/questdb/questdb/pull/2685
We want to thank our new contributors:
Full Changelog: https://github.com/questdb/questdb/compare/6.5.4...6.5.5
This release includes new SQL features from our amazing Hacktoberfest contributors as well as performance and stability improvements.
string functions:
lower()
and upper()
string functions by @blackstar-baba in https://github.com/questdb/questdb/pull/2582
position()
alias for string function strpos()
by @DarthBenro008 in https://github.com/questdb/questdb/pull/2599
starts_with
string function by @akki1306 in https://github.com/questdb/questdb/pull/2608
lpad
and rpad
string functions by @eigengravy in https://github.com/questdb/questdb/pull/2631
split_part()
by @daniel-shuy in https://github.com/questdb/questdb/pull/2644
numeric functions:
ceiling
with double and float arguments by @bsnxsourav in https://github.com/questdb/questdb/pull/2600
log10
math function to calculate base 10 logarithm of double argument by @blackstar-baba in https://github.com/questdb/questdb/pull/2579
log()
to ln()
for natural logarithm function by @Ln11211 in https://github.com/questdb/questdb/pull/2605
others:
CREATE TABLE LIKE
syntax by @akki1306 in https://github.com/questdb/questdb/pull/2580
ALTER DROP
of the last partition by @marregui in https://github.com/questdb/questdb/pull/2606
row_number() over()
and row_number() over(ORDER BY)
by @blackstar-baba in https://github.com/questdb/questdb/pull/2656
We want to thank our new contributors:
Full Changelog: https://github.com/questdb/questdb/compare/6.5.3...6.5.4
This is a maintenance release, which includes important stability fixes and improvements for the SQL engine and core.
DEALLOCATE
statement support for PHP Data Objects by @puzpuzpuz in https://github.com/questdb/questdb/pull/2530
GROUP BY
over empty table by @puzpuzpuz in https://github.com/questdb/questdb/pull/2535
FROM
in SELECT *
by @lookharm in https://github.com/questdb/questdb/pull/2516
SAMPLE BY
with invalid values by @puzpuzpuz in https://github.com/questdb/questdb/pull/2540
rollback()
that may lead to incorrect writes or crashes by @ideoma in https://github.com/questdb/questdb/pull/2512
We want to thank our new contributor:
Full Changelog: https://github.com/questdb/questdb/compare/6.5.2...6.5.3
This is a maintenance release focusing on important bug fixes for 6.5.1.
min
/max
and regexp_replace
functions for string values by @puzpuzpuz in https://github.com/questdb/questdb/pull/2478
GROUP BY
based on function key by @puzpuzpuz in https://github.com/questdb/questdb/pull/2484
WHERE
execution logic by @glasstiger in #2492ALTER TABLE
syntax error message by @puzpuzpuz in #2493Full Changelog: https://github.com/questdb/questdb/compare/6.5.1...6.5.2
This release adds new SQL statements for detaching and re-attaching partitions to tables, enabling cold storage as a solution to the disk size limitation. In addition, we include several fixes and improvements for the overall reliability of the SQL engine.
ALTER TABLE DETACH PARTITION
as a complement to ATTACH
by @marregui in https://github.com/questdb/questdb/pull/2280
count_distinct()
by @puzpuzpuz in https://github.com/questdb/questdb/pull/2439
table_writer_metrics()
and memory_metrics()
table functions for better observability by @jerrinot https://github.com/questdb/questdb/pull/2318/
ORDER BY
in over clause not accepting direction by @bziobrowski in https://github.com/questdb/questdb/pull/2406
min()
on int/long/date/ts column with max values only by @puzpuzpuz in https://github.com/questdb/questdb/pull/2429
SAMPLE BY
with filter by @puzpuzpuz in https://github.com/questdb/questdb/pull/2448
ORDER BY DESC
with async filter by @bziobrowski in https://github.com/questdb/questdb/pull/2451
Duplicate column
exception in INTERSECT query by @eugenels in https://github.com/questdb/questdb/pull/2472
Full Changelog: https://github.com/questdb/questdb/compare/6.5...6.5.1
This release brings a new algorithm for the SQL COPY
statement [Reference], which is specifically designed to enable fast and easy CSV Import for large and unsorted data files. This feature comes in handy when there's a need to migrate a large amount of data from other databases to QuestDB [Guide: CSV import]. In addition, there are several improvements and fixes for the Influx line protocol, Core, and SQL engine.
No gotchas when you are upgrading from the start or 6.3, as it is just a software update. If you're upgrading from 6.2.1 or earlier, please refer to the gotchas of 6.3
hs_err_pid
crash logs into db
directory with Docker package by @bluestreak01 in https://github.com/questdb/questdb/pull/2346
COPY
statement for unordered CSV files by @eugenels, @bziobrowski in https://github.com/questdb/questdb/pull/2155
substring
to extract some characters from a string by @HeZean in https://github.com/questdb/questdb/pull/2251
CREATE TABLE
SQL by @ideoma in https://github.com/questdb/questdb/pull/2335
error
and critical
log messages by @ideoma in https://github.com/questdb/questdb/pull/2345
chown
errors by @mariusgheorghies in https://github.com/questdb/questdb/pull/2354
LIMIT
query with indexed symbol filter by @puzpuzpuz in https://github.com/questdb/questdb/pull/2336
ORDER BY
priority in case of duplicate column names by @jerrinot in https://github.com/questdb/questdb/pull/2339
INSERT
SQL by @bluestreak01 in https://github.com/questdb/questdb/pull/2366
int
column with high cardinality @bziobrowski https://github.com/questdb/questdb/pull/2353
boolean
numeric representation @Zahlii https://github.com/questdb/questdb/pull/2386
Full Changelog: https://github.com/questdb/questdb/compare/6.4.3...6.5