PRQL is a modern language for transforming data — a simple, powerful, pipelined SQL replacement
0.9.2 is a hotfix release to fix an issue in the 0.9.0 & 0.9.1 release pipelines.
Check out the 0.9.0 release notes for a useful changelog.
0.9.1 is a hotfix release to fix an issue in the 0.9.0 release pipeline.
Check out the 0.9.0 release notes for a useful changelog.
0.9.0 is probably PRQL's biggest ever release. We have dialect-specific standard-libraries, a regex operator, an initial implementation of multiple-file projects & modules, lots of bug fixes, and many many internal changes.
We've made a few backward incompatible syntax changes. Most queries will work with a simple find/replace; see below for details.
The release has 421 commits from 12 contributors.
A small selection of the changes:
Language:
The major breaking change is a new syntax for lists, which have been renamed to tuples, and are now represented with braces {} rather than brackets [].
To convert previous PRQL queries to this new syntax simply change [ ... ] to { ... }.
We made the syntax change to incorporate arrays. Almost every major language uses [] for arrays. We are adopting that convention — arrays use [], tuples will use {}. (Though we recognize that {} for tuples is also rare (Hi, Erlang!), but didn't want to further load parentheses with meaning.)
Arrays are conceptually similar to columns — their elements have a single type. Array syntax can't contain assignments.
As part of this, we've also formalized tuples as containing both individual items (select {foo, baz}), and assignments (select {foo=bar, baz=fuz}).
Some significant changes regarding SQL dialects:
sql.duckdb target supports DuckDB 0.8 (@eitsupi, #2810).sql.hive target is removed (@eitsupi, #2837).New arithmetic operators. These compile to different function or operator depending on the target.
Breaking: Operator / now always performs floating division (@aljazerzen, #2684). See the Division docs for details.
Truncated integer division operator // (@aljazerzen, #2684). See the Division docs for details.
Regex search operator ~= (@max-sixty, #2458). An example:
from tracks
filter {name ~= "Love"}
...compiles to;
SELECT
*
FROM
tracks
WHERE
REGEXP(name, 'Love')
...though the exact form differs by dialect; see the Regex docs for more details.
New aggregation functions: every, any, average, and concat_array. Breaking: Remove avg in favor of average.
Breaking: We've changed our function declaration syntax to match other declarations. Functions were one of the first language constructs in PRQL, and since then we've added normal declarations there's no compelling reason for functions to be different.
let add = a b -> a + b
Previously, this was:
func add a b -> a + b
Experimental modules, which allow importing declarations from other files. Docs are forthcoming.
Relation literals create a relation (a "table") as an array of tuples. This example demonstrates the new syntax for arrays [] and tuples {}. (@aljazerzen, #2605)
from [{a=5, b=false}, {a=6, b=true}]
filter b == true
select a
this can be used to refer to the current pipeline, for situations where plain column name would be ambiguous:
from x
derive sum = my_column
select this.sum # does not conflict with `std.sum`
Within a join transform, there is also a reference to the right relation: that.
Breaking: functions count, rank and row_number now require an argument of the array to operate on. In most cases you can directly replace count with count this. The non_null argument of count has been removed.
Features:
We've changed how we handle colors.
Options::color is deprecated and has no effect. Code which consumes prql_compiler::compile should instead accept the output with colors and use a library such as anstream to handle the presentation of colors. To ensure minimal disruption, prql_compiler will currently strip color codes when a standard environment variable such as CLI_COLOR=0 is set or when it detects stderr is not a TTY.
We now use the anstream library in prqlc & prql-compiler.
(@max-sixty, #2773)
prqlc can now show backtraces when the standard backtrace env var (RUST_BACKTRACE) is active. (@max-sixty, #2751)
Fixes:
1e9 — are now handled correctly by the compiler (@max-sixty, #2865).Integrations:
Internal changes:
Annotations in PRQL. These have limited support but are currently used to specify binding strengths. They're modeled after Rust's annotations, but with @ syntax, more similar to traditional decorators. (#2729)
@{binding_strength=11}
let mod = l r -> s"{l} % {r}"
Remove BigQuery's special handling of quoted identifiers, now that our module system handles its semantics (@max-sixty, #2609).
ClickHouse is tested in CI (@eitsupi, #2815).
New Contributors:
0.8.1 is a small release with a new list-targets command in prqlc, some documentation improvements, and some internal improvements.
This release has 41 commits from 8 contributors.
From the broader perspective of the project, we're increasing the relative prioritization of it being easy for folks to actually use PRQL — either with existing tools, or a tool we'd build. We'll be thinking about & discussing the best way to do that over the next few weeks.
0.8.0 renames the and & or operators to && & || respectively, reorganizes the Syntax section in the book, and introduces read_parquet & read_csv functions for reading files with DuckDB.
This release has 38 commits from 8 contributors. Selected changes:
Features:
Rename and to && and or to ||. Operators which are symbols are now consistently infix, while "words" are now consistently functions (@aljazerzen, #2422).
New functions read_parquet and read_csv, which mirror the DuckDB functions, instructing the database to read from files (@max-sixty, #2409).
0.7.1 is a hotfix release to fix prql-js's npm install behavior when being installed as a dependency.
This release has 17 commits from 4 contributors.
0.7.0 is a fairly small release in terms of new features, with lots of internal improvements, such as integration tests with a whole range of DBs, a blog post on Pi day, RFCs for a type system, and more robust language bindings. Our April 2023 update is attached below.
There's a very small breaking change to the rust API, hence the minor version bump.
The release has 131 commits from 10 contributors. Particular credit goes to to @eitsupi & @jelenkee, who have made significant contributions, and @vanillajonathan, whose prolific contribution include our growing language bindings.
A small selection of the changes:
Features:
prqlc compile adds --color & --include-signature-comment options. (@max-sixty, #2267)Web:
Internal changes:
compile function's Options now includes a color member, which determines whether error messages use ANSI color codes. This is technically a breaking change to the API. (@max-sixty, #2251)Error struct now exposes the MessageKind enum. (@vanillajonathan, #2307)New Contributors:
Here's our April 2023 Update, from our Readme:
April 2023 update
PRQL is being actively developed by a growing community. It's ready to use by the intrepid, either as part of one of our supported extensions, or within your own tools, using one of our supported language bindings.
PRQL still has some minor bugs and some missing features, and probably is only ready to be rolled out to non-technical teams for fairly simple queries.
Here's our current Roadmap and our Milestones.
Our immediate focus for the code is on:
- Building out the next few big features, including types and modules.
- Ensuring our supported features feel extremely robust; resolving any priority bugs.
We're also spending time thinking about:
- Making it really easy to start using PRQL. We're doing that by building integrations with tools that folks already use; for example our VS Code extension & Jupyter integration. If there are tools you're familiar with that you think would be open to integrating with PRQL, please let us know in an issue.
- Making it easier to contribute to the compiler. We have a wide group of contributors to the project, but contributions to the compiler itself are quite concentrated. We're keen to expand this; #1840 for feedback.
0.6.1 is a small release containing an internal refactoring and improved bindings for C, PHP & .NET.
This release has 54 commits from 6 contributors. Selected changes:
Fixes:
DISTINCT when a take 1 refers to a different set of columns than are in the group. (@max-sixty, with thanks to @cottrell, #2109)0.9.0 to 0.9.2. 0.9.0 has a bug that causes an infinite loop. (@eitsupi, #2110)Documentation:
Integrations:
Internal changes:
web & bindings, simplifying the repo's structure. There's also grammars & packages (@max-sixty, #2135, #2117, #2121).0.6.0 introduces a rewritten parser, giving us the ability to dramatically improve error messages, renames switch to case and includes lots of minor improvements and fixes. It also introduces loop, which compiles to WITH RECURSIVE, as a highly experimental feature.
There are a few cases of breaking changes, including switching switch to case, in case that's confusing. There are also some minor parsing changes outlined below.
This release has 108 commits from 11 contributors. Selected changes:
Features:
loop language feature, which translates to WITH RECURSIVE. We expect changes and refinements in upcoming releases. (#1642, @aljazerzen)switch function to case given it more closely matches the traditional semantics of case. (@max-sixty, #2036)case syntax to use => instead of -> to distinguish it from function syntax.\n \t).and now has a higher precedence than or (of same reason as the previous point).let, func, prql, case are now treated as keywords.1.).--format option to prqlc parse which can return the AST in YAML (@max-sixty, #1962)"sql.any". When "sql.any" is used as the target of the compile function's option, the target contained in the query header will be used. (@aljazerzen, #1995): to be elided in timezones, such as 0800 in @2020-01-01T13:19:55-0800 (@max-sixty, #1991).std.upper and std.lower functions for changing string cases (@Jelenkee, #2019).Fixes:
prqlc compile returns a non-zero exit code for invalid queries. (@max-sixty, #1924)Documentation:
Integrations:
.deb on each release. (Note that it's not yet published on each release). (@vanillajonathan, #1883)Internal changes:
New Contributors:
0.5.2 is a tiny release to fix an build issue in yesterday's prql-js 0.5.1 release.
This release has 7 commits from 2 contributors.
New Contributors: