Dataform is a framework for managing SQL based data operations in BigQuery, Snowflake, and Redshift
The dataform format
command is now dramatically improved and has better formatter output for each SQL dialect that Dataform supports. Thanks @pokutuna for the contribution!
Users of the CLI (who are otherwise opted-in to analytics tracking) may now pass --track=false
on an individual CLI command basis to disable analytics tracking.
--include-dependents
(for dataform run
) now properly includes dependents of selection actionsBoth SQLX and JavaScript now support a database()
function, which returns the (prefixed, as applicable) name of "this" action's database. (Contrived) examples:
publish("table", ctx => `select "${ctx.database()}" as col1`);
config { type: "table" }
select "${database()}" as col1
Assuming dataform.config
contains "defaultDatabase": "foo"
, these both generate a table action whose SQL is select "foo" as col1
. This is likely most useful in type: "operations"
scripts.
Note: For warehouses without databases such as Redshift, this will add an error to the compilationErrors
property.
Bugfix: The similar name()
function has been fixed to properly attach any prefix as appropriate.
Both SQLX and JavaScript now support a schema()
function, which returns the (suffixed, as applicable) name of "this" action's schema. (Contrived) examples:
publish("table", (ctx) => `select "${ctx.schema()}" as col1`);
config { type: "table" }
select "${schema()}" as col1
Assuming dataform.config
contains "defaultSchema": "foo"
, these both generate a table action whose SQL is select "foo" as col1
. This is likely most useful in type: "operations"
scripts.
This release introduces no new features, but it does change the default compilation mode to something we call main
compilation: a cleaner and more maintainable compilation script.