CUE has moved to https://github.com/cue-lang/cue
This release contains several bug fixes.
The most important bug fix, perhaps, is that it fixes a long-standing regression where commands for cue cmd were allowed to be specified in non-tool files. The current implementation will require a command to be define in at least one *_tool.cue file.
Performance fixes have been put on hold for the moment to allow some stabilization before the final v0.3.0 release.
35e19b2d cmd/cue: fix bug in -o, which would not write files for some commands 7c5d28e6 cmd/cue: fix race in testscript tests that use go/packages 81aa8cd6 cmd/export: fix help d4cb2c8d cue: add locations for disallowed field eb7b60da doc/ref/spec.md: fix ListLit production fec7a9c8 encoding/openapi: fix title handling regression bbe68e1a internal/core/adt: allow pattern constraints alongside fields 5049ab79 internal/core/adt: fix nil interface panic 1f9300d7 internal/core/adt: handle error properly for failed pattern constraint 2a937a95 internal/core/adt: improve error message dd5083f7 tools/flow: add clarifications in the API comments
This release has some important bug fixes, fixing some regressions in disjunctions and trim, mostly, among some other bug fixes.
Some of the regressions in this release slipped in because a swat of tests was accidentally disabled in the move to the new evaluator. All former tests are now reenabled and a few more corresponding issues have been resolved.
There are some performance issues remaining. We are considering leaving those in and do a v0.3.0 release, addressing these in a subsequent release. In this case, we are getting fairly close to a v0.3.0.
There will still be a beta.7 release, which addresses a regression, allowing commands in non-tool files, that we would like to address in a separate release.
91abe0de ci: fix goreleaser config
a113048d ci: move to using go1.16
f014c14c ci: set a "trybot" tag for trybot repository dispatch flow
5d3afa97 cmd/cue/cmd: correct documentation for -d flag
de1f80c5 cmd/cue/cmd: remove skip tests
304a3e0e cmd/cue/cmd: use unify semantics merging across packages
ae438126 cmd/cue: make get go operate on a best-efforts basis
cf5c48e3 cmd/cue: only ignore type and parse errors with get go
7326cfa6 cue: Value.Equal should not only consider values
830cfbc5 cue: remove build.Context reference
17ea1d5a doc/ref/spec.md: a few fixes
f5a19ef0 doc/ref/spec.md: fix escaping
3ca86804 doc/ref/spec.md: fix int_lit 0
e4d0d134 doc/ref/spec.md: remove faulty use of \x in double-quoted string
7df9fa47 doc/tutorial/basics: fix example
b51368ea doc/tutorial/kubernetes: add required go mod init step
430c3ddf encoding/jsonschema: fix dropping of struct for additionalProperties: true
1347dd40 general: add .unity-bin to gitignore
a0e19707 general: add unity configuration to codereview.cfg
351a77e4 internal/core/adt: add more error positions
79644c3b internal/core/adt: allow hidden fields alongside embedded lists
70e2f2ec internal/core/adt: discard errors in field verification
452c7bd9 internal/core/adt: don't count optional fields as proof of struct
3d926afd internal/core/adt: don't eliminate incomplete disjuncts prematurely
1589a074 internal/core/adt: get default when comparing ot bottom
e12e26e9 internal/core/compile: fix let resolution bug
77e6b51a internal/core/compile: reenable disabled tests
2c86835c internal/core/eval: minor performance improvements for disjunctions
e1e70315 internal/core/runtime: decouple cue API from Runtime
c6b7ab2e internal/cuetest: add support for issue-based conditions/checkers
38f0f634 pkg/internal: fix decimal and string list errors
af3c9dce pkg/list: fix list.Unique dedupping
dcf932f3 tools/trim: fix too aggressive elimination for disjunctions
This release has some important bug fixes. It also is a bit stricter in considering some errors that were previously evaluation-time errors to be compile time errors.
There is also a slight change in default handling, which should not affect most configurations.
The expression >10 <20 is syntactically valid CUE, but can never meaningfully evaluate to something. It parses as >(10 < 20) and thus evaluates to >true, which is undefined. The error messages associated with this were often confusing to the user. Also, leaving this to be detected during runtime would sometimes hide the error away.
Consistent with the “fail early” mantra, these cases are now detected and reported by the compiler.
Previously, for
#def: {
*{} | {a: int} | {b: int}
*{} | {c: int} | {d: int}
}
x: #def & {a: int}
would officially resolve to
x: {a: int} | {a: int, c: int} | {a: int, d: int}
Thanks to a hack in the current v0.3 implementation, the first disjunct was made the default and all resolved fine. We have now slightly changed the definition of defaults to handle this case. Basically, now if during a unification all terms of a disjunction that are marked as default are eliminated, the remaining terms will behave as if they originated from an unmarked disjunction. In practice this means defaults behave as users typically expect them to behave. This may also cover cases that were not previously handed
Note that v0.2 solved things differently: if in the result of a disjunction one term subsumes another, then the latter could be dropped. So in v0.2, the answer would be x: {a: int}, even in the absence of defaults. The problem with this approach is that it could lead to confusing answers and that the default could not always be accurately determined. A goal of v0.3 was to have a simpler story for disambiguation and let simplifications be merely a matter of optimization.
There is still a slight discrepancy in the current implementation and the spec that may cause unexpected results. We expect this to be sufficiently rare and we intend to address this in a rework of disjunction computation, aimed at further performance optimizations.
1b03c9fa ci: move to non-Docker version of goreleaser
ae51469c ci: skip known flakey test on go1.15 windows
500e431d doc/ref/spec.md: add missing binary_digits
89cada6f doc/tutorial: address cuelang/cuelang.org#126
77b475f8 doc: fix install instructions in main README and contributing
25ba1d94 internal/core/adt: comment out leak test
b25147c3 internal/core/adt: control fixes for if clauses
267379b6 internal/core/adt: fix trim bug
48d255a1 internal/core/adt: some optimizations for pattern constraints
87d43b74 internal/core/adt: strip "notDefault" status for disjunctions with unused defaults
760aa115 internal/core/compile: make >10 <20 a compile error
b20ac5a2 internal/core/convert: fix conversion from float
bcd752ad internal/core/eval: handle embedded top correctly
a5daa168 internal/core/export: fix quoting of definitions
cb4fae90 internal/cuetest: consolidate testscript/cuetxtar helpers
bf6c3fce internal/cuetest: move Kubernetes related helper to where they are used
15934d98 tools/trim: add tracer
1d9aa173 tools/trim: don't trim shared nodes
af5ea978 tools/trim: fix defaults bug
This release has some important bug fixes related to performance, disjunctions, and performance of disjunctions.
As a result of the disjunction rework, we introduced another internal workaround for dealing with the fact that there is no good way to represent proto oneOf fields in CUE. Right now this is done as
*{} | { a: int } | { b: int }
or
*{} | { a?: int } | { b?: int }
The need for the default is because one cannot say that a field must exist. So the default is used to disambiguate. This works well for one set of oneOfs, but not in the presence of multiple oneOfs.
A previous hack would unmark a disjunction as having a default if all but one of the defaults remains. After the critical performance improvements, this still worked, but only because these changes introduced a bug. Now this is fixed we can no longer rely on it. To keep protobuf working, there is now another hack that recognizes this pattern and adjusts defaults in a similar manner.
This is an unacceptable solution, of course. The idea is to allow specifying required fields instead, somewhat like this
{} | { a!: int } | { b!: int }
and transition to this model.
The required field specification has many benefits and solves several modeling limitations of CUE we’ve encountered since its inception. It can also almost, if not entirely, eliminate the need for ?, provides a solution for some limitations of the proposed query syntax, and generally provides better alternatives for cases where one finds the need to abuse defaults.
We plan to officially introduce this construct in a proposal soon.
09cd07bb all: add tests for the GitHub workflow process f0d1fa4e ci: add first version of copybara workflow 10a48e90 ci: drop go1.13 from build matrix; add go1.16rc1 e097927d ci: fix broken build 34333ce1 ci: fix job dependency in test workflow 726d93e4 ci: fix start message of CI jobs for CL builds 9b7fc4b5 ci: fix workflow definitions 3088513d ci: only go test -race on latest Go+linux, and master 5b3d5514 ci: refactor CI setup 2338b593 ci: rename repository dispatch workflow ef59f61a ci: update version of Go for go generate to 1.14.14 257486c8 cmd/cue/cmd: allow fmt to pass with errors dfce25c5 cmd/cue: add goproxytest instance for script tests b8d99727 cmd/cue: allow CUE_UPDATE to update cmd/cue/cmd testscript tests a6169255 cmd/cue: embed packages required for get go checking 22aafc2a cmd/cue: fix typo in cmd documentation headline 37c16cc5 cmd/cue: use gotooltest within testscript tests d9d04872 cue/scanner: fix JSON conformance ba8eb371 cue: allow LookupDef on embedded scalars c6f1287a cue: deprecate Merge f73e78f8 doc/../43_schema.txt: remove unnecessary word 6c3be7f8 doc: fix typo in types tutorial (closed -> close) 2df92cb5 doc: remove duplicate command from Kubernetes tutorial b360fe43 encoding/gocode: use FieldByName in generated code 7110020e general: move from godoc.org to pkg.go.dev for documentation d5d5fd15 general: remove junk files 63104c84 internal/core/adt: "fix" disjunction resolution 2ca6c5e1 internal/core/adt: cache evaluation results for dynamic fields c866a515 internal/core/adt: don't check cycles for inlined arcs 6b960014 internal/core/adt: fix benign memory leak b8b48925 internal/core/adt: fix bottom cycle error c0912749 internal/core/adt: fix bug that left errors of incompleted expressions unreported 9c5489fc internal/core/adt: fix memory leak ee78ec3a internal/core/adt: fix nil panic 8d537aac internal/core/adt: fix regressions in disjunct elimination e6a2fb02 internal/core/adt: fix special cycle condition bedb047a internal/core/adt: more fixes to cycle handling 8334a9e1 internal/core/adt: more merging of adt and former eval d0124e2a internal/core/adt: refactor node lookup 0ad02fdf internal/core/adt: restrict notification d2aa995f internal/core/export: don't embed proper conjuncts 76a72c4f internal/core/export: fix optional output bug fdccfbc5 internal/core/export: retain top in embedded scalars 34587b9c internal/core/export: support package and struct attributes dfaea59a internal/filetypes: fix test breakage 10180b64 pkg: consistently use 0666 as perm when creating regular files f55f17ea tools/fix: eliminate x & _ and unnecessary parens 611d6227 tools/flow: replace package doc example with real example
This release addresses some important bug fixes, including a performance regression. It also sports a complete reimplementation of trim, utilizing the possibilities of the data structures of the new evaluator.
The biggest new feature of trim is that definitions now can remove fields of non-definitions. Other than that, it mostly is just bug fixes. The data structures of the new evaluator make it considerably easier to implement, leaving less than a quarter of the code.
This package now links in the full set of builtins, obviating the need for API users to have to do this.
fdc7010f cmd/cue/cmd: fix cue cmd crash a8de61c6 cue/testdata/cycle: fix another cycle bug 28cfa742 cue: exempt hidden fields also in Unify and UnifyAccept ca5d2db5 doc/tutorial/kubernetes: update files of Kubernetes demo f0c025c1 encoding/protobuf: enforce loading builting packages 08e814f8 internal/core/adt: add support info for trim 8e6e7959 internal/core/adt: change closeInfo.embedded to a flag field 0e401d66 internal/core/adt: fix performance issue with disjunctions 82bda840 internal/core/adt: treat files as belonging to same struct c58e0ffc internal/core/eval: strip arcs for nodes with structural cycles 0f570a90 internal/core/validate: pick up non-concrete values in defaults f03928d2 tools/fix: allow fixing non-essential simplification 68988976 tools/trim: implement using adt
This release predominantly addresses some regressions and bug fixes. There are also some additional small performance fixes.
There are a couple of bug fixes that may lead to backwards incompatibilities.
There were some bug fixes where disjunctions were discarded as duplicates prematurely. This was already an issue since v0.3.0-alpha1. As a result, you may now get an “incomplete value” error where before you didn’t. For instance, {a: int, c?: int} | {a: int, b?: int} now will remain as two separate disjuncts, whereas previously it was assumed evaluation was done too early that the optional fields were irrelevant.
In v0.2.2 this would also not be disambiguated. There are, however, still some differences between v0.2 and v0.3. Pre-v0.3 CUE used subsumption to disambiguate disjuncts. Subsumption is inherently imprecise and allows false negatives. This would inherently make it hard to predict when values were disambiguated. As subsumption was used at no other place during evaluation, v0.3 takes a simpler approach to only check for equality, whereby optional fields are only considered equal if they originate from the same original structs.
The general idea is that one should typically not have disjunctions of structs without those having discriminator fields (i.e. the disjuncts would fail if unified with each other, for instance with a kind field with a specific value, like K8s does). If one still needs this, then one of the structs should be marked as default to disambiguate.
In other words, v0.3 may give an “incomplete error” where v0.2 did not. For instance: #Fields: _ | [string]: #Fields would normalize away with v0.2, but won’t with v0.3. The simple solution is to use _ or {...}. The v0.3.0-alpha releases had a bug that would prevent these alternatives from working properly, but that has been fixed as of beta.1.
We may still consider allowing more disambiguation at the point of making values concrete. This will need to be considered carefully, though. We may also allow some compile-time optimizations, like translating _ | X to _. We have not done so yet, as a set of rules for such translations has the danger of making disambiguation arbitrary; we would like If people have trouble migrating to v0.3.0-beta.2, we can prioritize such simplification rules for beta.3.
We intend to come up with a set of clear and predictable rules before v0.3.0 on when CUE may disambiguate disjunctions. Until that time, please ensure that disjunctions with structs are disambiguated using defaults and discriminator fields. This should generally be considered to be proper style and highly recommendable anyway. Note that any protocol buffer and all K8s structs, for instance, follow a pattern that can be disambiguated easily.
The value #B: {#A} & {a: 1} is now interpreted as #B: #A & {a: 1}. Previously, this was interpreted as #B: {#A, a: 1}, allowing field a regardless of whether #A allows it or not. This was a leftover of the old closedness rules of pre-v0.3.0. The old rules were somewhat convoluted and based on user feedback and experience we simplified the rules and made them more intuitive for v0.3.0.
We have added the list.Concat and list.Repeat builtins in anticipation of removing the * and + operators for lists in CUE. These operators are quite unintuitive as it is not clear how they interact with closedness, which has raised repeated questions. They also complicate the implementation. This has been mitigated by, internally, translating list addition to list of two list comprehensions. They also complicated automated analysis, which is harder to work around. Better to get rid of them altogether.
The next steps for the transition is to let cue fix automatically rewrite list addition and multiplications in terms of their builtins. Ultimately list addition could be replaced with a query shorthand, like [a] + [b] -> [ a.*, b.* ], using a.* as the hypothetical notation for all values of a.
We have seen some considerable performance improvements for some configurations. There is still plenty of room for improvement though, both algorithmically, which can bring long-running evaluations down to below a second or two, and by reducing the constant overhead, by itself allowing a running time reduction of at least another 90% in many cases. So if there are any pressing performance issues, we would love to hear about them.
ed7d254d ci: delete CL build branches once workflow is complete 8c416223 ci: fix dispatch workflow ccbfbd2e ci: move to build branch model 237a4aef ci: use the latest GitHub workflow definition f1b18040 cmd/cue/cmd/getgo: don't be clever ff48658e doc/ref/spec.md: disallow new definitions in closed structs 172f0060 doc/ref/spec.md: get rid of list operators 5691606b internal/core/adt: add test for incomplete references in lets bef7b263 internal/core/adt: fix cycle bug a086f74c internal/core/adt: fix disjunct dedupping regression ae1203cd internal/core/adt: fix overzealous disambiguation 826bd7bd internal/core/adt: fix regression in close builtin 34a91b9a internal/core/adt: fix some regressions 663939eb internal/core/adt: improve errors for validator conflicts 13b68c81 internal/core/adt: minor cleanup 355cccd2 internal/core/adt: precompute optional field type info 9eee235e internal/core/adt: remove special "tentative" mode 2ef72d85 internal/core/eval: fix result of closed embedding 03aab971 internal/core: re-allow new definitions in closed structs 0ffde15b pkg/internal: fix error codes for decimal and string lists 867f71fe pkg/list: add builtins for list concatenation and repetition
This release is focussed predominantly on performance. It includes several algorithmic improvements that eliminate possible exponential processing and in general takes more care to avoid recomputation.
Aside from algorithmic changes, these changes also prepare for numerous further optimizations. The algorithmic changes already already give enough of an improvement (especially for some large files) to warrant another release.
This release also introduces a slight language change regarding disjunctions. Consider
x: *1 | a
a: *2 | 3
Previously, the result of x would be *1 | *2 | 3, in other words, a default value of 1 | 2. With the new semantics, the default in this case will be 1. In general, with the new semantics, if a disjunction has a term with a default, the other terms can never result in a default value.
Superficially, this makes it easier to see where defaults may come from and eliminates the need to check all terms of a disjunction. But this change also prevents some unintuitive behavior and even cases where making values more specific may lead to a less specific result.
The changed disjunction rule may cause backwards incompatibility.
There are some minor bug fixes as a result of these changes that may change the behavior For instance, [...] | [] was previously normalized to [].
There are also some changes to error messages as a result of these changes.
cc418539 doc/ref/spec.md: change disjunction rule d5b4ca85 internal/core/adt: add logging utilities for debugging 0c3791ea internal/core/adt: add memory management 008d37d2 internal/core/adt: catch errors for circular failures b619e0ca internal/core/adt: consider closed status for equality 396202cc internal/core/adt: don't pre-expand disjunctions 69e0c96f internal/core/adt: introduce HasDefaults for Disjunction 514082ac internal/core/adt: keep nodeContext in Vertex a595db28 internal/core/adt: skip partial unify if unnencessary 02ff29bf internal/core/convert: fix faulty label update 778719ed internal/core/eval: breadth-first disjunction resolution 2be20f84 internal/core/eval: do not precompute matchers 399ba762 internal/core/eval: export fieldSet fields 0ef6caa8 internal/core/eval: make arcMap a list 1fa69d59 internal/core/eval: recursive disjunctions 9ca9e043 internal/core/eval: separate out dynamic fields 0e1e7b5a internal/core/eval: small cleanup of evaluation code 062b3d88 internal/core/eval: sort empty disjunction message to top of group 359685af internal/core/eval: use env from fieldSet d4c600fb internal/core/export: extract docs from nested Vertex values c62d8a05 internal/core: merge adt with eval package cea55b28 internal/core: reimplementation of closedness algorithm 93c79a34 tools/flow: record dependencies per value based on path
This release is characterized by closing some significant remaining gaps in with the language specification.
There are the usual bug fixes and improvement to error messages, though.
The language has now been brought closer to the spec, implementing some of the remaining gaps. There have also been some simplifications that will allow further generalizing the language down the road more easily and without introducing backwards incompatibility.
As the spec already allowed, it is now possible to include multiple comprehensions per list.
For instance:
src: [ "John", "Mary" ]
cond: true
a: [
for x in src { x },
if cond { "Sasha" }
]
The list operators (concatenation and multiplication) are now also implemented in terms of comprehensions. There is a great deal of confusion about list operators (is the result open, closed, when do overlapping constraints apply?). This confusion is eliminated by writing the equivalent as list of multiple comprehensions. Our intention is to remove these operators and have comprehensions (and later queries) be the only way in which lists can be concatenated or multiplied. Of course, cue fmt will help in the transition there as usual.
Just as it is possible to write
[ #a, #b ]
#a: 1
#b: 2
at the top of a CUE file, it is now also possible to have an embedded scalar along with definitions in any struct:
x: {
[ #a, #b ]
#a: 1
#b: 2
}
Originally, the top-level construct was allowed to make CUE strictly compatible with JSON. Generalizing this to any struct makes the use of imported file objects consistent with normal structs. Moreover, it turns out that this construct can be quite useful overall.
Old CUE code is not affected by this change.
[pattern]: value fields (no change after all)The spec recently included a change that proposed to use the JSON schema semantics for pattern constraints ([k]: v), rather than the old CUE semantics. To aid in the transition, cue fmt rewrote these fields to be enclosed in curly braces, which mimics the old semantics under these planned rules.
It has now been decided to keep the old semantics and stop rewriting old uses.
As it turns out, the old semantics was used quite a bit and was actually the majority use case. The argument previously was that the JSON schema semantics was not easily expressible in terms of the old semantics, but that the old semantics is easy to simulate with the JSON schema semantics.
Instead we now intend to allow the JSON schema semantics additionally by writing [...k]: v. The nice thing here is that any pattern which is to be matched additionally, is now always preceded by a .... It also is a nicer generalization of the syntax.
But above all, it removes the need for a potentially painful transition.
CUE allowed integer division and remainder per the div, mod, quo, and rem operators. These were relatively rarely used and complicated the CUE syntax, formatting and other parts of various subsystems.
Moreover, they made it lexically impossible to allow indexing of the form a.3 (instead of a[3]) down the road. We are not certain if we will allow this, but allowing it makes the syntax considerably more regular and consistent. With this change, we have the option to do so in a backwards compatible way.
These operators have now been replaced with builtins. As usual cue fmt will rewrite old uses to use the builtin for some time to come. It will use the __foo variant to avoid shadowing issues. These can be manually rewritten to remove the leading underscores if necessary.
The model for builtins, which are still underspecified in the spec, is narrowed down more. It is now possible to define variable argument builtins.
The main reasons:
The first target was pkg/path which now supports multiple OSes (see below).
Errors in nested comprehension clauses were sometimes dropped, causing a failure in a nested if clause to be silently ignored. This has now been fixed.
Other fixes includes crashes related to let clauses and bytes interpolation.
There are various areas where error messages have been improved.
cue fmtThere are various printing bugs that were fixed. Overall the stability of fmt should have been improved notably.
The API got a few fixes related to backwards compatibility and should now be more like v0.2.0 where this makes sense.
pkg/pathThis package has now been extended to support multiple operating systems simultaneously. Most builtins have been extended to allow an additional optional argument to indicate the OS for which the builtin is to be applied. For instance:
path.Split("foo\bar", os)
will now return a different value depending on whether os is path.Windows or path.Unix. The os argument can also be omitted, in which case it defaults the existing behavior, which is identical to path.Unix.
Note that os will not default to the currently running OS. This would make the behavior non-hermetic. The idea is that the existing injection mechanism can be used (or a small extension of it to make this specific case easier) to inject an environment-specific value.
The cue tool is now a bit more aggressive to panic if it encounters an internal inconsistency error. In return, it now allows the user to define a CUE_DEBUG=0 environment variable to disable this. In many cases the crashes can be ignored while still getting useful results.
Not using this option and filing bugs can help improve the stability of CUE though.
Main backwards incompatibility to note:
cue fmt.e77ccb1c cmd/cue/cmd: don't bail for non-matching files
fecdd834 cmd/cue/cmd: ignore possible top-level tasks
bbe493a7 cmd/cue/cmd: more position information for field errors
fe73e8f9 cue/format: format let clause
110d0bff cue/format: pad floats with leading or trailing periods
fac5d2e3 cue/load: allow ./... to load all packages
555fb73d cue/parser: fix comment placement
1c904cc2 cue: allow multiple comprehensions in lists
8872b98f cue: fix PathCorrection tests
6c49cf0f cue: implement embedded scalars
d5177fd1 cue: map API values to more native ADT
b687b7f7 cue: remove aliases used for refactoring
19f6f3f6 doc/ref/spec.md: perpare for allowing a.4.f
f52a0ed8 doc/ref/spec.md: revert to the old semantics of [k]: v
76ea22cb internal/core/adt: add Assert for debugging
37293f95 internal/core/adt: add methods for concreteness
41afc87e internal/core/adt: automatic renaming
2d568d30 internal/core/adt: better message for permanently non-concrete values
f62bfed9 internal/core/adt: clean up Builtin and Validator semantics
4db8ffb1 internal/core/adt: do list artihmetic with comprehensions
cbcb701f internal/core/adt: error message improvements
d0dd8887 internal/core/adt: improve error message for optional fields
08a1652d internal/core/adt: introduce base value type
6de877a9 internal/core/adt: make progression check an assert
8b137527 internal/core/adt: move decimal logic into one place
b8c852d4 internal/core/adt: pass errors up in nested comprehensions
36f20510 internal/core/compile: add integer division builtins
9c6ded8d internal/core/compile: don't panic for incorrect labels
13e4af16 internal/core/compile: fix issue in and builtin
ff3e32f3 internal/core/convert: allow results of math to be integer
31896af4 internal/core/eval: add more tests for embedding
d49777fe internal/core/eval: better incomplete error messages
55602d94 internal/core/eval: fix closedness issue for lists
9f1fec6f internal/core/eval: hoist vertex processing logic
cefe38b1 internal/core/eval: keep semantics for [K]: T
acc35f11 internal/core/eval: track fieldSets for data values
e7e27e00 internal/core/export: allow showing recursive errors.
039fb590 internal/core/export: fix bytes printing bug
2ea30a22 internal/core/export: fix crash with let comprehension export
16809ead internal/core/export: support byte interpolations
63594eff internal/core/runtime: assign first label to _.
08116621 internal/core/subsume: tests embeded scalars
a4d09289 internal/core/subsume: various bug fixes
136e51b4 internal/core: allow variable arg functions
e841714b internal/core: move equality checks from eval to adt
d174bc09 internal/cue/adt: remove use of Default in node
fa6308f2 internal/encoding: don't clobber files on crash
ec6f95d8 internal/filetpes: improve error messages
fb3b02ed pkg/internal/builtin: use normal formatting for builtin test results
a50960c8 pkg/path/testdata: adapt Go implementation
a2692ef4 pkg/path/testdata: move files to minimize diffs
c77b9b0d pkg/path/testdata: stage filepath files from Go
85f8b466 pkg/path: activate OS-dependent version
64434c46 tools/fix: rewrite integer division
d5ced741 tools/flow: ignore faulty tasks outside the root
This release is mainly focused on various bug fixes and error message improvements. This change also continues to factor out code from cmd/cue to make the functionality available outside of this tool.
Below is a selection of the fixes.
Although this is officially alpha, it is probably a more solid release to use than v0.2.2. For this reason the "Pre-release" tag has been left off from this release.
Various bugs related to cycle handing were fixed, including hangs and spurious detection of cycles.
Hidden fields are now scoped per package.
pkg/sortSort now always uses stable so that reproducible outcomes between different compiles of CUE can be guaranteed.
cue.PathThe introduction of #-style definitions are not handled well by the old API. This is a consequence of them living in a separate namespace, which is not expressible in the old API.
The new cue.Path type is introduced to start addressing this. The first method that supports this is LookupPath.
This API is designed to be compatible with the planned query extension.
tools/flowThis new package allows arbitrary user-defined CUE schema to be interpreted as task definitions, automatically handling dependency analysis and scheduling tasks based on these dependencies.
This was written from scratch and makes use of a new native dependency analysis package, allowing for more precise analysis.
cue commandThe translation of CUE to Go is now brought in line with Go’s encoding/json, fixing some bugs that caused the interpretation to differ.
cue cmdThis now uses the new tools/flow package, fixing a bunch of outstanding bugs. Most notably, it now allows dynamic task creation, for instance by using comprehension, and depending on such dynamic tasks: $after should now only be needed if there are really no dependencies at the CUE level.
b4aa96d7 all: implement hidden identifiers scoped per-package aa878875 ci: drop use of set-env in CI scripts 4cda4a3c cmd/cue/cmd: allow inlined embedding of pointer types c715b94f cmd/cue/cmd: use tools/flow e05eee7d cue/errors: correct handling of wrapping list errors 54b13dbe cue/literal: fix multiline quotes 6165736e cue: allow access to Selectors of Path f4f2a02f cue: dig up wrapped error in valueError 74329a13 cue: doc improvements 56eb4b16 cue: generate error node rather than panic on faulty output. 03abe876 cue: remove most uses of xerrors 409dacfa cue: support getting and looking up paths fcd96de9 doc/ref/spec.md: define "regular"; fix typos 86612555 encoding/protobuf: fix trailing comment in oneof bug b5821dc2 interanal/core/runtime: move building logic from cue package b886b0f1 internal/core/compile: fix mutual dependent let clauses 3585705a internal/core/convert: fix for embedded structs are not honored b99fd082 internal/core/cue: fix hang on cycles in embeddings in comprehensions c60e1153 internal/core/dep: first stab at precise dependency analyzer 7463d11d internal/core/dep: support dynamic dependencies 1888d65b internal/core/eval: fix bug in cycle handling 1d8c688c internal/core/eval: fix bug with nested embeded comprehensions aee9955c internal/core/eval: fix cycle bug for comprehensions 1a2105eb internal/core/eval: fix hang in cyclic in comprehension evaluation 3cfb4ab3 internal/core/eval: fix irregular dynamic fields d1cbe10a internal/core/eval: fix spurious cycle detection 635fbdd4 internal/core/eval: fix spurious cycle for bulk constraints 7980ec5d internal/core/eval: handle disjunction failure more properly ab0a2f39 internal/core/eval: re-allow ... with embeddings 30704a7c internal/core/eval: reduce per-node allocations 1e0faf0f internal/core/export: dedup let clauses for merged structs 187c7343 internal/core/export: handle alias and lets cd621ffa internal/cue/eval: fix faulty status in tentative result dbfa73b0 internal/cuetxtar: allow multiple golden files f2a4a42b internal/filetypes: hard-wire common case 737a103f internal: hoist a single embedding in ToExpr 99d18dcf pkg/list/sort: always use stable sort 00f345b8 tools/flow: API for CUE-based workflow engine d3ff4a17 tools/flow: add IgnoreConcrete option
This is a relatively small release. But given that it contains some significant bug fixes and performance improvements, it seemed useful to get these out sooner.
This also
The rules surrounding closedness have been further clarified in the spec where this was previously ambiguous. Most notably, it was previously unclear whether B.b in
#A: {a: int}
B: {
#A
b: c: int
}
should be closed or not. This is now clarified to it not being closed. In other words, fields added alongside an embedded definition that were not part of the original embedding are not closed automatically.
This does not change the current behaviour.
There was also a bug fix related to closing structs that could leave struct open if they were referenced through a selector path like #A.b, where the reference to #A failed to trigger a close.
There remains one known gap in the implementations compared to the spec: embedding a closed struct will currently cause the field to be closed, whereas the spec says that the struct in which it is included should already be closed. This will be a backwards-incompatible change.
field not allowed errorAs far as is known, aside from the spec deviation mentioned above (which is somewhat esoteric), there are no more known deviations that result in a spurious error of this kind.
There is, however, one known case where this error seems to appear erroneously. Consider the following:
#a: {b: int} | {}
b: #a & {b: string}
This may still result in a “field b not allowed” error. The problem is that the error reporting around disjunctions is still lacking. What happens is that only the error of one of the failed disjuncts is reported. In this case it fails on the first disjunct (as int != string) and the subsequently fails on the second, which doesn’t allow b. As only one of the errors is reported, it may appear that it fails on the first disjunct, which is just confusing.
The intention is to considerably improve disjunction errors soon. In the meantime, the added position information for conflict errors will hopefully help a hand to disambiguate such errors.
let clauses are now allowed in comprehensionsThis has been a part of the spec for a long time, but now has finally been implemented.
... anywhere in structThis was already allowed by the spec, compiler, and evaluator, but now is also allowed by the parser.
Another thing that has been allowed by the spec for a while is the use of string literals as selectors. So a.”foo”.bar is now allowed. Only single-line strings without interpolations are allowed at the moment (as per the spec).
cli.AskThis is now supported.
Error reporting has been improved further. Most notably, positions for conflict errors (as resulting from 3 & 4), now have positions added and should be on par with v0.2.2. This also adds the fundamentals for adding richer position notation.
This release fixes one big performance issue, which could lead to exponential runtime for disjunctions. This can have a big impact for many configurations. For one user, it reduced the running time from 5 hours to less than 2 seconds.
As part of this fix, we now introduced a benchmark suite.
There are tons of significant performance improvements still possible, so this is not the last of it.
CUE used to piggyback on Gos strconv.Quote` for quoting strings. This was not ideal, as CUE and Go strings are not the same. It also resulted in some bugs. This is now fixed. This results in better string formatting overall.
-The cue command line tool sometimes hung when a directory contained a file named -. This is now handled.
Until now, CUE would output YAML in version 1.2 and read it in version 1.1. And it seems that the “right thing to do” is to implement version 1.2 these days. This has now provisionally been fixed to read it mostly as version 1.2. A more proper implementation would be to replace internal/third_party/yaml to use the yaml.v3 package.
This means, for instance, that the value no is now treated as a string instead of the boolean value false. (And thus the demo on cuelang.org will have to be replaced.)
Handling of message options in .proto files would previously fail. They are now translated into declaration options in CUE with the format: @protobuf(option name=value).
As YAML 1.2 is not backwards compatible with YAML 1.1, inputs to CUE that relied on a YAML 1.1 interpretation may now yield different results.
The refinement of the rules of closedness are largely backwards compatible (and what is not is not yet implemented), but some of the bug fixes may give slightly different results nonetheless.
8ad07fa Readme.md: remove Go Report Card 2ac4d85 cue/format: indent multiline string literals based on context. c886094 cue/literal: implement CUE-specific quoting 11b507c cue/load: don't hang on files named - 6354c40 cue/parser: allow ... anywhere in struct 241c5bf cue/parser: allow let clause in comprehension 80f0bee cue/parser: better error message for old-style definitions 732b6d5 cue/parser: support string selector labels e9286f0 doc/ref/spec.md: fix builtin func ff306b7 doc/ref/spec.md: fix spec bug and broken link 15e8a05 doc/ref/spec.md: refinement of closedness 65468d5 doc/tutorial/kubernetes: update import 3af2683 encoding/protobuf: parse message options d23402c internal/core/compile: fix alias resolution bug 742593f internal/core/compile: fix hidden definition bug da69dc0 internal/core/eval: ... does not apply to defintions 0eb7cc5 internal/core/eval: add more positions to conflict errors 3914ef8 internal/core/eval: check for nil pointer 03fa5a8 internal/core/eval: fully evaluate nested unification f0df4df internal/core/eval: performance: unify an arc only once per node 1fdc02a internal/core/eval: simplify validators upon evaluator 1c29042 internal/third_party/yaml: drop support for non-standard scalars 5bfe37d pkg/tool/cli: support ask task