CUE has moved to https://github.com/cue-lang/cue
This release mainly introduces per-file inclusion based on tags, bug fixes for bugs introduced in the new evaluator, and tests to close issues raised on the old evaluator that are now verified to be fixed.
Attribute declarations are now allowed before the package clause and import declarations. This is used to implement file-wide inclusion.
This includes boolean and byte values. Boolean values are represented as their JSON value. In string interpolations, bytes are converted to valid UTF-8 by replacing the maximal subpart of ill-formed subsequences with a single replacement character (W3C encoding standard).
CUE now allows files to be conditionally included in a build. To conditionally include a file in a CUE program, one can now write:
@if(prod)
package acmecorp
and use the -t prod flag to cause this file to be included. The expression can be any valid CUE expression using only boolean operators and boolean values.
See cue help injection for more details.
A large number of bug fixes related to an incorrect “field not allowed” error are now in. The algorithm for closedness has been significantly simplified and streamlined based on the benefit of hindsight.
Note that one may still have a misleading “field not allowed” error if a field is disallowed in one disjunct and allowed in another, but where there is another field in conflict. Right now only one of the errors is reported.
The CUE API is now moved to the new evaluator.
Builtins now live in their own package and are no longer copied into the cue API. This was part of the effort to break the implementation into logical pieces, instead of having one large package.
This has the additional advantage that each builtin package now only lives in its own directory and is not copied. This makes maintenance easier and also removes some of the limitations imposed by the old implementation.
Unfortunately, this means that custom loader implementations (ones that do not use cue/load for loading packages) will have to include
import _ "cuelang.org/go/pkg"
somewhere in the package to trigger linking in the builtin packages.
cue toolThis release also marks the first steps of moving functionality from the cue command line tool to the CUE APIs. The intention is that the cue tool will be mostly a wrapper around core functionality present in the API. Other tooling could provide a different “view” without being second-class citizens in the ecosystem and while providing consistency between such tools.
The first step towards this is the injection mechanism. The existing parts of this were moved to the cue/load package. The new file inclusion functionality, meanwhile, was directly implemented in cue/load.
The next step will be to move non-CUE file support and packaging mechanisms to cue/load and friends. Then next up is the tooling layer, starting with an API for dependency analysis.
There are two outstanding issues on the table to come to a beta release:
Although there are no known semantic bugs related to disjunctions, the code needs some restructuring to improve error messages and to fix some performance issues. This is slated for the alpha4 release.
There are still some bugs outstanding related to structural cycles. The plan is to address all of these in an alpha5 (or possibly beta) release.
Support for go1.12 was dropped. In exchange, this release is verified to build with Go’s tip (1.16).
The CUE API has now fully been moved over to use the new evaluator. The old evaluator code has been removed.
cue.Merge: the purpose of Merge is to unify the underlying instances as data. It now no longer also unifies definitions. Bug fixes in closedness exposed this previously unnoticed oversight.Custom loader implementations (ones that do not use cue/load for loading packages) will have to include
import _ "cuelang.org/go/pkg"
somewhere in the package to trigger linking in the builtin packages.
CUE code that previously expected a boolean or bytes value substituted in a string interpolation to fail my now break.
cue cmdCommands that operate on multiple instances simultaneously (e.g. cue cmd foo ./...) can no longer reference definitions in the underlying commands. For more info see cue.Merge in the API section.
d69319c all: add missing copyright 4c10692 all: fix four previously ignored errors 6c27cef ci: drop go1.12 add go1.15 c3f30d8 ci: name dispatch workflow differently to default branch push workflow 1c54297 cmd/cue/cmd: fix get go package handling 1e8906a cmd/cue/cmd: move injection mechanism to cue/load faf617c cmd/cue/cmd: verify issue 236 is fixed e71970c cmd/cue/cmd: verify issue 322 is fixed 8776561 cmd/cue/cmd: verify issue 425 is fixed ba5708b cmd/cue: get go: add --package flag 1a9b88d cmd/cue: get go: use constant.Value.ExactString to extract constant f25c04d cmd/help: fix example in doc 089f461 cue/load: add support for build tags bc101b3 cue/load: add tool and test files to BuildFiles if requested 20c637a cue/load: relax places where @tag is allowed bd3dd75 cue/load: rewrite pre-resolved files after injection c264475 cue/parser: allow attributes before package and import clauses 4cce6c4 cue: apply wrapping to all errors in list d2fdbf0 cue: clean up handling of preamble declarations 80e70b9 cue: fix instance error handling c76a530 cue: implement fill in terms of Value.Fill 845df05 cue: make internal cue port main API implementation abce145 cue: move to external builtin packages ce28b20 cue: prepare to hoist builtin code 9acd624 cue: prepare to hoist builtins, step 2 40a6bcd cue: remove dead code 6f84d11 deps: upgrade to github.com/rogpeppe/go-internal v1.6.1 edea6f3 doc/ref/spec.md: fix len count for structs 043c534 doc/ref/spec: clarification on cycles dd0fa88 doc/ref/spec: consider definitions for closedness c7c14fd doc/tutorial/kubernetes: remove a duplicate kind field e3a03a1 internal/core/adt: exclude definitions in data conversion db18b37 internal/core/adt: prevent nil interface bug for Source f5fa009 internal/core/adt: remove all closedness IDs in ToData 30ca062 internal/core/adt: support interpolation of bytes and bool df01042 internal/core/compile: better error message for user-defined errors d55e2b5 internal/core/eval: allow lists resulting from dependent expressions c72ce5d internal/core/eval: fix empty data vertex bug 0b57e43 internal/core/eval: fix interpolation of numbers e0c19d1 internal/core/eval: fix issue 507 304b02e internal/core/eval: nested comprehension fix 62528c3 internal/core/eval: rewrite of closedness algorithm dc2401c internal/core/eval: verify issue 258 is fixed ddd99d5 internal/core/eval: verify issue 293 is fixed ea5d964 internal/core/eval: verify issue 295 is fixed 480e1a1 internal/core/eval: verify issue 299 if fixed 541bf03 internal/core/eval: verify issue 306 is fixed 5905c22 internal/core/eval: verify issue 332 is fixed 121ace7 internal/core/eval: verify issue 342 is fixed 07ce2f6 internal/core/eval: verify issue 353 is fixed 4ecec98 internal/core/eval: verify issue 370 is fixed fbf9de3 internal/core/eval: verify issue 398 is fixed c6ade67 internal/core/export: fix bug 473 b9b2c54 internal/core/export: better handling of generated CUE 1e1fe6f internal/core/export: handle scalar values in adt.Vertex 67d1d42 internal/core/export: suport default picking df9468e internal/core/export: verify issue 349 is fixed 60c207f internal/core/runtime: collect errors for all files in resolve a6ce869 internal/core: finalize list in slice 148f67a internal/core: fix incorrect error handling 3d863fc internal/core: move CloseID from Environment to Conjunct bf882b0 pkg/internal: copy cue builtin code to separate package dda7de4 pkg/math: fix issue 418 4f4b7a3 pkg/tool/file/append: create file if none exists. c8a4514 pkg/tool/http: fix handling of errors 248794e pkg: generate per-directory tests from builtin_test.go 1bcb73a update go-internal to pull in fix for Go 1.16
DO NOT USE THIS RELEASE
This release implements the long-anticipated move of cue to the new evaluator. This particular alpha release only switches over the cue command. A follow-up release will switch over the API as well. This allows people to upgrade to the tool without having to worry about API breakage just yet.
Note that this is an alpha release. This release includes an almost complete rewrite of CUE core. Although it will see many benefits, it can be expected to need some more ironing out. This includes new bugs in semantics and performance. But it also means dealing with bugs in the old evaluator that are now uncovered.
This release of the new evaluator mostly aims to address old bugs and doesn’t introduce many new things. However, this still means it addresses some important things:
Considerable performance improvements. More on that later.
Many bug fixes: the new evaluator is designed with embeddings, definitions, and optional constraints in mind, instead of them being an afterthought.
Structural cycle detection: this should put an end to stack overflows (but see caveats below).
Non-monotonic constraints: this allows things like structs.MinFields(3) to work. This also enables the implementation of things like not (for JSON Schema support), association lists, and all sorts of other goodness.
A more official ticking off the box of existing issues is still forthcoming.
Under the hood it has an almost complete implementation of the following features:
JSON Schema semantics for pattern constraints (bulk optional fields) and additional constraints (allowing ...T in structs as well).
Embedded scalars: embedding scalars is allowed at the top-level, but now would be allowed anywhere: with the same rules that apply at the top-level. For instance, { "foo", #field: 2 } would be allowed.
These features will be enabled in a later release (in most cases by simply relaxing the parser), once they are better tested.
Under the hood, the new evaluator has been broken up into about 10 components, instead of being one large monolith, separating out clear responsibilities. The largest package is now about 1/15th the size of the original implementation. This clarifies dependencies and responsibilities.
Previously, *_tool.cue files were interpreted in a different namespace that lived as an overlay over the other configurations. This was hard to explain for users and also results in many awkward situations, like the inability to print _tool.cue files.
This functionality is not implemented in the new evaluator. This means the command section is merged in with the other files. This may cause clashes.
The idea behind not implementing it is that the new #-style definitions, which live in a different namespace, allow for another, better approach: commands can simply be declared as definitions to avoid clashes with regular fields.
For cases where the user also has no control over the definition names, the idea is to allow for a convention where a tooling logic can be in a separate pkg_tool package to accompany a regular package, not unlike a foo_test package in Go. To CUE this would then just be a regular package. Feedback appreciated.
if comprehension fixThe old evaluator used to silently ignore an error of the condition of an if clause, making it the same meaning as false. This was incorrect and led to several bugs. The new evaluator now treats errors as errors.
If the old behavior was desired, one can achieve the same result using disjunctions. One such case was:
foo: string
if len(foo) > 0 {
}
The old (faulty) behavior can be simulated writing this as follows:
foo: string
if *(len(foo) > 0) | false {
}
In the old evaluator, sometimes allowed a selector to index into an unresolved disjunction, causing (null | { c: 2 }).c to resolve. This is against the spec and is always illegal in the new evaluator. So one may find cases where one will have to write (d&{}).c to first resolve the disjunction.
Please don’t send bug reports on faulty error messages for this release.
The new evaluator implements a different approach to error message generation. This allows the creation of very tailored and detailed error messages. It is just not done yet for the most part. Net effect: a lot of the line number information is missing, the path is sometimes missing (but also improved in some cases, as the path information is now more reliable), some of the error messages are still sloppy.
This means CUE should no longer crash on structural cycles.
Please submit bug reports if CUE hangs in an infinite loop or has a stack overflow
The spec disallows structural cycles. This includes reentrancy for the form
foo: { n: int, out: foo&{n:2}.out+1 }
The new evaluator also has some performance stats built in. This showed that CUE really isn’t great at doing reentrancy and will quickly venture into unexpected exponential behavior.
CUE’s strength is to combine results of computation, not do the computation itself. So instead, the way forward for CUE is to make it easier to shell out to other languages like Go, Python, Javascript and/or WASM.
That said, at the moment shelling out to other languages can only be done by creating custom invocations in the tooling layer. A convenient way to do this is not yet supported.
For the time being, we could consider supporting a structural cycle allowance e.g. as a command line flag or environment variable that allows a certain number of structural cycles to be ignored by CUE. This could help people transition off the use of reentrancy. Feedback appreciated.
Printing of CUE values has been completely rewritten and as a result will result in the inevitable differences. CUE now has a stricter distinction between values and expressions. This may cause printed values to be rendered differently. Also, printing of definitions still needs some work and can be expected to have some bugs.
There will always be the inevitable cosmetic changes, but:
please report where changes of printing results in bad incompatibilities.
CUE used to print fields in the order that a field name was declared anywhere. This often resulted in fields being sorted in schema order, but this was clearly not always the case.
The new evaluator does a topologic sort on fields before printing them. This means that fields will observe the order of fields of all structs that combine into a result. Insofar there remains ambiguity, the fields will be combined arbitrarily. In case different structs give conflicting orderings, the result is undefined.
The algorithm does not yet take the relative ordering of embedded structs into account.
tools/trimTrim relied on very intricate behavior of the API as well as various deprecated API features. It made assumptions that no longer held as the language progressed even in the old evaluator.
The code has been adjusted to use the new APIs. It now often does not remove the top-level element that was inserted by a comprehension. Otherwise it should largely give the same results. The new evaluator also fixes some inconsistencies that fixes known bugs in the old evaluator that failed to remove some fields in some cases.
One of the benefits of the “always evaluate everything” approach is that CUE can annotate a result of evaluation with cycle points. This means it is possible to rely on a single point where cycles are detected. This is great, as cycle detection is complex! Previously, stack overflow detection was implemented in many components, each with their own peculiarities.
This means that cycle detection has been removed from many components. As a result, there may be bugs in the assumptions in components like the printer, for instance, that will still result in cycles. For instance, printing debug strings of values that are being evaluated may be treacherous. In general, though, the idea is that the evaluator clearly marks cyclic points in the graph, forcing an API to make a conscious decision about descending into a cycle.
Please submit bug reports if CUE hangs in an infinite loop or has a stack overflow
Note that although cycle detection is complex, it is not expensive. The implementation is based on unification algorithms that have cycle detection as a nearly free side-effect. The cost is currently minimal and can probably be improved still.
CUE now always evaluates an entire configuration value. This is required by the spec to be fully correct. It also simplifies things greatly, and is paramount in supporting certain new capabilities. It does mean that the API may have very different performance characteristics. cue.Unify may do more than expected, while cue.Validate now just collects errors from an evaluated tree so may run a lot faster.
An aim of the new evaluator was to remove any gratuitous exponential computation. The old evaluator could run into exponential behavior for aliases, for instance. The new algorithm is a non-copying unification algorithm: it therefore no longer needs to descend the graph to make copies. This was a performance quagmire in the old evaluator.
The new evaluator has been designed to allow for O(n) execution under certain circumstances (not using comprehensions or disjunctions without discriminators). It has not been implemented that way though! Currently the performance characteristics can be expected to be strictly better than the old implementation, though, barring having a worse constant in some cases.
That said, there may still be performance bugs that still need to be ironed out. Please report performance issues.
For one, there are still known cases where work is unnecessarily duplicated. The plan is to remove these so that each node only gets processed once.
{} / embedded scalarsThe new evaluator is in an advanced state of allowing embedded scalars. For the most part, removing the restrictions in the parser will enable it.
There are a few kinks to work out though. For instance, what does [] & {} mean? One could argue that {} means {_} (embedded any value) and thus that [] & {} means []. One could also argue that {} is a struct by default. But as per the spec, defaults are not resolved before applying &, so even then [] & {} should be considered legal to mean [].
The new evaluator currently aims to mimic the old semantics as much as possible, but error messages may sometimes be unexpected as a consequence of the partial implementation of embedded scalars.
0528376 all: adjustments package to work with new evaluator 7f52c10 all: switch to using cue port 0cfe411 cue/internal/adt: add builtin and import support bdb45b3 cue: fix misspells b2ea648 cue: hoist functionality to cue/internal/runtime 94fac7f cue: more renamings to minimize diffs 3efd10d cue: prepare to hoist index type a675387 cue: refactor marshal to simplify move 03092d9 cue: rename fields to ease transition to package adt 0dcc335 cue: simplify implementation b6a3a4b cue: use Kind and Op from Package adt e2a58e9 cue: use higher-level API when possible 1d29ac3 internal/core/adt: add Resolve helper 651d379 internal/core/adt: add path location for majority of errors f159888 internal/core/adt: apply Default also to list 88b4b1f internal/core/adt: distinguish dynamic fields from pattern constraints 7504519 internal/core/adt: finalize comprehension arcs 6a08830 internal/core/adt: initial commit 1f42c81 internal/core/compile: add new compiler cf94469 internal/core/compile: allow resolution in custom scope e986a8f internal/core/compile: separate out phase of let clauses 08a9fdb internal/core/convert: initial commit 6be2b4f internal/core/debug: add compact mode 17ade83 internal/core/eval: add Stats counters cfc6c8c internal/core/eval: add more positions for closedness errors d15def3 internal/core/eval: allow closedness override d857f23 internal/core/eval: centralize type checking b68adfe internal/core/eval: fix Environment linkage for Disjunctions 07e40c6 internal/core/eval: fix Evaluator sync bug 3acdde8 internal/core/eval: fix double increase of close ID d464582 internal/core/eval: fix multi-conjunct pattern constraints ff8373c internal/core/eval: fixes to closedness algorithm 711bbb1 internal/core/eval: handle structural cycles 0f935f8 internal/core/eval: implement core evaluator 9b4e65a internal/core/eval: tweak to disjunction 7903529 internal/core/eval: use cue's import support in tests 311f5bc internal/core/export: allow testing imports 18736bf internal/core/export: explicit generation of adt types 5378b08 internal/core/export: export let 0d12c33 internal/core/export: initial commit 7eafd8b internal/core/runtime: add Build method 5cf7756 internal/core/runtime: temporary helpers to aid transition 0ca4098 internal/core/subsume: initial implementation 38a19f8 internal/core/validate: initial implementation 489eb90 internal/core/validate: more cases to ignore concreteness 765f87e internal/core/validate: use defaults in concrete mode 6a495ae internal/core: add runtime hooks 50dac24 internal/cuetxtar: add WriteFile method 29dd250 internal/diff: use compact printing mode bc76c5f internal/diff: use field Selector instead of Name 76d8c61 internal/diff: use new definitions in test 8912ee5 internal/legacy/cue: copy relevant cue files for new implementation 7d54042 internal/legacy/cue: update to new build 5a51083 internal/legacy/cue: use UnifyAccept and some other changes fbc6f86 pkg/tool/os: check error of value 1dd08f3 tools/fix: remove support to rewrite old definitions dd312be tools/trim: fix list element removal bug
This is the last release before commencing the move to the new evaluator.
yaml.ValidateThis now uses Unification again. This brings it in line with the json package.
This change was done to facilitate moving to the new evaluator. This seems no longer necessary. Either way, using subsumption for validation seems wrong. It is better if the desired validation can be specified in the language itself.
This is the last version that supports :: under the hood and the last release that will allow moving from :: to new-style definitions using cue fix. The cue tool now supports the CUE_SYNTAX_OVERRIDE environment variable to specify the features to allow at a specific version. So even though this feature disallows :: by default, it can be allowed again by setting this environment variable to v0.1.0, for instance.
This will not re-enable features that have been permanently removed.
OpenAPI is not always compatible with JSON Schema. For instance, it may use an older version. CUE used the new interpretation of exclusive ranges, whereas OpenAPI v3.0.0 and before required the old interpretation. Both interpretations are now supported and are switched based on the selected version (an API feature).
cue completionThis release exposes the Cobra command completion feature. This is currently only tested for bash and is known to not work, for instance, for Fish.
cue get goThis will now generate unexported constants using the _# prefix.
Now allows a cue.Value to be set anywhere in a Go value.
list.SliceRunesAllow slicing of strings per rune.
Added ast.Embed helper to specify embeddings in ast.NewStruct.
Most txtar-based tests now support the CUE_UPDATE environment variable for updating tests. This allows running
CUE_UPDATE=1 go test ./...
to update all these tests at once. It is not possible to use the --update flag for a bulk update, because not all tests implement it, causing such tests to fail.
29fa119 cmd/cue/cmd: allow overriding syntax limitation ab43a15 cmd/cue/cmd: include tool and test files in fix 94db2eb cmd/cue/cmd: include unexported constants for get go f0f62b8 cue/ast: allow use of Embed in NewStruct 10fbc21 cue/ast: fix typo a20e1d4 cue/errors: don't equate error messages without line info 0ccdee2 cue/gen.go: fix bug in generation. 1784fca cue: add test cases 097cf16 cue: allow Value to be set at any path in Fill 8f07f84 cue: print hex numbers correctly ed61df2 cue: return empty struct for disallowed unification 21ca371 doc/ref/spec.md: alternative definitions for closedness and friends ecfbd72 encoding/openapi: date-time & string format 616d9e8 encoding/openapi: fix generation bug in CRD path d3c0096 encoding/openapi: update tests c615c91 encoding/yaml: fix bug where an empty document is not treated as null 9ebfa80 feat: add completion command 16a2b89 internal/cuetxtar: add Load function 97b215f internal/cuetxtar: allow CUE_UPDATE env variable. 8a0ac3e internal: add ToStruct helper. 57f8242 pkg/encoding/yaml: validate concreteness instead of instance of for Validate f01cfc5 pkg/strings: add SliceRunes
The new evaluator will no longer support some of the legacy syntax. Also, it will adopt JSON Schema semantics for bulk optional fields (to be called pattern constraints) and will have more flexible semantics for ellipsis to allow encoding “additionalConstraints” more easily.
To allow for a smooth transition, this release disables the older deprecated semantics and syntax to remind people to run cue fix to upgrade their configurations, as cue fix won’t work for these deprecated features once the new evaluator comes around.
To facilitate moving the API as well, LookupDef now automatically adds a # prefix if a field cannot be found and is not a valid definition in the new syntax. This is an unambiguous addition.
The current OpenAPI implementation used the current JSON Schema semantics for representing exclusive minima and maxima. OpenAPI v3.0.0, however, mandates the use of the deprecated encoding. The OpenAPI generator now generates such constraints using the legacy syntax. The OpenAPI generation API now has a Version field that allows enabling the new behavior if set to 3.1.0.
As for now, the JSON schema reader will accept the legacy encoding unconditionally in addition to accepting the standard encoding.
$ref changesOpenAPI defines that a$ref replaces all other fields in a schema. The JSON Schema RFC seems to allow at least some fields to be alongside $ref and there are some side comment notes that seem to imply that this is explicitly allowed. In order to avoid any confusion, however, $ref is now wrapped in an allOf if it may otherwise cause a conflict.
The original JSON Schema made the erroneous assumption that a constraint for a certain type implies that the value must be of this type. This is not the case. So
type: [string, integer]
minimum: 3
maxLengh: 4
will now be encoded as
strings.MaxRunes(4) | (int & >=3)
cue get goThe cue get go tool now also converts Go constraints encoding the field tags. This bring is in line with the API converters.
The tool now also has a --local flag to allow generating files in the Go package directory, rather than the corresponding cue.mod/gen subdirectory. This is not set to the default as to not break existing behavior.
ddc8eb4 .github: disable generation of schema
083afe7 all: update most tests to not use ::
78cc785 ci: fix vendor path of GitHub workflow definitions
c47316f ci: only run gorelease check on latest versions of Go
2f69bfa ci: re-add Go 1.12.x to the build matrix
019060d cmd/cue/cmd: extract cue constraints in tags in get go
4d31efd cmd/cue/cmd: get go: import files to cue module if in main module
8bfdfe3 cue/ast/astutil: add Expr to File conversion (ToFile)
f5315d4 cue/ast: don't require ellipsis to be at end of struct.
197f4c8 cue/ast: introduce NewNull()
53d18cc cue: allow omitting leading "#" for LookupDef
7efeb70 cue: deprecate :: and combining bulk optional fields with other fields
cb21170 encoding/gocodec: Validate helper function
7aa2eb7 encoding/jsonschema: add support for OpenAPI nullable
a91b869 encoding/jsonschema: constraints don't imply types
eb7706b encoding/jsonschema: fix printing of error message
91d267a encoding/jsonschema: record more line information
50b9067 encoding/openapi: wrap $ref in allOf when needed.
2a8b4ed encoding/openapi|jsonschema: allow bool for exclusiveNum
3e62450 go.mod: use github.com/rogpeppe/go-internal/testscript
519db58 internal/encoding: don't simplify interpreted encodings
1d213a3 pkg/list: remove use of :: (Incompatible!!!)
5515ee9 tools/fix: move fix functionality to separate package
This release makes several minor and bigger language changes. The changes are a result of analyzing CUE’s interoperability with other languages and what is needed for this. It is expected that these are the last major changes and that this fixes the look and feel of the language. In addition to the usual cue fmt-based rewrites, we now also provide a cue fix command to aid in the transition. Also, the old formats are still supported for now. Minor language changes and additions to the language are still planned on the path to stability.
The new language changes allow for a simpler JSON Schema mapping. This, in turn, allows for a more straightforward combination of schema and data on the command line.
Before, definitions were indicated with a double colon ::. They lived in the same namespace as regular fields. This property complicated defining automated conversions of some other data formats to CUE. For instance, JSON Schema keeps separate sections for schema and definitions. As there is no enforced convention, as in Go, for naming the two kinds differently, one could not map these to the same struct. This forced schemas to be moved to another location, which turned out to be cumbersome and unnatural.
Another issue with the old syntax is that one could not determine from a reference a.b whether b would refer to a definition or regular field. Not a big problem per se, but it lacked clarity.
As of v0.2.0, definitions are denoted with special identifier starting with a #. As this was not a legal identifier before, such identifiers can safely be used exclusively for definitions. Effectively, definitions now have their own namespace. Other than that, definitions work as before. They still close a struct, they are not output during export, and you simply use their identifier to refer to them.
Before:
A :: b: int
b: c :: string
D: A
E: b.c
After
#A: b: int
b: #c: string
D: #A
E: b.#c
The #c notation is just an identifier: no special syntax is needed to handle them. As before, "#foo" still denotes a regular field with a name starting with #.
Using an initial # to distinguish between definitions and regular fields was inspired by Go's use of initial casing to distinguish between exported and non-exported identifiers. Indeed because CUE is a JSON superset and interop focused, there is little control over casing. As we explain later in the Hidden Fields section, we also resurrect _ as a means of excluding fields and definitions from output.
The new notation may take a bit of getting used to, but the increased readability and the flexibility from the separate namespaces are big wins. So far the feedback has been overwhelmingly positive.
The Go, Protobuf, JSON Schema, and OpenAPI mappings, as well as the tutorials, have been ported to use #-style definitions. Please beware that if one has other templates based on this they need to be rewritten as well.
The new definition syntax is more restricted than the old syntax. In general, it is no longer allowed to have dynamic definition names. For instance, the syntax disallows using interpolations for creating names. Also bulk optional fields only apply to regular fields. We think this actually benefits static analysis and is overall a good change. If need be, though, the old semantics can be simulated by containing a struct within a definition used as a map. In case this proves to be too limiting, we have a possible language change up our sleeves that would allow dynamic definitions again.
Old style definitions will keep being supported up to the next minor release. Use cue fix to rewrite current files (see below).
The current API assumed definitions and fields lived in the same namespace. The current API is therefore inherently broken. The Struct.FieldByName has been deliberately broken to force users to disambiguate. To evaluate a reference, it is recommended to use Dereference instead of passing the result of Reference to Lookup.
There are some good ideas to make the API considerably simpler and more powerful. This is contemplated to be built on top of the new adt package, which is developed as part of the evaluator rewrite, which is planned for the next minor release.
cue fixTo aid in the conversion to #-style definitions, the cue fix command is introduced. This command does a module-wide update of packages by default, but allows specifying individual files and packages.
This is no longer piggybacked on cue fmt. Unlike previous rewrites, the definition rewrite requires evaluation. In addition, not all current definitions are legal in the new syntax. The converter will not remap those. Also, unfortunately, due to API limitations, the converter can also not handle all legal cases. We hope that such cases will be rare, though.
Cases that cannot be handled are clearly marked with locations that may reference the definition included. Definitions with issues have are marked with a deliberately obscure @tmpNoExportNewDef attribute, indicating they need manual fixing
There is also a possibility there will be undetected breakage. This may happen if a selection is made in a disjunction where the field may be either a definition or regular field. The fact that this is even possible indicates that the old model was probably too flexible and possibly more bug prone. With new-style definitions such shenanigans are no longer possible.
Unfortunately, cue fix currently does not handle updating multiple packages within the same directory at once unless manually specified. We intend to allow this at some point.
Previously, the cue tool handled schemas and data differently. In practice, the distinction is not all that clear, making this distinction somewhat “forced”. The cue tool now treats data and schemas as the same thing and always unifies them without distinction by default. This greatly simplifies its use and makes it a very powerful tool for operating directly on non-CUE schema.
There are two exceptions to the new “always unify all files” rules: 1) with cue vet multiple data files are still individually verified against a single schema. 2) the -d option still separates schema from data files. One could argue that the latter isn’t really necessary anymore, as we will see next.
The following command
cue export schema.json data.yaml
converts the schema.json (which has a proper value for $schema to detect the format) to a schema and data.yaml to data, and unifies the results. This will naturally fail if the contents of data doesn’t correspond to the schema. There is no special logic needed to detect these cases, other than knowing when to interpret a data file as JSON Schema.
Note that #-style definitions were key to make this possible. Without them, there would be no obvious way to map JSON Schema to the root of a config as merging fields and schema to the same namespace could result in conflicts.
Another example of the new capabilities is that one can now merge schema as one would merge data. For instance, the following command merges two OpenAPI files and then outputs it again as OpenAPI:
cue def openapi: file1.json file2.json --out openapi
The user won’t even see the intermediate CUE.
This release also officially resurrects hidden fields (identifiers written as, for example, _foo). They were removed from the spec as with the introduction of definitions they were believed to be an incongruent feature. They fit, however, quite nicely with the new-style definitions. They were still in use and proved to be more useful than expected. As they were never really removed from the implementation nothing changes. However, their comeback is now official.
For clarity, both hidden fields and definitions are not shown in exported output. They serve different functions, though.
A definition defines a new composite type, the most general specification of something. They are at the other end of the spectrum to concrete instances. As definitions are supposed to fully define a schema, CUE can use them to catch typos in field names by detecting fields that are not supposed to be there.
A hidden field can be any value and is not subjected to such scrutiny. They are used to define fields that are not converted to output without making them a definition.
Although this is not yet enforced, hidden fields are local to a package. To make the matrix complete identifiers starting with _# denote hidden definitions, or definitions local to a package. These can be used today, although this restriction is also not yet enforced.
List comprehensions are now written as
[ for x in src { x + 1 } ]
This brings them in line with field comprehensions, where the value also comes after the comprehension clauses. Overall, this greatly increases readability.
This also harmonized the syntax. It simplifies both spec and implementation, and prepares for some of the constructs mentioned in the query proposal.
The one thing that may require some getting used to is the curly braces alluding to the value being a struct. If the curly braces contain a single scalar value, however, it is promoted to be the result of an iteration. CUE users may already be familiar with this construct at the top-level scope, where “embedded scalars” are the final result of evaluation. This construct was originally invented to allow omitting curly braces, while still being a superset of JSON (JSonnet uses the same trick). It turns out, however, that this trick can be quite useful when generalized to apply to any struct, not just the top-level one. We continue to investigate this possibility. Consider this the first application.
Using cue fmt will automatically update old list comprehensions to use the new format.
Previously, it was allowed to have a bulk optional field alongside regular fields within a struct
a: {
[string]: T
foo: S
}
Many people expected T to only apply to fields other than foo, but not foo itself. This is, in fact, how JSON Schema’s patternProperties and additionalProperties work. With hindsight, this approach turns out to have many benefits.
The plan is to transition to these new semantics. We will first disallow bulk optional fields alongside other fields altogether. This can easily be worked around by singling out such fields by wrapping them in curly braces:
a: {
{ [string]: T }
foo: S
}
Bulk optional fields of this style
a: [string]: int
are effectively already singled out and can remain unchanged.
The resulting embedding means the same thing in CUE. After the original semantics of mixing these fields has been banned for some time, we can introduce the original syntax, but now with “fallback” semantics analogous to JSON Schema.
For this release cue fmt simply rewrites bulk optional fields to be singled out. Forcing the rewrite is planned for a next release, followed by dropping support, waiting and then changing the semantics.
A noteworthy addition to the API is astutil.Sanitize. This function aids automated rewrites of CUE ASTs. Adding fields to arbitrary CUE ASTs may inadvertently shadow references. Inserting references to imports may cause similar issues. Not handling these cases correctly was a common source of bugs. Sanitize shares code with the actual resolver and can therefore be expected to be more accurate than any parallel implementation.
The Sanitizer automatically detects and resolves shadowed fields and fixes the AST in place accordingly. It uses the following trick: like in Go, the CUE resolver uses an extra field in ast.Ident (a reference) to mark what an identifier points to. The Sanitizer runs resolution on an AST which already may have such fields filled out from previous resolutions. If the result is not consistent, it will modify the AST to make it so.
Users can manually set such resolutions as well. For instance, setting the resolution to an ast.ImportSpec will cause the spec to be inserted automatically or the node to resolve to a preexisting equivalent one.
More functionality can be added in the future. The key is that all converters can share the same logic. It has already been used to fix import inserting in JSON Schema and to fix import disambiguation in the protobuf converter.
CUE uses GitHub Actions for its CI. The configuration for that was YAML. The lack of validation causes some breakages here and there when these needed to be adjusted. The configuration is now written in CUE! We'll let you know how it goes.
We also now have a TryBot for the Gerrit repo, which should also prevent breakages in the future.
Space-separated labels are now no longer supported and will give a compiler error complaining about a missing :.
Legacy (“back-style”) field comprehensions are now also no longer supported.
Some APIs have deliberately been broken to force handling the distinction between identifiers and strings.
f1c757f Update broken link in tutorial docs README.md
238d821 all: re-run go generate using Go 1.14.3
4c7d062 ci: add -f (fail) flag to curl calls in GitHub Actions trigger
37b1de2 ci: define repository_dispatch build
6bbacf9 ci: fix bad testscript file name and update CI with module get check
8fcefc8 ci: fix error in workflow generation task
8f1d260 ci: fix the GitHub actions repository_dispatch workflow
99f4ce0 ci: fix up Yaml error in specifying constraint on generate step
063fa12 ci: fix up some build breakages
3527700 ci: re-run go generate against stable Go version as part of CI
a801188 ci: tidy up and refactor GitHub workflow specifications
65163a0 ci: use CUE for GitHub Actions workflow specifications
a846fcc cmd/cue/cmd: add fix command for new-style definitions
d8d240d cmd/cue/cmd: bug fix: allow vet to combine schema and data
c114dbb cmd/cue/cmd: fix build breakage
dc62e36 cmd/cue/cmd: fix typo 'filetype' -> 'filetypes'
8f09dde cmd/cue/cmd: fix windows test breakage
50bcd97 cmd/cue/cmd: fmt rewrites alias to let clauses
5114891 cmd/cue/cmd: merge data files by default
5163831 cmd/cue/cmd: print full error messages in tools mode
99117ba cmd/cue/cmd: remove unused func mustParseFlags
8be0ffe cmd/cue/cmd: update "get go" to use new-style definitions
7ed2fc1 cmd: fix cli.Print usage in docstring
b19b41a cue/ast/astutil: add and use Santize function
00a8bb9 cue/ast/astutil: refactor to prepare for Sanitize func
afe86c1 cue/ast: allow LetClause in NewStruct
b70cc97 cue/ast: print StructLit in regular mode by default
a5c0866 cue/build: fix typo reference to cue/load package
9954ecd cue/cmd/cue: disallow quoted identifiers by default
aeb0a8c cue/format: don't simplify when parent field has comment
8938f35 cue/format: fix formatting of generated ImportsDecl
989e984 cue/parser: allow keywords in more places
8040ce7 cue/parser: remove support for legacy field comprehensions
c1c4c63 cue/parser: remove support for space-separated labels
7491622 cue/testdata: replace spaces in txtar file names
dbf1c00 cue: allow # and disallow #0 as identifiers
56c994d cue: disallow bulk optional fields with other fields
1370f0a cue: implement "front-style" list comprehensions
724da18 cue: introduce let declaration
fb7bdab cue: make test for Expr more precise
dd262ee cue: merge attributes for top-level fields
12927e8 cue: prevent crash in JSON conversion
0240d4e cue: some API adjustments for new-style definitions
9c9cdba cue: support for #-style definitions
f416ee8 cue: support new-style definitions in LookupDef
8837685 cue: uniform handling of definitions
b7083ff cue: use _# instead of #_ for hidden definition
e027d80 deps: put all tool deps in root go.mod file
c174a08 doc/ref/spec.md: allow ... in an ast.File
86e1a64 doc/ref/spec.md: fix doc bug in Label production
cb8f4f5 doc/ref/spec: alternative syntax for definitions
de0c53d doc/ref/spec: harmonize list and struct comprehension
f395f12 encoding/jsonschema: fix a few json schema bugs
b5b7521 encoding/jsonschema: improve handling of $id and $ref
faf1dfc encoding/jsonschema: introduce extra pass for metadata
53e5561 encoding/jsonschema: use astutil.Sanitize to handle imports
435989a encoding/jsonschema: use new definition mapping
a85238c encoding/openapi: make title and version defaults
98aaff9 encoding/openapi: update to new-style definitions
0212bb9 encoding/protobuf: fix typo in example
2b118fb encoding/protobuf: move to new-style definitions
3970555 encoding/yaml: use idiomatic indentation (regression)
be0649e internal/filetypes: print attributes for definitions
4d8d154 internal: factor out FileComment code
0d4abd7 testdata: convert old tests to txtar format
be60cd9 tools/trim: fix package doc
This release contains several fixes and enhancements related to CUE conversions.
It fixes several issues with the related to JSON Schema conversion, including an incorrect implementation of the JSON Schema spec. Most notably, one may not infer from a constraint for a certain type that a value must be of that type. This means that if a schema, for instance, defines a properties section, but does not specify the type to an object, the resulting CUE value will be a disjunction from this object and all non-object top-level types. This is not a bug.
Another change is that the cue tool now may silently ignore imprecise conversions, as long as the produced result does not give false negatives. The --strict option will emit the original errors.
0059b2b cmd/cue/cmd: allow lossy conversion by default and add --strict option a38e811 cmd/cue/cmd: fix bug with help flag b99d364 encoding/jsonschema: add several validators bcc19ce encoding/jsonschema: encode minItems differently 78e21c1 encoding/jsonschema: fix type exclusion logic ed4fc4d encoding/jsonschema: wrap comment lines 904f561 encoding/openapi: remove unused code 9b62b36 encoding/protobuf: fix required option parsing efb89b8 github: fix release.on to push.tags
The release fixes a critical regression that affected cmd eval as well as various other minor regressions and documentation and formatting issues.
Support for space-separated labels and old-style field comprehensions is now disabled by default. The intention is to remove these entirely soon.
This release now also allows OpenAPI inputs to be expressed in CUE itself, instead of just JSON or YAML.
6f37a71 cmd/cue/cmd: fail earlier on faulty flags 83b0996 cmd/cue/cmd: fix newline discrepancies cd2b3ef cmd/cue/cmd: fix regression that allowed space-separated fields again d6fc6be cmd/cue/cmd: link commands help text c5fc236 doc/tutorial/kubernetes: update to minimize diffs down the line 8dcec3c internal/encoding: allow CUE as OpenAPI input d6f1eec internal: fix crash affecting eval
This release introduces various improvements to the cue command, including a streamlined CLI interface and improved interoperability between different file formats.
Many of the command-line flags were previously rather arbitrarily implemented for some commands and not others. This release fixes most of that. This release also makes it possible to convert between different formats, such as YAML, JSON, JSON Schema (input only), OpenAPI, and Protbuf (input only).
This release also starts a new versioning strategy. CUE exists of several fairly independent parts. Each next milestone focusses on a one of these parts. This round it was the cue command-line tool. Next round will be the evaluator. Each next minor version will correspond to one such milestone. This is not to say that upgrades and bug fixes cannot happen to other parts within a cycle—they can—but it helps focus.
The new strategy will also make it easier to do intermittent releases. We will use the minor releases to signify big additions and backwards-incompatible changes. Patch releases may include backwards-incompatible changes that include bug fixes or disabling previously deprecated features. We intent to follow this trategy until providing a general backwards-compatibility guarantee for v1.0.0. As usual, we intend to provide a smooth migration path for each incompatible language change.
As of this release, a Docker image is generated as part of the release process.
A frequently asked feature was to be able to interpret files as anything. CUE now allows qualifiers to "cast" a file to any supported format. For instance,
$ cue eval json: foo.data
interprets foo.data as JSON and then prints it as CUE. This also shows that the input for eval no longer needs to be CUE but can be any format that is convertible to CUE.
This means that CUE can now also be used between non-CUE formats. For instance,
$ cue export foo.json -o foo.yaml
will convert JSON to YAML.
Packages can now be qualified by package name (for instance ./foo:bar), not just directory.
Run cue help inputs and cue help filetypes for more information.
The import command already had a set of flags that maps JSON and YAML files to a prescribed location within a CUE configuration. These flags are now also available for many other commands so that JSON and YAML files, as well as other formats, can be processed by the cue command without first having to convert them to CUE.
For instance,
$ cue eval -l 'strings.ToLower(kind)' -l name foo.yaml
puts any document in foo.yaml at a path based on its kind and name fields.
Also the output flags (file and format), among others, have been harmonized across commands. This sometimes led to backwards-incompatible changes.
Run cue help flags for more information.
The qualifier mechanism can also be used to restrict what language constructions can be used in .cue files. Most notably,
$ cue eval data: foo.cue
specifies that the file foo.cue may only contain data fields, but no references, definitions, builtin invocations, or any other CUE construct other than simple data. In other words, this allows CUE to be used as an alternative JSON or YAML format.
In the future we may add a file extension to indicate this CUE dialect.
cue def commandThe cue eval command was configured to generate cue that seemed pleasing to read to humans, but it would generate something that is neither useful as data nor as a schema. cue export could already be used to generate data, but there was nothing to conveniently generate a schema.
cue def is also the command of choice converting schema-based formats to CUE.
Many users have requested a convenient way to defining environment-specific variants of configurations and ways to inject values based on environemt variables and command line flags.
We have added support for environment variables at various points in the tooling layer. Due to their non-hermetic nature, though, environment variables are not an appropriate mechanism for CUE configurations in general.
An important criterion for allowing such injection was retaining the ability to analyze configurations and make intent of how to use a configuration explicit. Simultaneously, we did not want to impose on the structure of a configuration.
Our solution is to use attributes to explicitly mark injection points. A field that with a "tag" attribute of the following form:
environment: "prod" | "staging" @tag(env)
can be set to "prod" on the command line using -t env=prod. The tag attribute may also explicitly define some shorthands. For instance, by specifying
environment: "prod" | "staging" @tag(env,short=prod|staging)
the same can be accomplished using -t prod.
This mechanism makes it very clear to both the user and the cue tooling which non-concrete values may be set by the user and which should be resolved by values within a configuration.
In the future we may consider to also allow whole-file injection analogous to Go build tags.
This release does not include support for command-line flags in the tooling layer. This is still being considered for a future release.
Run cue help injection for more information.
The cue tool now has basic support for converting OpenAPI definitions to CUE. This is a first stab, does not yet support external references, and is not as feature rich as the CUE-to-OpenAPI Go API that is used by Istio to generate its OpenAPI. But it should be a good start.
OpenAPI definitions can be imported using cue import or just used on the fly using cue eval or cue def. OpenAPI can be defined in JSON or YAML. For instance:
$ cue def openapi: foo.yaml
will interpret the foo.yaml file as OpenAPI and generate corresponding CUE.
It is also possible to convert CUE to OpenAPI. OpenAPI can be represented in JSON, YAML, or even CUE. For instance,
$ cue def ./pkg -out openapi+yaml
writes the evaluated CUE as OpenAPI in YAML format to stdout.
The cue tool has very basic support for mapping JSON Schema to CUE. This is still experimental and the converse (mapping CUE to JSON Schema) is not yet supported. The typical conventions followed in OpenAPI make it easy to find a mapping from OpenAPI to CUE. The same does not hold for JSON Schema. This is partly complicated by fields and definitions sharing the same namespace in CUE. Also, the top-level schema of JSON Schema is unnamed, making it hard to find an appropriate destination.
There is a proposal for CUE to have definitions and fields live in a separate namespace. This will make such a mapping considerably easier. For now, consider the mapping experimental.
As with OpenAPI, external references and anchors are not yet supported.
The cue tool now interprets files with a .json, .yaml and .yml extension as either data, OpenAPI, or JSON Schema based on markers in the data. For JSON Schema this is a $schema field that refers to a schema on json-schema.org. OpenAPI is selected if a file contains a top-level openapi field with major version 3, an info.title, and an info.version field.
The use of qualifiers disables auto detection. So auto detection for a JSON file, for instance, can be disabled using a json: or data: qualifier.
cue import modesThe cue import tool now has import modes that specify which kinds of files to import. The default is the old default: JSON and YAML files, but with auto detection enabled.
Run cue help import for more details.
The new cue import proto command can import Protobuf definitions.
A successful run imports all transitive dependencies of a .proto file into a CUE module: .proto files located within the module are converted to a CUE package at that location. The package name is derived from the go_package option, if present. Imported .proto files that can be found in a given include directory are put in the corresponding path in the cue.mod/gen directory.
The existing Protobuf API has been modified to generate CUE types hierarchically (instead of mapping all files to the top level) to make them closely correspond to the Protobuf definitions.
Protobuf definitions can be annotated with CUE-specific options to specify additional constraints for fields.
The task section in commands is now optional. A task can be included anywhere in a command. So commands with a single task can now be written as
import "tool/cli"
command: echo: cli.Print & {
text: "Hello world!"
}
The cue tool will look for tasks anywhere nested within the run command where each task is considered a leaf node in this nesting.
Tasks can also be defined at the top of the file. These will only be run if they are referenced by a task defined in an executed command
Sometimes task must run after another task, even though it does not depend on any of its outputs. The $after field can be used in that case to explicitly mark a dependency.
See https://pkg.go.dev/cuelang.org/[email protected]/pkg/tool?tab=doc for more information.
In addition to the flag restructuring, it is now possible to check a data file against a CUE package.
A backwards-incompatible change is that one now needs to use the -d flag, instead of the -e flag, to select a schema against which to compare. The -e flag was used by other commands to mean something different and this was cleaned up for consistency sake.
The cue trim command now checks that the resulting rewrite results in the same output when exported. This prevents a bug in the trim algorithm from inadventently compromising a configuration.
Run the command with the -i flag to bypass this check.
Go API:
cue/ast: Many new AST construction helperscue/literal: Added CUE number parsing utilitiescue: Added LookupDef, Dereference, IsClosed and various other methods.cue: Fill now accepts Value.CUE API various additions to
pkg/encoding/yaml
pkg/file
pkg/list
pkg/math
pkg/regexp
pkg/tool/exec
pkg/tool/os
@foo(bar) can be used as a declaration.)Changes in flags:
--attributes flag has been renamed to --show-attributes to be consistent with the --show-hidden and --show-optional flags.-e flag for cue vet has been renamed to -d. The -e flag was already used for other commands and means something similar, but different. This allows -e and -d to be simulatenously supported.-o flag now always defines an output file and the -out flag now always defines an output format.API changes:
cue: Instance.Name was changed to Instance.PkgName.cue: Iterator is now returned as a pointer.cue: Syntax now includes a package clause if a package is defined.encoding/protobuf:
oneOf, which previously required at least one of the one-of fields to be specified..proto files. It previously adopted the Go convention, mapping all types to the top-level.There were several bug fixes. Some of them might break your configuration. For instance, a regression was introduced that allowed required imports to be omitted in *_tool.cue files.
e6c7748 .github: small improvements to the bug template
58cdb77 .github: update issue templates
fb54e1b Add help text to show yaml export option
0227ac0 Fix typos in 'cue mod'
066ede6 all: gofmt -s
30a4bee all: make goimports happy
52cc7f1 ci: rebuild tip.cuelang.org on master commits
0846b93 cmd/cue/cmd: add --tags flag
d10e59f cmd/cue/cmd: add --with-context flag for import
116e9fd cmd/cue/cmd: add def command
af71ef2 cmd/cue/cmd: add injection help topic
5208956 cmd/cue/cmd: add safeguard to trim operation
67d084a cmd/cue/cmd: allow commands to be documented by comments
f89aa48 cmd/cue/cmd: allow decoding JSON Schema and OpenAPI
a920a11 cmd/cue/cmd: allow mix of data and package files
694303f cmd/cue/cmd: allow specifying field type for path flag
4d5f5c7 cmd/cue/cmd: allow task hierarchy
511584e cmd/cue/cmd: allow top-level tasks
38a114d cmd/cue/cmd: bail out from errors earlier
4d6e269 cmd/cue/cmd: consolidate placement logic to single file
e3a7a72 cmd/cue/cmd: control task dependency using before and after fields
4616a3b cmd/cue/cmd: enable placement logic for vet, export, and eval
3b17f6a cmd/cue/cmd: err on space-separated syntax
67e25d4 cmd/cue/cmd: eval flag changes
eb7c51d cmd/cue/cmd: factor out common flag code
7a02899 cmd/cue/cmd: fail in case flag evaluation fails
5a8279f cmd/cue/cmd: fix Windows compatibility of test
00757a9 cmd/cue/cmd: fix crash
8af2efe cmd/cue/cmd: fix description for --show-optional flag
27b9ca3 cmd/cue/cmd: fix export status and output
485f389 cmd/cue/cmd: fix stdin usage for import
6eed1e9 cmd/cue/cmd: harmonize output flags
e7e0139 cmd/cue/cmd: hoist expression logic into iterator
8fbf67e cmd/cue/cmd: improve $after semantics and errors
1974705 cmd/cue/cmd: introduce config struct
2beaa9c cmd/cue/cmd: introduce help topics for new CLI
de9cc69 cmd/cue/cmd: make error messages identical on Windows
6dfecc3 cmd/cue/cmd: make eval use output flags
dd87908 cmd/cue/cmd: make task references absolute
ed5689f cmd/cue/cmd: move export to filetypes
337e2d1 cmd/cue/cmd: move fmt to new infrastructure
6e48383 cmd/cue/cmd: move get to to use the ast
abb5401 cmd/cue/cmd: move import to filetypes
01fbfa1 cmd/cue/cmd: move vet to filetypes
bfdc423 cmd/cue/cmd: prevent infinite loops while building dependency graph
c82a793 cmd/cue/cmd: require imports for tooling
f8afaed cmd/cue/cmd: simplify usage of --path flag
f733746 cmd/cue/cmd: streaming instance iteration
dcffaef cmd/cue/cmd: support -e for export
d4ce8ef cmd/cue/cmd: support import of proto and openapi
77ffe9d cmd/cue/cmd: support openapi out and proto in
a32d5dd cmd/cue/cmd: track dependencies in definitions
d8757db cmd/cue/cmd: unshadow type names in get go
882d88d cmd/cue/cmd: update cmd documentation
88b9bd5 cmd/cue/cme: fix import error message regression
bd5faf9 cmd/cue: fix cue import -p
3db1509 cmd/cue: make fmt work on tool files
6ad04b3 cmd/cue: use module information if no -X version information specified
2b93e11 cue/ast/astutil: catch cycle for label expressions
e076cd1 cue/ast/astutil: improve LabelName logic
8c70185 cue/ast: add NewBinExpr helper
e2650e0 cue/ast: add NewBool helper
e803ab8 cue/ast: add NewLit helper
b236d4a cue/ast: add NewStruct helper
d4faf4b cue/ast: add PackageName method
e2e548b cue/ast: allow CommentGroups in NewStruct
8e11671 cue/build: remove unnecessary duplicate initialized
6409c68 cue/cmd/cue: hoist placeOrphan from import
33c7705 cue/cmd/cue: hoist placement logic from import
442518f cue/errors: make compile again with Go 1.12
a96b3bb cue/errors: remove unnecessary first slice length on list.Reset()
e7f46c0 cue/format: always remove space before ':'
d05ea95 cue/format: fix simplification bug
af31654 cue/format: fix string label rewriting
78c24c5 cue/format: only format valid identifiers
ffd21e6 cue/format: print empty struct on one line by default
9b2f6b0 cue/format: replace bulk labels with ellipsis if possible
cb1baf4 cue/literal: factor out number parsing code
fcdf3fe cue/load: allow mixing anonymous packages in files mode
28d5c83 cue/load: allow using path:pkgname on command line
1288f89 cue/load: expose error types
4fd96d9 cue/load: factor out addFiles.
86a67bc cue/load: fix test breakage for Windows
f6f7791 cue/load: remove unnecessary colon check
104d266 cue/load: remove unnecessary import alias
334313c cue/load: use build.File to support "-"
a16973a cue/parser: fix parsing bug
9b301e4 cue/scanner: completely remove block comment support
83fa8f4 cue: add '-local cuelang.org/go' flag for buildins.go
024f1f5 cue: add Dereference and fix value paths
53ca13f cue: add IsClosed method
c7c70a6 cue: add Subsume to replace Subsumes
060a3c8 cue: add __foo variants for predeclared identifiers
2ca34b3 cue: add a test case for list.Equals
d56457f cue: add fallback mode for unresolved disjunctions
d9e4b53 cue: allow a Value to be passed to Fill
2540e8a cue: allow alias reuse in subscopes
72d8a92 cue: allow export of schema and other options
b9d22d6 cue: allow schema mode in subsumption
58ada0e cue: define Fill on Value, LookupDef and bug fixes
1fecca8 cue: detect incomplete value for value method
cc60dee cue: do not evaluate list elem for evalPartial
1e76e22 cue: fail when converting invalid UTF-8 to a CUE string
847d37b cue: fix Issue 289
76a47bc cue: fix Value.Subsumes example to correct function naming convention
5119440 cue: fix bug that dropped optional fields in def
e084198 cue: fix compiler bug and improve debug print
5ad8510 cue: fix crash if yielded value is a disjunction
ab2c7e5 cue: fix detection of incomplete value
8108ef2 cue: fix export bug
8398107 cue: fix handling of incomplete calls
76252f4 cue: fix import usage detection for partially resolved files
72dba3d cue: fix marshaling bug
d06f2d4 cue: fix number output
aa5e0c7 cue: fix struct instance matching of references
0a6f7c9 cue: implement attribute declarations
442bf2d cue: interpret nil as _ or top
c3c1624 cue: keep path in API error value
1ffc615 cue: limit number of position errors
b992abe cue: prioritize missing field over incomplete errors
fe9722f cue: rename Name to PkgName and add DisplayName
31206eb cue: return pointer from Struct.Fields.
b0678ab cue: simplify numLit
f4649b2 cue: support export package clause and docs
a805655 cue: track incompleteness across marshal errors
4ec09a4 deps: upgrade cobra to v0.0.7 to use new SetIn/SetOut
0cb140e doc/ref/spec.md: reformat to deal with Hugo bug
4dd9630 doc/ref/spec: attribute changes
107863a doc/ref: fix a go-ism in the spec
6320089 doc/tutorial/basics: change title of section
0e4130f doc/tutorial/basics: remove reference to % operator
275105a doc/tutorial/kubernetes: a few fixes
7e3bdda doc/tutorial/kubernetes: fix eval command typos
3171634 doc/tutorials/basics: remove duplicate text
7739e72 doc/tutorials/basics: typo
d64abbe docker: add Dockerfile and .dockerignore
0308a53 encoding/gocode: make work for definitions
b756712 encoding/json: don't simplify hidden fields
e752347 encoding/json: fix Doc comments for Extract and Decode
f81c40b encoding/json: fix JSON import for numeric keys
671b956 encoding/jsonschema: add features to support OpenAPI
47d9870 encoding/jsonschema: expose API of JSON Schema decoder
60b4d91 encoding/jsonschema: fix Windows path issue
6cb0878 encoding/jsonschema: initial JSON Schema to CUE conversion
bd98b13 encoding/jsonschema: require explicit package name
3c437bd encoding/openapi: encode title, version, and contact as fields
f51c879 encoding/openapi: generate CUE expressions
64b39e6 encoding/openapi: implement Extract
5c2b086 encoding/openapi: make API backwards compatible for Istio
eac3ea2 encoding/openapi: more accurate handling of disjunctions
56509a5 encoding/openapi: sort schema alphabetically
ed90d00 encoding/openapi: support nested defintitions
439f857 encoding/openapi: supported nested definitions
09ad0d2 encoding/protobuf: allow specifying package
f9fa4d8 encoding/protobuf: define messages as closed
0797e46 encoding/protobuf: don't rely on verbatim Ident formatting
c3efeb0 encoding/protobuf: generate nested definitions
0f03725 encoding/protobuf: proper encoding of oneOf
08c8a8f encoding/protobuf: remove dependency on load
cf72796 encoding/yaml: eliminate use of ghodss yaml pkg
ede1e7d github: cache ~/go/pkg/mod
d9ca690 github: enable docker feature
2e67318 github: enable run goreleaser automatically when bump new tag
c24f394 github: fix lack of v prefix on build_args
d0a6eef go.sum: run go tidy
aae80a0 internal/cli: hoist cli parsing code from tool/env
7988b62 internal/cmd/qgo: update Copyright year to 2020
29e9d00 internal/diff: avoid cycle when diffing
5c3a17b internal/diff: fix bug that caused hang
1d1295b internal/encoding/json: implement Encode
40cc12c internal/encoding/yaml: fix go1.12 breakage
975776e internal/encoding: always format with tabs
6e8fdd4 internal/encoding: support cue file type
81a20d5 internal/encoding: support input validation
f9f8e62 internal/filetypes: add auto interpretation
9b1dff7 internal/filetypes: add options and mode defaults
38362f8 internal/filetypes: better error message for files without extension
bd2173e internal/filetypes: expose package test internally
26e63ec internal/filetypes: fix Windows breakage
429bd0f internal/filetypes: package for interpreting command line args
c1b5e62 internal: add MakeInstance
4805941 internal: add ToFile and ToExpr
b8cad1a internal: hoist attribute parsing logic
a494f2e internal: switch defaullt GenPath to be in cue.mod
0198c89 module: update google/go-cmp to v0.4.0
9e71832 pkg/encoding/yaml: request concrete data for syntax
a9b38db pkg/encoding/yaml: validate using instance check
9242bf5 pkg/file: Fix file.Glob to generate correct CUE structure
c441dd3 pkg/list: add sort functionality
df7da7b pkg/list: documentation fix.
0afa377 pkg/list: fix typo
fe7a58a pkg/list: pick default before flattening
7008195 pkg/math/bits: reduce maximum bit index
2db865d pkg/regexp: add Valid function
72f8e8a pkg/tool/exec: allow passing env vars in exec.Run
a3a4b71 pkg/tool/exec: fix stdin processing
05a3f7d pkg/tool/exec: use os.Stdin on null only
42346df pkg/tool/os: add tasks for environment variables
c22d46d pkg/tool/os: remove env manualipulate tasks
a7b4ca8 pkg/tool: improve error reporting
2d8560d pkg/tool: update documentation and remove $before
44df1a8 pkg/tool: use $id instead of kind to identify task types
d1cfa57 tools/trim: fix build breakage
c242505 tools/trim: fix trimming bug
fd90d12 tools/trim: undo faulty optimization
This release focusses on bug fixes and stability improvements.
There is one language change, and that is to allow the $ character in identifiers. The main reasons for this is that it is a commonly used character in identifiers in JSON. The single $ is currently reserved.
6f9c049 Add YAML support to export command
503fecd Decode null as NULL token in yaml decoder
c4f74cc Revert "cue: fix lookup for cue API"
8a043dc Use baseValue from source in binOp to make gc working
4dc023b Use cache during walk
170697a cmd/cue/cmd: also make field optional if it is a pointer
d372ba8 cmd/cue/cmd: convert package block comments in cue get go
d100ae6 cmd/cue/cmd: small help command fixes
a9fdbe7 cmd/cue/cmd: track use of indexing op for dependencies
f16c065 cue/format: improve indentation of in comprehensions
8e72f0b cue/load: cleanup and made code more defensive
ff96e8d cue/parser: improve error message for misused keywords
8a2df96 cue/scanner: allow $ as a letter in identifiers
b3cde9d cue: Reference handles concrete values for index operations
3a2e63b cue: always check for errors before calling builtin functions
630935e cue: detect errors in struct when comparing to bottom
afdbec3 cue: disallow references to string labels (per spec)
5c2d937 cue: don't turn of manifesting in normalization
324cc0f cue: don't validate optional fields
279ca6a cue: fix bug related to non-structural direct self-references
13628f1 cue: fix compilation bug using $foo identifiers
3d0b204 cue: fix crash on alias use in lists
40d2667 cue: fix exporting of label aliases
c937e7a cue: fix lookup
353ede2 cue: fix lookup for cue API
8bd2426 cue: fix lookup for cue API
52b04d4 cue: fix optional handling in subsumption
6cd62ea cue: ignore optional fields for Equals
5a0a3f2 cue: no quotes needed for identifiers with $
d256a2c cue: prevent regression to bug in older version
030b9b7 cue: remove uses of log.Panicf
d7d2837 describe the go get step before cue get go in the tutorial
c8d6c39 doc/ref/spec.md: clarify newline behavior for multiline strings
c6e5d17 doc/ref/spec.md: fix typo
4d29dde doc/ref/spec.md: simplify attributes
0e038f6 doc/tutorial/kubernetes: update to latest
3960473 doc/tutorial/kubernetes: update to use new syntax consistently
8f9ef31 doc/tutorial/kubernetes: update to v1 versions
815dca2 doc/tutorial: fixing bad link in string literals
80ffe27 doc/tutorial: removing duplicate word in bounds
5b4fa8b doc: fix typos in language spec
085ce9c fix typos in docs
31cd2b7 internal/diff: first stab add diffing functionality
5cb6ffe internal/yaml: import null value properly
7fdff9c pkg/math/bits: complete operations
7334a13 pkg/math: use higher precision implementations
6ffe55d tools/trim: fix spurious removal of top-level declarations
cba0372 tools/trim: move trim logic to package