JSON to JSON transformation library written in Java.
Jackson Dependency upgrade squashNull feature fix
BELL-5125: Fix security vulnerability in Jolt and release a new version Add support for Object[] to Function Added support for Object[] in Cardinality spec Update README.md Update OWNERS
BELL-5125: Fix security vulnerability in Jolt and release a new version Add support for Object[] to Function Added support for Object[] in Cardinality spec Update README.md Update OWNERS
String manipulation
leftPad
and rightPad
trim
substring
split
Math
intSubtract
, doubleSubtract
, and longSubtract
Object Manipulation
squashNulls
- works on Lists and MapsrecursivelySquashNulls
- recursively works on Lists and MapsFixes for
size
modify functionNo other code changes. We had to move Jenkins build machines, took this opportunity to update Java, maven, dependency versions.
More "modify-beta" functions : "divide" and "divideAndRound".
Turns out the "elementAt" function did not break, it was just that params in the "method signature" were flipped.
This release is being used by Bazaarvoice in production.
Do not use this release if you use the "modify-beta" transform.
Historically, Jolt has been focused on fixing / operating on the "format" of the input JSON. It did not have a good solution / Transform for modifying the actual data, namely the "right hand side" of the input.
"modify" is the product of a bunch of refactoring of Shiftr.
Note, Modify operates on the same in memory copy of the data, whereas Shift creates a new top level output Map to populate.
Modify comes in 3 "flavors" that control how it operates on the input data, both at a leaf level but also as it is walking the tree.
The idea is to pick the base flavor you want, and then tweak modify's behavior on a case by case basis by applying node specific overrides.
Additionally the "?" character can suppress the modify operation, to only operate if the key/index exists
Example, say we want to process a document that may or may not have an "address" subsection.
Spec :
{
"address?" : { // If address exists in the input JSON, then match otherwise skip
"~state" : "Texas" // means if "state" does not exist or is null, then make it be "Texas"
}
}
Everything on the "right hand side" of modify is actually a "function". In the example above the "right hand side" of "Texas" is actually the function "insert this literal value".
Beyond that, you can invoke "named" functions by using the "=" special character.
Example : Say the input has a list of scores, and we want the min and max values of the list.
input :
{
"scores" : [ 4, 2, 8, 7, 5 ]
}
spec :
{
// Pull individual data out of the scores array
"firstScore" : "=firstElement(@(1,scores))",
"lastScore" : "=lastElement(@(1,scores))",
// Assuming that the scores array is always size of 5
"scoreAtMidPoint" : "=elementAt(@(1,scores),2)"
}
output :
{
"scores" : [ 4, 2, 8, 7, 5 ],
"firstScore" : 4,
"lastScore" : 5,
"scoreAtMidPoint" : 8
}
Note all off the Type Conversion functions support
Use an Array on the "right hand side" to specify series of functions to run / try until one returns a non-Optional.absent() result.
Purpose, allows for looking up a value, but if it is not found, applying a default.
Example, back to the "min and maxScore" example from above,
spec :
{
// if the input document did not have a "scores" entry,
// or it was empty,
// or it did not contain any 'numbers'
// then fall back to null and zero
"maxScore" : [ "=max(@(1, scores))", null],
"minScore" : [ "=min(@(1, scores)), 0]"
}
Introduced a Function interface, marked as @Deprecated as a warning
Changed baseSpec#apply(...) signature to supply availability of input
The new Modify Transform in this release is sufficient for the Bazaarvoice internal project that needs it. Usecases beyond that are not fully thought out / tested.
In general, it still feels like it needs some work / polish before we consider it done, but the ability to do Type conversions and String concatenation are compelling and often requested features.
Things to do to finish the "beta"
After beta
Even further out
Primary driver is users wanting to be able to use Jolt Shiftr to process Json-LD flavored documents.
See this test for examples. https://github.com/bazaarvoice/jolt/blob/c8e6ce62a2ea7551333d666d16ddba9f11feb7c4/jolt-core/src/test/resources/json/shiftr/escapeAllTheThings.json
Note that, because Java, the "backslash" as to be doubled.