Conquery Versions Save

What's Changed

• Hotfix: move sorting out of disabled searchables from the job to the filter by @thoniTUB in https://github.com/ingef/conquery/pull/3380
• Configurable interning by @awildturtok in https://github.com/ingef/conquery/pull/3384
• humonguous to string by @awildturtok in https://github.com/ingef/conquery/pull/3386
• shorten full message queue messages without trace by @awildturtok in https://github.com/ingef/conquery/pull/3388
• Adds progess on search finalizer job by @thoniTUB in https://github.com/ingef/conquery/pull/3385

Full Changelog: https://github.com/ingef/conquery/compare/v2.0.0-2024KW15...v2.0.0-2024KW15-HOTFIX-BIG-MESSAGES

What's Changed

• Removal of Dictionaries, towards string-Array based Store by @awildturtok in https://github.com/ingef/conquery/pull/3131
• fix not testing nested ReusedQueries, and fix not checking DateFormat names, in containsDates. by @awildturtok in https://github.com/ingef/conquery/pull/3326
• Fix date restriction and validity date conversion by @jnsrnhld in https://github.com/ingef/conquery/pull/3325
• open and close all tested stores by @thoniTUB in https://github.com/ingef/conquery/pull/3321
• Implement SecondaryId conversion by @jnsrnhld in https://github.com/ingef/conquery/pull/3317
• Reintegrate Master by @github-actions in https://github.com/ingef/conquery/pull/3328
• unroll span in MatchingStats.Entry#addEvent by @awildturtok in https://github.com/ingef/conquery/pull/3330
• drop subclasses of CDateRange as vtables are expensive by @awildturtok in https://github.com/ingef/conquery/pull/3331
• Cleanup BooleanStatsCollector and all other collectors for performance issues by @awildturtok in https://github.com/ingef/conquery/pull/3329
• Feature/short cut admin UI login by @thoniTUB in https://github.com/ingef/conquery/pull/3336
• Feature/clean up ftl by @thoniTUB in https://github.com/ingef/conquery/pull/3337
• Role matching OIDC by @awildturtok in https://github.com/ingef/conquery/pull/3335
• Rework SqlTables by @jnsrnhld in https://github.com/ingef/conquery/pull/3332
• properly count entities/lines by @awildturtok in https://github.com/ingef/conquery/pull/3334
• speedup DateColumnStatsCollector: by @awildturtok in https://github.com/ingef/conquery/pull/3340
• i deleted the wrong field: We need months not quarters by @awildturtok in https://github.com/ingef/conquery/pull/3341
• Use java.nio.file.Files::createDirectories to create the dumpfile parent by @SebChmie in https://github.com/ingef/conquery/pull/3342
• adds endpoint for form backend version info by @thoniTUB in https://github.com/ingef/conquery/pull/3286
• Fix z-index stacking for forms navigation by @Kadrian in https://github.com/ingef/conquery/pull/3350
• Remove overly complicated drag handle disabling by @Kadrian in https://github.com/ingef/conquery/pull/3352
• Fix accessing entity not inCBlock by @awildturtok in https://github.com/ingef/conquery/pull/3351
• allow Statistics for all SingleTableResults by @awildturtok in https://github.com/ingef/conquery/pull/3344
• Select input after selecting item from multiselect by @Kadrian in https://github.com/ingef/conquery/pull/3354
• Hotfix: restore history after new preview release by @Kadrian in https://github.com/ingef/conquery/pull/3353
• Move frontend Dockerfile to root; add git describe by @Kadrian in https://github.com/ingef/conquery/pull/3349
• set log message to trace by @thoniTUB in https://github.com/ingef/conquery/pull/3356
• Reintegrate Master by @github-actions in https://github.com/ingef/conquery/pull/3355
• drop SqlManagedQuery, and instead move SQL concerns into SqlExecution… by @awildturtok in https://github.com/ingef/conquery/pull/3347
• adds finalize message to support multiple ReactionMessage per worker by @thoniTUB in https://github.com/ingef/conquery/pull/3357
• drop limit for large MultiSelects, as the performance penalty is no longer an issue by @awildturtok in https://github.com/ingef/conquery/pull/3362
• adds more explicit logging for WorkerHandler by @awildturtok in https://github.com/ingef/conquery/pull/3364
• don't compute dates stats when no dates in query. by @awildturtok in https://github.com/ingef/conquery/pull/3346
• Fix: total search result for empty search by @thoniTUB in https://github.com/ingef/conquery/pull/3376
• fix casting of query result preview by @awildturtok in https://github.com/ingef/conquery/pull/3377
• Merge Release by @github-actions in https://github.com/ingef/conquery/pull/3366

Full Changelog: https://github.com/ingef/conquery/compare/v2.0.0-2024KW09...v2.0.0-2024KW15

What's Changed

• Improve log message during import by @thoniTUB in https://github.com/ingef/conquery/pull/3291
• Fix any char regex for SQL conversion by @jnsrnhld in https://github.com/ingef/conquery/pull/3297
• Cleanup and restructure sql conversion dir by @jnsrnhld in https://github.com/ingef/conquery/pull/3298
• rip out TreeChildPrefixIndex.java and all its dependants by @awildturtok in https://github.com/ingef/conquery/pull/3295
• use ngram in TrieSearch as it performs very similarly but at much low… by @awildturtok in https://github.com/ingef/conquery/pull/3216
• Make all connector conversion CTE steps mandatory by @jnsrnhld in https://github.com/ingef/conquery/pull/3300
• filter nulls before they reach the template by @thoniTUB in https://github.com/ingef/conquery/pull/3299
• Support multiple connector SQL conversion by @jnsrnhld in https://github.com/ingef/conquery/pull/3301
• Fix/form scan fail independently by @thoniTUB in https://github.com/ingef/conquery/pull/3304
• fix dupe insertion for small words by @awildturtok in https://github.com/ingef/conquery/pull/3312
• Localize Date info for EntityPreviewExecutionStatus by @awildturtok in https://github.com/ingef/conquery/pull/3307
• Fix resolving conditions of all parents of concept tree children by @jnsrnhld in https://github.com/ingef/conquery/pull/3309
• Update authentication.md by @thoniTUB in https://github.com/ingef/conquery/pull/3293
• Simplify SqlTables by @jnsrnhld in https://github.com/ingef/conquery/pull/3316
• fix nested query ordering by @awildturtok in https://github.com/ingef/conquery/pull/3303
• fixes dropping sharedDictionaries when the same one is referenced mul… by @awildturtok in https://github.com/ingef/conquery/pull/3318
• use password4j to generate and check hashes by @thoniTUB in https://github.com/ingef/conquery/pull/3313
• adds switch for output type in script adminend by @thoniTUB in https://github.com/ingef/conquery/pull/3322
• remove apitoken related classes and code by @thoniTUB in https://github.com/ingef/conquery/pull/3310
• Query stats followup by @awildturtok in https://github.com/ingef/conquery/pull/3255
• Feature/preview v2 by @awildturtok in https://github.com/ingef/conquery/pull/3327
• Merge Release by @github-actions in https://github.com/ingef/conquery/pull/3324

Full Changelog: https://github.com/ingef/conquery/compare/v2.0.0-2024KW06...v2.0.0-2024KW09

Migrations

This Release deprecates the ApiToken-Realm (https://github.com/ingef/conquery/pull/3310):

• If you have the a Realm configured with the type API_TOKEN remove it
• remove the api_token store in your storage folder
• If you have an initial User defined e.g. authorizationRealms.initialUsers.[0].credentials modify it to authorizationRealms.initialUsers.[0].credential and ensure that this member is a single object not a list

What's Changed

• Reintegrate Master by @github-actions in https://github.com/ingef/conquery/pull/3239
• Implements endpoint to fetch query result statistics by @awildturtok in https://github.com/ingef/conquery/pull/3220
• Fix/OIDC client setup by @thoniTUB in https://github.com/ingef/conquery/pull/3236
• Improve multi select load more input by @Kadrian in https://github.com/ingef/conquery/pull/3248
• Fix resize when tooltip is hidden by @Kadrian in https://github.com/ingef/conquery/pull/3244
• Take selectConfig into account when initializing concepts in form dropzones by @Kadrian in https://github.com/ingef/conquery/pull/3245
• Search owner name and highlight when found by @Kadrian in https://github.com/ingef/conquery/pull/3247
• Fix open / close of folders using the new resizable panels by @Kadrian in https://github.com/ingef/conquery/pull/3246
• Refactor SqlSelects and Filters into SqlAggregators by @jnsrnhld in https://github.com/ingef/conquery/pull/3241
• Move select and filter SQL conversion logic to nodes itself by @jnsrnhld in https://github.com/ingef/conquery/pull/3242
• hold reference to results until metrics are collected by @thoniTUB in https://github.com/ingef/conquery/pull/3249
• Add SQL conversion of conditions by @jnsrnhld in https://github.com/ingef/conquery/pull/3252
• Implement diffsum/distinct SumSelect/SumFilter conversion by @jnsrnhld in https://github.com/ingef/conquery/pull/3251
• fixes accessing externalId after wrapping as object by @awildturtok in https://github.com/ingef/conquery/pull/3261
• Rework test infrastructure to easily switch between Worker and SQL mode by @jnsrnhld in https://github.com/ingef/conquery/pull/3266
• Bump org.apache.shiro:shiro-core from 1.10.1 to 1.13.0 in /backend by @dependabot in https://github.com/ingef/conquery/pull/3269
• Cleanup Jooq usage by @jnsrnhld in https://github.com/ingef/conquery/pull/3270
• Fix dynamic input group buttons by @Kadrian in https://github.com/ingef/conquery/pull/3273
• Adjust invalid concepts logic with restricted selects by @Kadrian in https://github.com/ingef/conquery/pull/3274
• undo gzipping WorkerMessages as it was causing ImportJob to slow down significantly by @awildturtok in https://github.com/ingef/conquery/pull/3272
• Implement FLAG select and filter SQL conversion by @jnsrnhld in https://github.com/ingef/conquery/pull/3264
• Fix nested AND/OR node SQL conversion by @jnsrnhld in https://github.com/ingef/conquery/pull/3262
• Add COUNT_QUARTERS SQL conversion for DATE columns by @jnsrnhld in https://github.com/ingef/conquery/pull/3263
• Bump fastutil from 8.5.9 to 8.5.12 by @dependabot in https://github.com/ingef/conquery/pull/2970
• Implement Select filter value auto-completion for SQL mode by @jnsrnhld in https://github.com/ingef/conquery/pull/3277
• Implement saved queries for SQL connector by @jnsrnhld in https://github.com/ingef/conquery/pull/3275
• check if the Execution has been deleted before resetting by @awildturtok in https://github.com/ingef/conquery/pull/3282
• Combine shared dict merging by @awildturtok in https://github.com/ingef/conquery/pull/3232
• don't use cachedThreadPool by @awildturtok in https://github.com/ingef/conquery/pull/3283
• Prepare SQL conversion of multiple connectors by @jnsrnhld in https://github.com/ingef/conquery/pull/3279
• use ThreadPool that doesn't reject (sorry!) by @awildturtok in https://github.com/ingef/conquery/pull/3285
• Draft: hoping to avoid keeping SecondaryIdQuery subPlans, that are not needed by @awildturtok in https://github.com/ingef/conquery/pull/3260
• Merge Release by @github-actions in https://github.com/ingef/conquery/pull/3290

Full Changelog: https://github.com/ingef/conquery/compare/v2.0.0-2023KW48-HOTFIX-CONCURENT-SHRINK...v2.0.0-2024KW06

What's Changed

• shrink in single thread by @awildturtok in https://github.com/ingef/conquery/pull/3238

Full Changelog: https://github.com/ingef/conquery/compare/v2.0.0-2023KW48...v2.0.0-2023KW48-HOTFIX-CONCURENT-SHRINK

What's Changed

• Fix getPredecessor for concept and data aggregation tables by @jnsrnhld in https://github.com/ingef/conquery/pull/3199
• Move from yarn to npm, update non-major & tooling dependencies by @Kadrian in https://github.com/ingef/conquery/pull/3201
• Fix missing MockDateNowSupplier in Sql integration tests by @jnsrnhld in https://github.com/ingef/conquery/pull/3207
• Allow inserting unknown filter values by @Kadrian in https://github.com/ingef/conquery/pull/3209
• Fix/e2e backend refresh to early by @thoniTUB in https://github.com/ingef/conquery/pull/3208
• Upgrade cypress by @Kadrian in https://github.com/ingef/conquery/pull/3205
• Renaming of CteStep classes by @jnsrnhld in https://github.com/ingef/conquery/pull/3214
• throws RTE on emtpy test sets for programmatic test by @thoniTUB in https://github.com/ingef/conquery/pull/3219
• Rework SQL backend tests by @jnsrnhld in https://github.com/ingef/conquery/pull/3215
• wraps realm in abstract binder by @thoniTUB in https://github.com/ingef/conquery/pull/3218
• Merge master by @Kadrian in https://github.com/ingef/conquery/pull/3223
• Reintegrate Master by @github-actions in https://github.com/ingef/conquery/pull/3203
• Allow unresolved filter values on expand by @Kadrian in https://github.com/ingef/conquery/pull/3224
• Cast SQL results to matching ResultTypes by @jnsrnhld in https://github.com/ingef/conquery/pull/3217
• makes IndexService a singleton, shared between namespaces by @thoniTUB in https://github.com/ingef/conquery/pull/3226
• Use builder for Selects by @jnsrnhld in https://github.com/ingef/conquery/pull/3221
• Add name generator for database identifiers by @jnsrnhld in https://github.com/ingef/conquery/pull/3222
• parse all signing keys and compare kids with token by @thoniTUB in https://github.com/ingef/conquery/pull/3228
• Reintegrate master by @awildturtok in https://github.com/ingef/conquery/pull/3233
• Reintegrate Master by @github-actions in https://github.com/ingef/conquery/pull/3231
• fixes key filtering and adds test by @thoniTUB in https://github.com/ingef/conquery/pull/3235
• use IntSet, as big arrays are mega-slow by @awildturtok in https://github.com/ingef/conquery/pull/3230
• Merge Release by @github-actions in https://github.com/ingef/conquery/pull/3237

Full Changelog: https://github.com/ingef/conquery/compare/v2.0.0-2023KW45-HOTFIX-DICT-IMPORT...v2.0.0-2023KW48

What's Changed

• BUGFIX: keep Dictionaries locked while storing/sending them in ImportJob by @awildturtok in https://github.com/ingef/conquery/pull/3229

Full Changelog: https://github.com/ingef/conquery/compare/v2.0.0-2023KW45...v2.0.0-2023KW45-HOTFIX-DICT-IMPORT

What's Changed

• Feature/contains dates at submissions by @awildturtok in https://github.com/ingef/conquery/pull/3169
• Increase clickable area of Calendar Button and center it by @Kadrian in https://github.com/ingef/conquery/pull/3166
• fix: datepicker blocks input field by @MarcoKorinth in https://github.com/ingef/conquery/pull/3164
• Reduce ChunkWriter#bufferSize to 2MB from 32MB by @awildturtok in https://github.com/ingef/conquery/pull/3172
• Share nested queries by @awildturtok in https://github.com/ingef/conquery/pull/3168
• Make all non-required field errors red by @Kadrian in https://github.com/ingef/conquery/pull/3167
• Add group select and group filter by @jnsrnhld in https://github.com/ingef/conquery/pull/3159
• sets the defaultValue of containsDates to true, so that old queries a… by @awildturtok in https://github.com/ingef/conquery/pull/3175
• Add no dates in results indicator by @Kadrian in https://github.com/ingef/conquery/pull/3173
• adds permissions and checks for QUERY_PREVIEW and ENTITY_PREVIEW by @awildturtok in https://github.com/ingef/conquery/pull/3177
• Add permission flags for history and preview by @Kadrian in https://github.com/ingef/conquery/pull/3179
• Iterate upload filters / concepts dropdown by @Kadrian in https://github.com/ingef/conquery/pull/3180
• Remember dropped filename for imported concepts / filter values by @Kadrian in https://github.com/ingef/conquery/pull/3181
• feat: renew datepicker year selection by @MarcoKorinth in https://github.com/ingef/conquery/pull/3178
• throw proper error message, when query cannot be resolved by @awildturtok in https://github.com/ingef/conquery/pull/3182
• use isExcludeFromTimeAggregation instead by @awildturtok in https://github.com/ingef/conquery/pull/3184
• apply all Dictionary operations in ImportJob with Stream.parallel by @awildturtok in https://github.com/ingef/conquery/pull/3185
• Improve preview: add sum and currency formatting; improve layout by @Kadrian in https://github.com/ingef/conquery/pull/3186
• Keep row prefix constant when reordering by @fabian-bizfactory in https://github.com/ingef/conquery/pull/3176
• adds healthcheck for connected shardNodes by @awildturtok in https://github.com/ingef/conquery/pull/3189
• send MatchingStats results per concept to reduce the size of messages by @awildturtok in https://github.com/ingef/conquery/pull/3190
• fixes not prettyPrinting for infos by @awildturtok in https://github.com/ingef/conquery/pull/3191
• move caching of selectedValues into QueryExecutionContext by @awildturtok in https://github.com/ingef/conquery/pull/3192
• fixes usage of PrintSettings: With didn't properly work with the inte… by @awildturtok in https://github.com/ingef/conquery/pull/3193
• Add interval packing by @jnsrnhld in https://github.com/ingef/conquery/pull/3183
• Add date aggregation by @jnsrnhld in https://github.com/ingef/conquery/pull/3188
• Improve file drop resolve; fix race condition with insert all by @Kadrian in https://github.com/ingef/conquery/pull/3200
• Copy concept prefix when copying concept in the same field by @fabian-bizfactory in https://github.com/ingef/conquery/pull/3187
• Add last and random select converters by @jnsrnhld in https://github.com/ingef/conquery/pull/3194
• Add count filter and select converter by @jnsrnhld in https://github.com/ingef/conquery/pull/3196
• Stop showing whether a field is optional in all form fields by @Kadrian in https://github.com/ingef/conquery/pull/3174

Full Changelog: https://github.com/ingef/conquery/compare/v2.0.0-2023KW36...v2.0.0-2023KW45

What's Changed

• Send WorkerMessage in ForwardToWorker gzipped by @awildturtok in https://github.com/ingef/conquery/pull/3149
• Rework adminui jobs by @MarcoKorinth in https://github.com/ingef/conquery/pull/3130
• Add support for multi column validity dates by @jnsrnhld in https://github.com/ingef/conquery/pull/3129
• Gzipped storage by @awildturtok in https://github.com/ingef/conquery/pull/3148
• Startup from the firehose by @awildturtok in https://github.com/ingef/conquery/pull/3153
• test if the query result contains dates by @awildturtok in https://github.com/ingef/conquery/pull/3155
• Add HANA support (#109) by @jnsrnhld in https://github.com/ingef/conquery/pull/3154
• Improve select validation in concept list fields by @Kadrian in https://github.com/ingef/conquery/pull/3156
• Simplify css and DropzoneBetweenElements by @Kadrian in https://github.com/ingef/conquery/pull/3158
• allow insertion of concepts between each other by @fabian-bizfactory in https://github.com/ingef/conquery/pull/3076
• Fix Highlighting in Tooltip by @fabian-bizfactory in https://github.com/ingef/conquery/pull/3152
• adds config option to always allow custom input in SELECT-style Filters by @awildturtok in https://github.com/ingef/conquery/pull/3163

Full Changelog: https://github.com/ingef/conquery/compare/v2.0.0-2023KW33...v2.0.0-2023KW26

This release contains breaking changes in storage layer: Xodus is now written in gzip format. If you wish to retain your old data use the following migration script to migrate all stores (shard and manager):

$ cat migrate-identity.groovy                                                                                                                                     

import com.fasterxml.jackson.databind.JsonNode

return {
    String env, String store, JsonNode key, JsonNode value ->
        return new Tuple(key, value)
}

$ java -Xmx100G -jar ../eva-executable.jar migrate --in-gzip false --out-gzip true --script ./migrate-identity.groovy --in ./storage/ --out ./gzipped

It is far easier to just reimport. After this release, the storage should take about 20% of memory on disk, while loading in 30% of the time.

What's Changed

• Decouple distributed execution by @torbsto in https://github.com/ingef/conquery/pull/3059
• Basic support for SQL connector by @torbsto in https://github.com/ingef/conquery/pull/3063
• Iterate editor v2 minor issues by @Kadrian in https://github.com/ingef/conquery/pull/3122
• Remove query dates validation by @Kadrian in https://github.com/ingef/conquery/pull/3123
• fixes synchronization of jobManagerStatus by @awildturtok in https://github.com/ingef/conquery/pull/3124
• Add SQL OR conversion by @jnsrnhld in https://github.com/ingef/conquery/pull/3126
• Add SQL NOT conversion by @jnsrnhld in https://github.com/ingef/conquery/pull/3128
• Server side error message rendering by @awildturtok in https://github.com/ingef/conquery/pull/3133
• implements cancelQuery for ExternalFormBackendApi by @awildturtok in https://github.com/ingef/conquery/pull/3134
• Remove error context handling by @Kadrian in https://github.com/ingef/conquery/pull/3135
• Iterate concept chart by @Kadrian in https://github.com/ingef/conquery/pull/3136
• Fix how we handle dates and sources columns in history by @Kadrian in https://github.com/ingef/conquery/pull/3140
• Support bigger and smaller right columns by @Kadrian in https://github.com/ingef/conquery/pull/3142

Full Changelog: https://github.com/ingef/conquery/compare/v2.0.0-2023KW27-HOTFIX-JOBQUEUE-SYNCH...v2.0.0-2023KW33