Snow Owl Versions Save

Bugs/Improvements

• [index] ensure that removed properties properly trigger index schema incompatibility error when detected during boot (45f622d, 28b058f)

Bugs/Improvements

• [core] authorize resource commit entries instead of letting anyone access them (#1088)
• [validation] fix potential NPE when extending a validation issue for a concept without any active descriptions (#1080)
• [api] support effectiveTime range filtering using <effectiveTimeStart>...<effectiveTimeEnd> range format (#1074)
• [fhir] support withHistorySupplements query parameter when expanding a Value Set (#1076)
• [fhir] properly fetch point-in-time version of the resource in datasets where information is not stored on the version entry (#1086, #1089)
• [fhir] add missing date property to versioned resource responses (#1085)
• [fhir] prevent reusing the same effectiveTime value for different versions when creating resources via FHIR POST endpoints (#1087)
• [log] improve log output of failed remote jobs (#1083)
• [ci] replace LGTM analysis with GitHub CodeQL Action flow (e94de72, cf3ac45, bde939b)

Core

• Support versioned resource access via dedicated GET endpoints (#1073)
  • GET /bundles/:id/:version
  • GET /codesystems/:id/:version
  • Content requests will now fetch and use the correct point-in-time version of the corresponding resource (HEAD, timestamped or versioned)

Bugs/Improvements

• [index] support disabling indexing on Map/Object type fields (#1071)
• [fhir] improve computation of CodeSystem.content property (#1069)
• [fhir] versioned FHIR resources should get their proper versioned state
• [snomed] skip empty lines when importing an RF2 archive (#1065)
• [snomed] improve validation error message when an RF2 column has an unexpected SNOMED CT component ID (#1066)

Bugs/Improvements

• [core] allow deleting a component if its container resource or any of its container bundles can be edited (#1062)
• [api] support pluggable converters on CodeSystems API (83731e8)
• [api] support active flag on generic concept model (f77a5c0)
• [fhir] add generic "constraint" filter (evaluated the same way as SNOMED CT's "expression" filter) (#1063)

Core

• Support negated filter values in owner and status filters in Resource API (006e44b, d9a0bb1)
• Support snomedFirst sort keyword in Resource API to display SNOMED CT resources before any other (#1061)

SNOMED CT

• Support ECL specification 2.1 (#1060)
• Support SNOMED CT Simple map to concept reference set types (#1043)
  • Complete read/write/export/import support for the new SNOMED CT Simple map to concept reference set

FHIR

• Support importing FHIR resources from FHIR JSON format (#1058)

Packaging

• Use Ubuntu 22.04 as base image instead of obsolete CentOS (#1052)

Bugs/Improvements

• [core] ecl labeling should handle Code System URI values (1103222)
• [ecl] throw HTTP 400 instead of HTTP 501 when encountering a not implemented ECL feature (6e2b411)
• [snomed] support knn vector search for SNOMED CT description documents (6cbe08f, 05fa3e8, f3ed8f7)
• [snomed] prevent NPE during computation of concept expand cache in SNOMED CT Concept API (cb2befe)
• [snomed] improve performance of query reference set evaluation (3de012a, 880e7e8)
• [snomed] improve performance of ECL query optimizer (670d8f0)
• [snomed] ensure that ECL query optimizer generates ECL expressions with FSN labels (d11fd38)
• [api] ensure that bulk suggest API responds even if one of the bulk item requests fails (f199e7c)

Bugs/Improvements

• [api] fetch hit count only in resource update requests (9165d40)
• [api] add OID filter to GET /resources endpoint (f32391a)

Bugs/Improvements

• [index] properly catch and report bulk indexing failures during index commit (e93433c)
• [api] support field selection in SNOMED CT Concept expansions (661c774)
• [api] add update and delete resource REST API endpoints (b79a13c)
• [api] reduce execution time of bulk suggest requests by performing a single authorization check at the beginning (f507964)
• [api] support other resource URIs not just Code Systems in suggest API (64c5559)
• [docs] update rapidoc to latest 9.3.3 version (2dce6a6)
• [docs] minor changes to the Suggest API OpenAPI definition (f550166)
• [log] include auth/authz related execution time in request response metrics (1654004)

Core

• Introducing the new Code System Concept Suggest API (#1049, 5aad00c)
  • Based on lexical and/or semantic information concepts can be retrieved for a given set of like and unlike texts or queries
  • Supports pluggable suggester implementations, currently available open-source suggesters are:
    • term suggester - based on the most frequent tokens in the given like/unlike corpus the implementation suggests concepts based on their terms using default Elasticsearch full-text queries (synonym support, ignore stopwords, fuzziness, min occurrence count configuration settings are available)
    • mlt suggester - based on the given most frequent tokens in the like/unlike corpus the implementation suggests concepts based on their terms using Elasticsearch More Like This query (MLT specific configuration values are available, works best for large number of like texts)
  • Add batchSize and batchSizeTimeout configuration parameters to POST /suggest/bulk API (05ae0ec)
• Support filtering resources by their settings key-value pairs (#1048, #1050)
• Support Elasticsearch 8 clusters and features via a dedicated high-level client (#1047)

Bugs/Improvements

• [docker] improve docker example (#1044)
• [index] add support for more-like-this Elasticsearch queries (#1049)
• [index] add support for knn Elasticsearch queries (#1049)
• [index] fix decimal value conversion issue (#1039)
• [index] allow configuration of number_of_shards and number_of_replicas in custom index specific configurations
• [index] ensure that local settings get updated even if there are no remote ES settings to update (5a8d165)
• [index] eliminate deprecation warning messages due to deprecated ignore_throttled query parameter in newer Elasticsearch versions (#1053)
• [index] increase default bulkActionSizeInMb setting value to 50 (da69def)
• [index] maintain order when merging term filters into a single filter array to ensure that the query is properly cached when requested (df0c594)
• [core] load only the default field selection in updatedAtCommit() expansion (baaf59a)
• [fhir] fix FHIR date parsing and formatting issue when timezone offset is specified (8e946e8)
• [api] support active and version parameters in content() expand in Resources API (0068a66)
• [core] allow querying draft resources with the special LATEST path using their current working branch (aka HEAD) (fa5898d)
• [core] add support for ECL query definition inside ResourceURI instances with the ?ecl= query part (1ded832, 79d112a)

Dependencies/Packaging

• Upgrade Jackson libraries to 2.12.7
• Upgrade Apache HTTP Core to 4.4.12
• Upgrade Apache HTTP Client to 4.5.10

Core

• Introduce configurable authorization service (#1020, #1026)
  • External authorization systems can now be configured to provide list of accessible resource IDs and/or check whether a certain permission is present for a user or not
  • The default implementation performs the same evaluation as before based on either the incoming JWT's permissions claim or using information provided by the underlying identity provider
• Introduce a new GET /toolings endpoint (#1023)
  • Lists all supported tooling (aka schema) values from the current deployment
• Support querying resources by their url value (#1027)
• Support settings metadata on Bundle resource types (#1034)

FHIR

• Accept wider range of FHIR inputs based on official R4 and third-party CodeSystem sources (#1017)
  • Support SUPPLEMENT content mode
  • Support code extension type
  • Make div validator more lenient and accept opening tag without attributes
  • Support all FHIR-approved date formats in dateTime properties
  • Support FHIR contact property (1247e96)

Bugs/Improvements

• [index] fix incorrectly reduced "must" and "filter" clauses causing invalid search results in certain queries (#1015)
• [index] restrict the number of prefix values and prefix clauses that can be present in a request (eb6df3b)
• [transport] enable eventbus call stack tracing (#1021)
• [core] fix serialization of User object (#1022)
• [core] prevent creating resources with the reserved -1 identifier (#1036)
• [api] support X-Author header in Versioning API (#1018)
• [api] support X-Author header in RF2 Import API (#1031)
• [api] support X-Author header in FHIR CodeSystem API (#1031)
• [api] support filtering resources by owner field in Resources API (#1035)
• [api] respond with HTTP 401 properly when trying to authenticate with a Snow Owl without any identity provider configured (7f16e9f)
• [fhir] fix potential NPE during lookup property validation (#1019)
• [cli] allow commands to be executed when no JWT signing is configured (b0e81e8)
• [classification] add missing resourcePath parameter to classification jobs (#1025)
• [snomed] fix incorrect RF2 export component type when exporting refsets (#1028)

Dependencies/Packaging

• Upgrade snomed-ecl to 2.0.4 (76c464b)
• Upgrade bundled JDK to 17.0.3 (#1016)

Core

• Secure connections are now by default enabled when using TCP connections to connect to the terminology server (#993, #1012)
  • New configuration values are available to configure SSL certificate (or let the server use a self-signed/generated certificate by default)
• Support Elasticsearch 8 via API compatibility mode (#1008)
  • Support running Elasticsearch in testcontainers
• Support externally customizable index type mappings via repository.index.<indexName>.mappings configuration key (#1010)
• Move ECL labeler API to core (#1009)

SNOMED CT

• Add batchSize configuration parameter to SNOMED CT RF2 import API (4109687)

Bugs/Improvements

• [index] backoff a bit when sending too much bulk requests to the connected Elasticsearch cluster on rejection errors, eg. high queue size (be85e38)
• [core] make toolingId/branch reflective access support a timestamp suffix (6df066f)
• [core] check bundleId before executing resource type specific logic (like automatic branch generation) to prevent creation of the corresponding branch as a side effect of an erroneous code system create request (c191ba3)
• [core] fetch extensionOf branch state from the code system's repository, not the one available from the context. The latter only stores resource metadata and no branches outside MAIN exist in it. (0d581a3)
• [core] add a generic implementation of ECL to use in other Code Systems and toolings when required (c5daa92, 76e353d, e6dbf9e)
• [snomed] do not kick off a concept map search in the SNOMED CT tooling if source tooling ID filtering is enabled, and snomed is not within the allowed list of tooling IDs. (b30d38d)
• [snomed] honor index.maxTermsCount when sending large queries to Elasticsearch during RF2 import (4a46daa)
• [snomed] reduce load of existing data from external Elasticsearch clusters when importing RF2 releases (cca6f87)
• [snomed] various performance optimizations when sending SNOMED CT commit requests sent to Elasticsearch (f52991a, 62101d6, 26d4a45)
• [suggest] avoid NPE when suggestion base concept does not have alternative terms (0364e6a)

Dependencies

• Bump Elasticsearch to 7.17.2 (87b4a22)
• Bump Spring to 5.3.19 (bfd162b)
• Add SQL, JSR223 (script engine) and date utility modules to Groovy wrapper (062cade, 8eb9486)