Opensearch Project Sql Versions Save

Query your data using familiar SQL or intuitive Piped Processing Language (PPL)

2.12.0.0

2 months ago

2.9.0.0

9 months ago

Compatible with OpenSearch and OpenSearch Dashboards Version 2.9.0

Features

  • Enable Table Function and PromQL function (#1719)
  • Add Spark connector (#1780)

Enhancements

  • Pagination: Support WHERE clause, column list in SELECT clause and for functions and expressions in the query (#1500)
  • Pagination: Support ORDER BY clauses and queries without FROM clause (#1599)
  • Remove backticks on by field in stats (#1728)
  • Support Array and ExprValue Parsing With Inner Hits (#1737)
  • Add Support for Nested Function in Order By Clause (#1789)
  • Add Support for Field Star in Nested Function (#1773)
  • Guarantee datasource read api is strong consistent read (compatibility with segment replication) (#1815)
  • Added new datetime functions and aliases to PPL (#1807)
  • Support user-defined and incomplete date formats (#1821)
  • Add _routing to SQL includes list (#1771)
  • Disable read of plugins.query.datasources.encryption.masterkey from cluster settings GET API (#1825)
  • Add EMR client to spark connector (#1790)
  • Improved error codes in case of data source API security exception (#1753)
  • Remove Default master encryption key from settings (#1851)

Bug Fixes

  • Fixed bug of byte/short not handling 0 denominator in divide/modulus equations (#1716)
  • Fix CSV/RAW output header being application/json rather than plain/text (#1779)

Documentation

  • Updated documentation of round function return type (#1725)
  • Updated protocol.rst with new wording for error message (#1662)
  • Updated documentation for temporal data types (#1826)

Infrastructure

  • stopPrometheus task in doctest build.gradle now runs upon project failure in startOpenSearch (#1747)
  • Bump guava to 32.0.1 (#1829)
  • Disable CrossClusterSearchIT test (#1814)
  • Fix flakytest when tests.locale=tr (#1827)

Refactoring

  • Simplify OpenSearchIndexScanBuilder (#1738)

2.8.0.0

11 months ago

Compatible with OpenSearch and OpenSearch Dashboards Version 2.8.0

Release date: June 6th, 2023

  • Support for pagination in v2 engine of SELECT FROM table queries (#1666)
  • Support Alternate Datetime Formats (#1664)
  • Create new anonymizer for new engine (#1665)
  • Add Support for Nested Function Use In WHERE Clause Predicate Expresion (#1657)
  • Cross cluster search in PPL (#1512)
  • Added COSH to V2 engine (#1428)
  • REST API for GET,PUT and DELETE (#1482)

Enhancements

  • Minor clean up of datetime and other classes (#1310)
  • Add integration JDBC tests for cursor/fetch_size feature (#1315)
  • Refactoring datasource changes to a new module. (#1504)

Bug Fixes

  • Fixing bug where Nested functions used in WHERE, GROUP BY, HAVING, and ORDER BY clauses don't fallback to legacy engine. (#1549)

Documentation

  • Add Nested Documentation for 2.7 Related Features (#1620)
  • Update usage example doc for PPL cross-cluster search (#1610)
  • Documentation and other papercuts for datasource api launch (#1530)

Maintenance

  • Fix IT - address breaking changes from upstream. (#1659)
  • Increment version to 2.8.0-SNAPSHOT (#1552)
  • Backport maintainer list update to 2.x. (#1650)
  • Backport jackson and gradle update from #1580 to 2.x (#1596)
  • adding reflections as a dependency (#1559)
  • Bump org.json dependency version (#1586)
  • Integ Test Fix (#1541)

2.7.0.0

1 year ago

What's Changed

New Contributors

Full Changelog: https://github.com/opensearch-project/sql/compare/2.6.0.0...2.7.0.0

(None)

1 year ago

Compatible with OpenSearch and OpenSearch Dashboards Version 2.6.0

  • [Backport 2.x] Remove in development filesystem support (#1284) (#1286)
  • [Backport 2.x] Add micro benchmark by JMH (#1291)
  • [AUTO] Increment version to 2.6.0-SNAPSHOT (#1248)
  • [Backport 2.x] Added Correctness Tests For Date And Time Functions (#1313)
  • [Backport 2.x] Fix ArgumentCaptor can't capture varargs (#1320) (#1322)
  • [Backport 2.x] Create custom integ test file for sql plugin. (#1332)
  • [Backport 2.x] Add GET_FORMAT Function To OpenSearch SQL Plugin (#1342)
  • [Backport 2.x] Support More Formats For GET_FORMAT Function (#1349)
  • [Backport 2.x] Add WeekOfYear Function To OpenSearch (#1351)
  • [Backport 2.x] Add last_day Function To OpenSearch SQL Plugin (#1350)
  • [Backport 2.x] Add release notes for 2.6.0 (#1369)
  • Integ Test Refactoring (#1383)
  • [Backport 2.x] Exclude OpenSearch system index when IT cleanup (#1381)

Enhancements

  • [Backport 2.x] Extend comparison methods to accept different datetime types. (#129) (#1294)
  • [Backport 2.x] Enable concat() string function to support multiple string arguments (#1297)
  • [Backport 2.x] Add more keywords as identifier in PPL (#1323)
  • [Backport 2.x] Update DATE_ADD/ADDDATE and DATE_SUB/SUBDATE functions. (#122) (#1325)
  • [Backport 2.x] Update TIMESTAMP function implementation and signatures. (#1340)
  • [Backport 2.x] Add TIME_FORMAT() Function To SQL Plugin (#1346)
  • [Backport 2.x] Updated EXPM1() and Tests to New Engine (#1348)

Bug Fixes

  • [Backport 2.x] Allow literal in aggregation (#203) (#1290)
  • [Backport 2.x] Datetime aggregation fixes. (#1312)
  • [Backport 2.x] Fix index not found reported as server error bug (#1355)
  • [Backport 2.x] Modified returning NaN to NULL (#1347)

Documentation

  • [Backport 2.x] Remove obsolete links from README. (#1307)

Maintenance

  • Refactor AWSSigV4 auth to support different AWSCredentialProviders (#1389)

2.5.0.0

1 year ago

Compatible with OpenSearch and OpenSearch Dashboards Version 2.5.0

Refactoring

Enhancements

  • Add low-level create table and table exists API (#834)
  • Add time window and window assigner (#950)
  • Add valueOf() to Expression (#1055)
  • Add Statement, QueryExecution and QueryManager (#845)
  • Add Streaming Source Impl (#994)
  • Add watermark generator (#959)
  • Add stream context and window trigger (#958)
  • Add micro batch streaming execution (#1044)
  • Add Streaming Plan Impl (#1068)
  • Add CBRT to the V2 engine (#1081)
  • Add CBRT function to the PPL (#1097)
  • Add timeout option to SQL CLI tool. (#1076)
  • Add Day_Of_Year Function To OpenSearch (#1128)
  • Add Week_Of_Year Function To OpenSearch (#1127)
  • Add Month_Of_Year Function To OpenSearch (#1129)
  • Add Minute_Of_Day Function To SQL Plugin (#1207 #1214)
  • Add Second_Of_Minute Function As An Alias Of The Second Function (#1231 #1237)
  • Add Support For Legacy Syntax For Match Function In New Engine (#1090)
  • Add MatchPhraseQuery As Alternate Syntax for Match_Phrase Function (#1103)
  • Use query execution start time as the value of now-like functions. (#1047)
  • Add Support for Alternate Legacy MULTIMATCH syntax (#1102)
  • Add position() function to V2 engine (#1121)
  • Add position() string function to PPL (#1147)
  • Add support for wildcard_query function to the new engine (#156 #1108)
  • Add reverse() string function to V2 SQL Engine(#1154)
  • Add table write operator and builder (#1094)
  • Add BETWEEN expression in v2 engine (#1163)
  • Adding UTC_DATE, UTC_TIME, UTC_TIMESTAMP (#1193 #1198)
  • Validate field and fields parameters in relevance search functions (#1067 #1199)
  • Add TIMEDIFF and DATEDIFF functions. (#131 #1195 #1234)
  • Add functions ADDTIME and SUBTIME. (#132 #1194 #1252)
  • Add Day_Of_Week Function As An Alias Of DayOfWeek (#190 #1228 #1239)
  • Add Minute_Of_Hour Function As An Alias Of Minute Function (#1253)
  • Add support for long value return for CEIL, CEILING and FLOOR math functions (#1205)
  • Add Alternate Syntax For Match_Query And Other Functions (#1166)
  • Add Support For TIME Type in DAY_OF_YEAR Functions (#199 #1223 1#258)
  • Add Day_Of_Month Function As An Alias Of DayOfMonth (#1227 #1265)
  • Add security patch for CVE-2020-15250 (#1095)
  • Add security patch for CVE-2022-45868 (#1107)
  • Add Hour_Of_Day Function As An Alias Of Hour (#1226 #1270)

Bug Fixes

  • Fix FLOAT -> DOUBLE cast. (#1025)
  • Fix error messaging from prometheus. (#1029 #1037)
  • Add query function as alternate syntax to query_string function (#1010)
  • Deprecate span collector (#990)
  • back quote fix (#1041 #1050)
  • Update DATE and TIME functions to parse string input as datetime (#991)
  • Integ tests fix for arm64 (#1069)
  • Fix history file usage in SQL CLI tool. (#1077)
  • Update Jackson to 2.14.1 and fix dependency resolution issues (#1150)
  • Change LIKE operator case-insensitive match (#1160)
  • Fix arithmetic operator precedence (#1172 #1188)
  • Fix back quoted alias of FROM subquery (#1189 #1208)
  • Fix truncate function (#1197 #1213)
  • Allow common keywords and scalar function name used as identifier (#1191 #1212)
  • Suppress report uploading failure in CI. (#1180 #1220)
  • Fixed error with single timestamp query (#1244 #1246 #1249)
  • Support JOIN query on object field with unexpanded name (#1229 #1250)
  • Bug fix for less than and greater than operators on @timestamp (#1267 #1272)

Infrastructure

  • Change ApplicationContext lifecycle to per node level (#822)
  • Catalog to Datasource changes (#1027 #1049 #1086)
  • Bump jackson to 2.14.0 (#1058)
  • Add metadatalog interface and default in memory implementation (#974)
  • Decouple function repository and DSL from IoC container for use anywhere (#1085)
  • Move DataSourceServiceImpl to core module (#1084)
  • Improve pushdown optimization and logical to physical transformation (#1091)
  • Using jackson and jackson_databind version defined in OpenSearch (#1169 #1173)
  • Add BWC tests for running against distribution bundle. (#1209)

Documentation

  • Support opensearch-sql:run and update developer_guide doc (#1099)
  • Updated MAINTAINERS.md to match recommended opensearch-project format. (#1224 #1233)

2.4.0.0

1 year ago

Compatible with OpenSearch and OpenSearch Dashboards Version 2.4.0

Features

Data Source Management

  • Catalog Implementation (#819)
  • Catalog to Datasource changes (#1027)

Prometheus Support

  • Prometheus Connector Initial Code (#878)
  • Restricted catalog name to [a-zA-Z0-9_-] characters (#876)
  • Table function for supporting prometheus query_range function (#875)
  • List tables/metrics using information_schema in source command. (#914)
  • [Backport 2.4] Prometheus select metric and stats queries. (#1020)

Log Pattern Command

  • Add patterns and grok command (#813)

ML Command

  • Add category_field to AD command in PPL (#952)
  • A Generic ML Command in PPL (#971)

Enhancements

  • Add datetime functions FROM_UNIXTIME and UNIX_TIMESTAMP (#835)
  • Adding CONVERT_TZ and DATETIME functions to SQL and PPL (#848)
  • Add Support for Highlight Wildcard in SQL (#827)
  • Update SQL CLI to use AWS session token. (#918)
  • Add typeof function. (#867)
  • Show catalogs (#925)
  • Add functions PERIOD_ADD and PERIOD_DIFF. (#933)
  • Add take() aggregation function in PPL (#949)
  • Describe Table with catalog name. (#989)
  • Catalog Enhancements (#988)
  • Rework on error reporting to make it more verbose and human-friendly. (#839)

Bug Fixes

  • Fix EqualsAndHashCode Annotation Warning Messages (#847)
  • Remove duplicated png file (#865)
  • Fix NPE with multiple queries containing DOT(.) in index name. (#870)
  • Update JDBC driver version (#941)
  • Fix result order of parse with other run time fields (#934)
  • AD timefield name issue (#919)
  • [Backport 2.4] Add function name as identifier in antlr (#1018)
  • [Backport 2.4] Fix incorrect results returned by min, max and avg (#1022)

Infrastructure

  • Fix failing ODBC workflow (#828)
  • Reorganize GitHub workflows. (#837)
  • Update com.fasterxml.jackson to 2.13.4 to match opensearch repo. (#858)
  • Trigger build on pull request synchronize action. (#873)
  • Update Jetty Dependency (#872)
  • Fix manual CI workflow and add name option. (#904)
  • add groupId to pluginzip publication (#906)
  • Enable ci for windows and macos (#907)
  • Update group to groupId (#908)
  • Enable ignored and disabled tests (#926)
  • Update version of jackson-databind for sql-jdbc only (#943)
  • Add security policy for ml-commons library (#945)
  • Change condition to always upload coverage for linux workbench (#967)
  • Bump ansi-regex for workbench (#975)
  • Removed json-smart in the JDBC driver (#978)
  • Update MacOS Version for ODBC Driver (#987)
  • Update Jackson Databind version to 2.13.4.2 (#992)
  • [Backport 2.4] Bump sql-cli version to 1.1.0 (#1024)

Documentation

  • Add Forum link in SQL plugin README.md (#809)
  • Fix indentation of patterns example (#880)
  • Update docs - missing changes for #754. (#884)
  • Fix broken links (#911)
  • Adding docs related to catalog. (#963)
  • SHOW CATALOGS documentation and integ tests (#977)
  • [Backport 2.4] Add document for ml command. (#1017)

2.3.0.0

1 year ago

Compatible with OpenSearch and OpenSearch Dashboards Version 2.3.0

Features

  • Add maketime and makedate datetime functions (#755)

Enhancements

  • Refactor implementation of relevance queries (#746)
  • Extend query size limit using scroll (#716)
  • Add any case of arguments in relevancy based functions to be allowed (#744)

Bug Fixes

  • Fix unit test in PowerBI connector (#800)

Infrastructure

  • Schedule request in worker thread (#748)
  • Deprecated ClusterService and Using NodeClient to fetch metadata (#774)
  • Change master node timeout to new API (#793)

Documentation

  • Adding documentation about double quote implementation (#723)
  • Add PPL security setting documentation (#777)
  • Update PPL docs link for workbench (#758)

2.2.0.0

1 year ago

Compatible with OpenSearch and OpenSearch Dashboards Version 2.2.0

Features

  • Add match_phrase_prefix (#661)
  • query_string Relevance Function Implementation in SQL and PPL (#675)
  • Add Highlight In SQL (#717)

Enhancements

  • implement transport api for PPL inter-plugin communication (#533)
  • Two single or double quote escapes single or double quote when string is surrounded by same type of quote (#696)

Bug Fixes

  • Reverted UseSSL flag to false and removed invalid test case (#671)
  • Update BI connectors and drivers readme files (#665)
  • Bump moment from 2.29.2 to 2.29.4 in /workbench (#702)

Infrastructure

  • Staging for version increment automation (#684)
  • Update tests and test data for relevancy search functions (#707)
  • Remove ODFE BWC tests (#721)
  • Github Actions fix for reference to OpenSearch-Dashboard not existing outside of main OS sql project (#704)

Documentation

  • Update support link for Tableau connector (#643)
  • Fix broken forum link (#694)
  • Fix links in the doc file. (#705)

Maintenance

  • change version bump under maintenance (#679)
  • Bump version to 2.2.0 (#729)

JDBC-2.0.0.0

1 year ago

This version includes

  • Bump JDBC driver's dependency version #735
  • Update Jetty Dependency #872
  • Update JDBC driver version #941
  • Update version of jackson-databind for sql-jdbc only #943
  • Removed json-smart in the JDBC driver #978

Thanks

@forestmvey, @GumpacG and @dai-chen for their valuable contribution.

Binaries

Please, see unsigned binaries below:

Notes

The signed driver should be published on Maven repository soon.