Datacube Core Versions Save

Open Data Cube analyses continental scale Earth Observation data through time

1.9.0-rc4

3 weeks ago

Datacube 1.9.0 release candidate 3

The last new major release of the Open Data Cube was v1.8.0 in May 2020, nearly 4 years ago.

ODC developers and the Steering Council have been working hard behind the scenes over the last couple of years to address some of the accumulated technical debt in datacube-core and prepare for new major releases.

The long-term plan includes a number of significant backwards-incompatible changes. An effort has been made to provide a smooth migration pathway wherever possible, with existing behaviour in 1.8.x versions being deprecated in 1.9.x, with alternatives being provided, then removing the deprecated behaviour in 2.0.x with the alternative approaches becoming the standard, but some minor backwards incompatible changes in 1.9.x were unavoidable.

This is the first release candidate for the 1.9.x release series. Please refer to the Migration Notes document before attempting to upgrade from a 1.8 series installation.

Changes since 1.9.0-rc3:

Mostly changes to ease migration from 1.8.x

  • Standardize resampling input supported to odc.geo.warp.Resampling (better odc-geo compatibility(:pull:1571)
  • Refine default behaviour for config engine to support easier migration from 1.8 (:pull:1573)
  • Convert legacy GeoBoxes to odc.geo GeoBoxes in the core API (:pull:1574)
  • Add URL component pseudo to config layer to expose components to the api when configured as a URL, and reformat whats_new for 1.9.0-rc4 release. (:pull:1575)

Includes contributions from @SpacemanPaul and @snowman2

Other changes since 1.8.18:

  • Merge in 1.8.x branch changes. (#1459, #1473, #1532, #1548, #1565)
  • External Lineage API (#1401)
  • Add lineage support to index clone operation (#1429)
  • Migrate to SQLAlchemy 2.0 (#1432)
  • Clean up deprecated code and add deprecation warnings to legacy methods, simplify DocReader logic (#1406)
  • Mark geometry module as deprecated and replace all usage with odc-geo (#1424)
  • Mark GridSpec as deprecated, replace math and cog functions with odc-geo equivalents, enforce new odc-geo conventions (#1441)
  • Rename gbox to geobox in parameter names (#1441)
  • Remove executor API (#1462)
  • Remove ingestion methods, GridWorkflow and Tile classes (#1465)
  • Fix postgis queries for numeric custom search fields (#1475)
  • Document best practice for pulling in changes from develop and update constraints.txt (#1478)
  • Postgis index driver performance tuning (#1480)
  • Cleanup and formalise spatial index API and expose in CLI (#1481)
  • Increase minimum Python version to 3.10 (#1509)
  • Virtual product tests using odc-geo GridSpec (#1512)
  • New Configuration API, as per ODC-EP10 (#1505)
  • Alembic migrations for postgis driver (#1520)
  • EP08 lineage extensions/changes to datasets.get(). (#1530)
  • EP13 API changes to Index and IndexDriver. (#1534)
  • EP13 API changes to metadata and product resources. (#1536)
  • Phase 1 of EP13 API changes to dataset resource - get_unsafe, get_derived, temporal_extent. (#1538)
  • Add product argument to spatial_extent method, as per EP13. (#1539)
  • Index driver API type hint cleanup. (#1541)
  • Deprecate multiple locations. (#1546)
  • Deprecate search_eager and search_summaries and add archived arg to all dataset search/count methods. (#1550)
  • Migrate away from deprecated Python pkg_resources module (#1558)
  • Add custom_offsets and order_by arguments to search_retunrning() - order_by still unimplemented. (#1557)
  • Fix and enhance typehints, automated static type checking with mypy. (#1562)
  • Improve SQLAlchemy join hints, addressing an recurring but intermittent bug. (#1564)
  • Improve typehints and update docstrings in datacube/api/core.py (#1567)
  • Add migration notes, update documentation and whats_new.rst for 1.9.0-rc1 release (#1569)

Includes contributions from @SpacemanPaul and @Ariana-B, with assistance from @omad and @robbibt

With thanks to the Open Data Cube Steering Council and all supporting organisations, especially Geoscience Australia.

1.9.0-rc3

1 month ago

This is re-pre-release of 1.9.0-rc1 to get the github and pypi pre-release version numbering back in sync.

Datacube 1.9.0 release candidate 3

The last new major release of the Open Data Cube was v1.8.0 in May 2020, nearly 4 years ago.

ODC developers and the Steering Council have been working hard behind the scenes over the last couple of years to address some of the accumulated technical debt in datacube-core and prepare for new major releases.

The long-term plan includes a number of significant backwards-incompatible changes. An effort has been made to provide a smooth migration pathway wherever possible, with existing behaviour in 1.8.x versions being deprecated in 1.9.x, with alternatives being provided, then removing the deprecated behaviour in 2.0.x with the alternative approaches becoming the standard, but some minor backwards incompatible changes in 1.9.x were unavoidable.

This is the first release candidate for the 1.9.x release series. Please refer to the Migration Notes document before attempting to upgrade from a 1.8 series installation.

  • Merge in 1.8.x branch changes. (#1459, #1473, #1532, #1548, #1565)
  • External Lineage API (#1401)
  • Add lineage support to index clone operation (#1429)
  • Migrate to SQLAlchemy 2.0 (#1432)
  • Clean up deprecated code and add deprecation warnings to legacy methods, simplify DocReader logic (#1406)
  • Mark geometry module as deprecated and replace all usage with odc-geo (#1424)
  • Mark GridSpec as deprecated, replace math and cog functions with odc-geo equivalents, enforce new odc-geo conventions (#1441)
  • Rename gbox to geobox in parameter names (#1441)
  • Remove executor API (#1462)
  • Remove ingestion methods, GridWorkflow and Tile classes (#1465)
  • Fix postgis queries for numeric custom search fields (#1475)
  • Document best practice for pulling in changes from develop and update constraints.txt (#1478)
  • Postgis index driver performance tuning (#1480)
  • Cleanup and formalise spatial index API and expose in CLI (#1481)
  • Increase minimum Python version to 3.10 (#1509)
  • Virtual product tests using odc-geo GridSpec (#1512)
  • New Configuration API, as per ODC-EP10 (#1505)
  • Alembic migrations for postgis driver (#1520)
  • EP08 lineage extensions/changes to datasets.get(). (#1530)
  • EP13 API changes to Index and IndexDriver. (#1534)
  • EP13 API changes to metadata and product resources. (#1536)
  • Phase 1 of EP13 API changes to dataset resource - get_unsafe, get_derived, temporal_extent. (#1538)
  • Add product argument to spatial_extent method, as per EP13. (#1539)
  • Index driver API type hint cleanup. (#1541)
  • Deprecate multiple locations. (#1546)
  • Deprecate search_eager and search_summaries and add archived arg to all dataset search/count methods. (#1550)
  • Migrate away from deprecated Python pkg_resources module (#1558)
  • Add custom_offsets and order_by arguments to search_retunrning() - order_by still unimplemented. (#1557)
  • Fix and enhance typehints, automated static type checking with mypy. (#1562)
  • Improve SQLAlchemy join hints, addressing an recurring but intermittent bug. (#1564)
  • Improve typehints and update docstrings in datacube/api/core.py (#1567)
  • Add migration notes, update documentation and whats_new.rst for 1.9.0-rc1 release (#1569)

Includes contributions from @SpacemanPaul and @Ariana-B, with assistance from @omad and @robbibt

With thanks to the Open Data Cube Steering Council and all supporting organisations, especially Geoscience Australia.

1.9.0-rc1

1 month ago

The last new major release of the Open Data Cube was v1.8.0 in May 2020, nearly 4 years ago.

ODC developers and the Steering Council have been working hard behind the scenes over the last couple of years to address some of the accumulated technical debt in datacube-core and prepare for new major releases.

The long-term plan includes a number of significant backwards-incompatible changes. An effort has been made to provide a smooth migration pathway wherever possible, with existing behaviour in 1.8.x versions being deprecated in 1.9.x, with alternatives being provided, then removing the deprecated behaviour in 2.0.x with the alternative approaches becoming the standard, but some minor backwards incompatible changes in 1.9.x were unavoidable.

This is the first release candidate for the 1.9.x release series. Please refer to the docs/MIGRATION-1.8-to-1.9.rst document before attempting to upgrade from a 1.8 series installation.

  • Merge in 1.8.x branch changes. (#1459, #1473, #1532, #1548, #1565)
  • External Lineage API (#1401)
  • Add lineage support to index clone operation (#1429)
  • Migrate to SQLAlchemy 2.0 (#1432)
  • Clean up deprecated code and add deprecation warnings to legacy methods, simplify DocReader logic (#1406)
  • Mark geometry module as deprecated and replace all usage with odc-geo (#1424)
  • Mark GridSpec as deprecated, replace math and cog functions with odc-geo equivalents, enforce new odc-geo conventions (#1441)
  • Rename gbox to geobox in parameter names (#1441)
  • Remove executor API (#1462)
  • Remove ingestion methods, GridWorkflow and Tile classes (#1465)
  • Fix postgis queries for numeric custom search fields (#1475)
  • Document best practice for pulling in changes from develop and update constraints.txt (#1478)
  • Postgis index driver performance tuning (#1480)
  • Cleanup and formalise spatial index API and expose in CLI (#1481)
  • Increase minimum Python version to 3.10 (#1509)
  • Virtual product tests using odc-geo GridSpec (#1512)
  • New Configuration API, as per ODC-EP10 (#1505)
  • Alembic migrations for postgis driver (#1520)
  • EP08 lineage extensions/changes to datasets.get(). (#1530)
  • EP13 API changes to Index and IndexDriver. (#1534)
  • EP13 API changes to metadata and product resources. (#1536)
  • Phase 1 of EP13 API changes to dataset resource - get_unsafe, get_derived, temporal_extent. (#1538)
  • Add product argument to spatial_extent method, as per EP13. (#1539)
  • Index driver API type hint cleanup. (#1541)
  • Deprecate multiple locations. (#1546)
  • Deprecate search_eager and search_summaries and add archived arg to all dataset search/count methods. (#1550)
  • Migrate away from deprecated Python pkg_resources module (#1558)
  • Add custom_offsets and order_by arguments to search_retunrning() - order_by still unimplemented. (#1557)
  • Fix and enhance typehints, automated static type checking with mypy. (#1562)
  • Improve SQLAlchemy join hints, addressing an recurring but intermittent bug. (#1564)
  • Improve typehints and update docstrings in datacube/api/core.py (#1567)
  • Add migration notes, update documentation and whats_new.rst for 1.9.0-rc1 release (#1569)

Includes contributions from @SpacemanPaul and @Ariana-B, with assistance from @omad and @robbibt

With thanks to the Open Data Cube Steering Council and all supporting organisations, especially Geoscience Australia.

1.8.18

1 month ago

Transitional Release

1.8.18 contains bugfixes, as well as enhancements and changes to help prepare for the migration to Open Data Cube 1.9.x. This release likely contains the last significant enhancements to the 1.8.x series. The first 1.9 series release candidate will be published imminently. Future development will focus on the 1.9.x series, although 1.8.x series maintenance and bugfix releases will continue for the forseeable future.

  • Add dataset cli tool find-duplicates to identify duplicate indexed datasets (#1517)
  • Make solar_day() timezone aware (#1521)
  • Warn if non-eo3 dataset has eo3 metadata type (#1523)
  • Update pandas version in docker image to be consistent with conda environment and default to stdlib timezone instead of pytz when converting timestamps; automatically update copyright years (#1527)
  • Update github-Dockerhub credential-passing mechanism. (#1528)
  • Tweak list_products logic for getting crs and resolution values (#1535)
  • Add new ODC Cheatsheet reference doc to Data Access & Analysis documentation page (#1543)
  • Compatibility fix to allow users to supply odc.geo-style GeoBoxes to dc.load(like=...) (#1551)
  • Fix broken codecov github action. (#1554)
  • Update documentation links to DEA Knowledge Hub (#1559)
  • Throw error if time dimension is provided as an int or float to Query construction instead of assuming it to be seconds since epoch (#1561)
  • Add generic NOT operator and for ODC queries and Not type wrapper (#1563)
  • Update whats_new.rst for release (#1568)

Includes contributions from @robbibt, @Ariana-B, @benji-glitsos-ga and @SpacemanPaul

1.9.0-pre02

1 month ago

Second pre-release of the 1.9 branch.

This is not a release candidate and is intended for internal use.

1.9.0-pre01

3 months ago

Administrative pre-release to facilitate team cooperation.

1.8.17

6 months ago

datacube-1.8.17: Maintenance release

  • Fix database permission-handling bug affecting datacube system init --no-init-users for the postgres index driver (#1504).
  • Switch to new jsonschema "referencing" API, and pin jsonschema>=4.18 (#1477)
  • Update whats_new.rst for 1.8.17 release (#1510)

With contributions from @Ariana-B and @SpacemanPaul. Thanks to @omad and @emmaai for identifying the issues.

1.8.16

6 months ago

Maintenance release

Various bug fixes and updates.

Previous releases of the ODC have supported Python 3.8+. With Python 3.8 now at end-of-life, and recent releases of key datacube dependencies (e.g. Xarray and PyProj) already requiring Python 3.9, this release officially drops support for Python 3.8.

Full list of changes:

  • Improve error message for mismatch between dataset metadata and product signature #1472
  • Mark --confirm-ignore-lineage, --auto-add-lineage, and --verify-lineage as deprecated or to be deprecated #1472
  • Default delta values in archive_less_mature and find_less_mature #1472
  • Fix SQLAlchemy calls and pin jsonschema version to suppress deprecation warnings #1476
  • Throw a better error if a dataset is not compatible with archive_less_mature logic #1491
  • Fix broken Github action workflow #1496
  • Support like=<GeoBox> in virtual product load #1497
  • Don't archive less mature if archive_less_mature is provided as False instead of None #1498
  • Raise minimum supported Python version to 3.9 #1500
  • Manually apply Dependabot updates, and update whats_new.rst for 1.8.16 release #1501

With contributions from @SpacemanPaul @Ariana-B and @uchchwhash

Special thanks to all supporting organisations, especially Geoscience Australia.

1.8.15

9 months ago

The previous release of the Open Data Cube (1.8.14) replaced a deprecated import API for driver loading. Unfortunately the new API used is only supported by Python>=3.10. This means that v1.8.14 was not compatible with Python 3.8 and 3.9.

The Open Data Cube aims to support Python >= 3.8 and we apologise to the community for this oversight.

This v1.8.15 release replaces the problematic API with one that is supported by Python 3.8 and 3.9 (#1469), and updates whats_new.rst (#1470). There are no other changes in this release.

Thanks to @emmaai for identifying the issue and @Ariana-B for fixing it.

1.8.14

10 months ago

Major changes in this release:

  1. The attempted fix to the projection bug in the previous release turned out to be inadequate, this release contains a fix for the bug fix.
  2. The behaviour of the archive-less-mature option on dataset-add has been enhanced (default time leniency increased to +/-500ms, with the option for the user to set their own leniency threshold.
  3. Several old and poorly maintained APIs have been formally deprecated in this release, and will be removed from datacube-core in the 1.9.0 release: ingestion (which the documentation has recommended against using for many years), executors and grid workflow. If you still use any of these APIs, you can continue to use them in this release (and will still be able to use them in all future 1.8.x releases) by simply ignoring the deprecation warnings. If you have workflows that still depend on these APIs, please raise your requirements on the ODC Slack community so we can help you identify alternative approaches. There is no planned release date for v1.9.0 at this stage.

Full list of changes in this release:

  • Second attempt to address unexpected handling of image aspect ratios in rasterio and GDAL. (#1457)
  • Fix broken PyPI publishing Github action (#1454)
  • Documentation improvements (#1455)
  • Increase default maturity leniency to +-500ms (#1458)
  • Add option to specify maturity timedelta when using --archive-less-mature option (#1460)
  • Mark executors as deprecated (#1461)
  • Mark ingestion as deprecated (#1463)
  • Replace deprecated pkg_resources with importlib.resources and importlib.metadata (#1466)
  • Update whats_new.rst for release (#1467)

Includes code contributions from @SpacemanPaul and @Ariana-B.

With thanks to @Kirill888, @robbibt and @snowman2 for their help in identifying and finding a viable fix for the projection bug and to Geoscience Australia for their continuing support of ODC development and maintenance.