Eoreader Versions Save

Remote-sensing opensource python library reading optical and SAR sensors, loading and stacking bands, clouds, DEM and spectral indices in a sensor-agnostic way.

v0.19.2

1 year ago

0.19.2 (2023-02-23)

Bug Fixes

  • FIX: Fixing stack when saved as integer for some special cases
  • FIX: Clipping negative reflectances to 0 (#79)
  • FIX: Fixing nodata mangement for Theia product
  • FIX: Fixing handling of SCS multi-swath Cosmo-SkyMed products (#78)
  • FIX: Writing spectral indices on disk to align with other bands (#80)
  • FIX: By default, calibration is not applied to slant range CSG data, avoiding producing an empty raster (#48)

Other

  • OPTIM: Using warped VRT instead of reprojecting DEM/VHR stacks to UTM (#58)
  • TYP: Fixing typos in typing
  • INTERNAL: Moving EOREADER_NAME and DATETIME_FMT into __init__.py
  • INTERNAL: Moving stacking function into utils
  • INTERNAL: Removing unused cache_property decorator
  • INTERNAL: Factorizing _load function
  • CI: Using assert_raster_almost_equal_magnitude in CI to better check according to bands' content (sertit 1.24.0)
  • DEPS: Officially handling Python 3.11 (adding weekly tests on Python 3.11) (#71)
  • DOC: Updating Custom notebook
  • DOC: Updating jupyter cache to match new way of handling outputs in readthedocs

v0.19.1

1 year ago

0.19.1 (2023-01-12)

Bug Fixes

  • FIX: Fixing a bug for DIMAP V2 products whith GML masks opening without CRS: assigning first the raw CRS before converting to the product's CRS
  • FIX: Fixing index creation when exotic bands not handled by ASI have been loaded in the same time (i.e. stacking NDWI with Oa21 band)

Other

  • CLEAN: Removing useless GCP functions regarding Sentinel-3 data
  • DOC: Adding a Remove Clouds notebook
  • LIB: Pinning sertit to 1.22.0
  • CI: Don't run tests when only __init__ or __meta__ is updated
  • CI: Some factorizing in gitlab-ci

v0.19.0

1 year ago

0.19.0 (2023-01-03)

Enhancements

  • ENH: Adding the support of Capella constellation (#74)
  • ENH: Allow the user to load bands with a window (pixels and geo) (#25, notebook)

Bug Fixes

  • FIX: Fix extent computation for CSG products with Shapely 2.0
  • FIX: Shapely 2.0 deprecation warnings

Other

  • DEPR: Add deprecation warning for EOReader spectral indices (used for legacy in 0.18.0) that are aliases of ASI names (#72):
    • AFRI_1_6: AFRI1600,
    • AFRI_2_1: AFRI2100,
    • BSI: BI,
    • NDGRI: NGRDI,
    • NDRE1: NDREI,
    • RGI: RGRI,
    • WV_BI: NHFD,
    • WI: WI2015,
    • RDI: DSI,
    • DSWI: DSWI5,
    • GRI: DSWI4,
    • WV_SI: NDSIWV,
    • PANI: BITM
  • DOC: Changing copyright from 2022 to 2023

v0.18.1

1 year ago

0.18.1 (2022-12-08)

Bug Fixes

  • FIX: Fix regression for missing EOReader aliases for spyndex spectral indices

Other

  • DOC: Add latest DOI link

v0.18.0

1 year ago

0.18.0 (2022-12-06)

Breaking Changes

  • BREAKING CHANGES: Refactoring spectral indices management (#47)
    • Using spyndex library, allowing to use all spectral indices listed here
    • SAR products may now compute indices if possible (see this list)
    • Old EOReader indices are still available for legacy purposes, with some changes:
      • For Sentinel-2 data, the band NIR and NARROW_NIR may be interchanged for some index ( see this discussion)
      • OSAVI formula has changed to stick with the original paper definition (see issue)
      • NDRE2/3 formula are fixed, now using VRE_2/3 and NDRE1 corresponds to NDREI and uses VRE_1
      • CI1 is renamed CI32 and CI2 is renamed CI21 for readability purposes
      • NDWI21 can be written NDWI2100 for homogeneity purposes
      • RDI (or DSI) uses now SWIR_1 instead of SWIR_2 (see this issue)
      • PANI equivalent is now BITM and is normalised ! (divided by 3)
      • SBI is normalized (divided by 2) to fit with BIXS definition
      • You may need to install the last spyndex directly from GitHub latest version to have all available indices
  • BREAKING CHANGES: Using pyresample to geocode Sentinel-3 data (#55)
    • Cleaner: better conversion from swath to grid
    • Faster: Up to 4 times faster
    • Allows code refactoring between OLCI and SLSTR
  • BREAKING CHANGES: For SAR product types that are not available in the Data Access Portfolio, default resolution is now the pixel spacing instead of the rg x az resolution
    • Changes mainly Sentinel-1 default resolutions (except from IW mode)

Enhancements

  • ENH: Adding the support of Harmonized Landsat-Sentinel constellation (#49)
  • ENH: Adding the support of GEOSAT-2 constellation (#59)

Bug Fixes

  • FIX: Fixing CustomProduct initialization when fields are set to None (instead of not declaring them)
  • FIX: SNAP cannot handle float predictors other than 1! Set it to 1 when saving ortho SAR images to disk, in order for SNAP to be able to despeckle them (https://forum.step.esa.int/t/exception-found-when-reading-compressed-tif/654/7). (#62)
  • FIX: Fixing mix in Sentinel-2 mapping for B8 (NIR, 10m resolution, large spectral bandwidth) and B8A (NARROW_NIR, 20m resolution, narrow spectral bandwidth)

Other

  • DOC: Add FAQ entry concerning SAR constellations extent KML files failing to be read (TLDR: needs ogr2ogr in your PATH)
  • DOC: Add Technical Note published in Remote Sensing MDPI in Readme
  • DOC: Update optical band mapping graphs (fix regression to 0.15.0 supported constellation)
  • DOC: Add information about DEM management in SAR notebook (#61)
  • DOC: Updating indices paragraphs
  • CI: Using actions/checkout@v3
  • CI: Updating versions of pre-commit hooks
  • LIBS: Updating requirements.txt and setup.py to add pyresample and zarr

v0.17.0

1 year ago

0.17.0 (2022-10-12)

Enhancements

  • ENH: Adding the support of RapidEye constellation
  • ENH: Handling Planet data with multiple subdatasets (#45)
  • ENH: Adding the support of Landsat Level-2 products (#49)
  • ENH: Adding the support of Pleides Neo SEN and PRJ products (needs GDAL 3.5+ or rasterio 1.3.0+)
  • ENH: Adding the function bands.is_thermal_band
  • ENH: Adding the ability for optical custom stacks to load indices
  • ENH: Adding BAIM (MODIS Burned Area Index) spectral index
  • ENH: Better management of raw units of the bands of optical products
  • ENH: Copying files from tmp_process when changing product's output

Bug Fixes

  • FIX: Stacks saved as integers on disk keep their original dtype (float32) in Python
  • FIX: Stacks with bands loaded "as is" are correctly saved as integers on disk (#52)
  • FIX: Using stack CRS (if projected) for DIMAP products instead of recomputing from lat/lon, solving potential discrepencies between stack and product CRS
  • FIX: Workaround for JP2 bug when updating an existing raster (maybe related to this bug)
  • FIX: Better management of SkySat datetime conversion from JSON to XML (deterministic way)
  • FIX: Fixing computation of invalid pixels for Sentinel-2 and DIMAP products (do not remove straylight mask)
  • FIX: Fixing reprojection resolution of VHR data
  • FIX: Computing Brightness Temperature of Landsat TIR bands instead of leaving them as is
  • FIX: Better management of Landsat Instrument values
  • FIX: Better radiometry attribute (adding brightness temperature and reflectance and brightness temperature values)
  • FIX: Changing Brilliance Temperature to the correct Brightness Temperature
  • FIX: Fixing pandas FutureWarning The frame.append method is deprecated and will be removed from pandas in a future version.
  • FIX: Fixing DeprecationWarning invalid escape sequence \.
  • FIX: Manage correctly Planet dubious pixels (especially for 8 bands products)

Optimizations

  • OPTIM: Reduce memory usage when updating all the bands attributes
  • OPTIM: Reduce memory usage when stacking as integers

Other

  • DOC: Add the need of using SNAP 8.0 up-to-date or SNAP 9.0 (#42)
  • DOC: Add the STAC session in API documentation
  • DOC: Add warnings for shifts when orthorectifying DIMAP SEN products (using RPCs) (#53)
  • DOC: Add limitations to custom stacks
  • DEPS: Dropping support of Python 3.7 (#18)
  • DEPS: Update minimum version of libs (geopadans 0.11.0+, rasterio 1.3.0+...)

v0.16.1

1 year ago

0.16.1 (2022-08-03)

Bug Fixes

  • FIX: Add the missing conversion to reflectance for Sentinel-3 OLCI
  • FIX: Better condition for the conversion to reflectance for Sentinel-2 THEIA
  • FIX: Add logs for SkySat data that cannot been converted to reflectance and fix the radiometry field of its band xarrays
  • FIX: Add the correct nodata (when overridden by the user) to stacks saved as uint16

Optimizations

  • OPTIM: Reduce memory usage during stacking

Other

  • CI: Test reflectance values

v0.16.0

1 year ago

0.16.0 (2022-08-01)

Enhancements

  • ENH: Adding the support of SuperView-1 constellation (#21)
  • ENH: Adding the support of SPOT-4/5 constellations (#39)
  • ENH: Allow the possibility to pass a constellation (or a constellation list) to Reader().open() to speed up the opening of a product
  • ENH: Add a quicklook search for Sentinel-3 products

Bug Fixes

  • FIX: Fix quicklook media type with JP2 files
  • FIX: Fix Sentinel-3 SLSTR F1 bands based on F grid
  • FIX: Correct the UTM projection for Sentinel-3 data
  • FIX: Fix handling of zipped Sentinel-2 L2Ap
  • FIX: Fix zipped Sentinel-2 with other XML files in GRANULE subdirectories

Other

  • Renaming master branch to main

v0.15.1

1 year ago

0.15.1 (2022-06-02)

Optimizations

  • OPTIM: Try to create Vision-1 footprint from the preview file instead of from the stack.
  • OPTIM: Create footprints for stacked products (i.e. Maxar, SkySat, Custom...) without mask by opening only the first band of the stack
  • OPTIM: Create footprints for Maxar Products with a resolution 10 times lower.
  • OPTIM: Footprints have now maximum 50 vertices in order to avoid pixelized footprints

Bug Fixes

  • FIX: Fixing condensed name to avoid duplicates:
    • adding the job_id for VHR products
    • adding the polarization channels for SAR products
  • FIX: Remove import of pystac in stac_utils
  • FIX: Fix bug for Vision-1 data looking for non-existing RPC files in case of ORTP product type
  • FIX: Fix quicklook regex for Vision-1 data
  • FIX: Fix regex for raw bands for extracted Sentinel-3 OLCI products
  • FIX: Fix PlanetScope identifying regex to handle products with a satellite_id containing a letter
  • FIX: Force metadata regex for Maxar products to look for a file with pattern {name}.XML to avoid other misplaced XML to be found in place of the true XML.
  • FIX: Fix regression for Landsat-7 footprint
  • FIX: Manage the case with cloud_cover = -999.0 for Maxar products (returns None)

Other

  • CI: Add new optical products to be tested for end-to-end tests

v0.15.0

2 years ago

0.15.0 (2022-05-30)

Breaking Changes

  • BREAKING CHANGES: Optical becomes Spectral when more appropriate
  • BREAKING CHANGES: Platform and Sensor become Constellation when more appropriate, to fit STAC vocabulary (#29):
    • Platform enum becomes Constellation
    • prod.platform becomes prod.constellation
    • prod.sat_id becomes prod.constellation_id
  • BREAKING CHANGES: File alias is removed, replaced by *_bands files and proper imports in bands.__init__
  • BREAKING CHANGES: Product attribute band_names becomes bands in order to be STAC compliant (#29)
  • BREAKING CHANGES: Better use of NIR and NARROW_NIR in the indices file (according to the gsd of Sentinel-2 bands composing the indices)
  • BREAKING CHANGES: Correcting Landsat product types to better manage processing levels and instrument. Landsat-8/9 condensed name may change!

Enhancements

  • ENH: Adding the support of SkySat (Collect) products (#20)
  • ENH: Bands in mapping are now objects, instead of just IDs (#29). This allows us to:
    • Add band metadata (such as center wavelength, bandwidth...)
    • Map spectral bands between STAC spec and EOReader format (#29)
    • Add a better __repr__ functions
  • ENH: Handling 8 bands PlanetScope data (#20)
  • ENH: Adding the GREEN1 mapped band, corresponding to PlanetScope GREEN I and Sentinel-3 OLCI Oa05 band
  • ENH: Handle some slightly broken Sentinel-2 products:
    • when the metadata files are corrupted or when the detfoo vectors are empty (#34)
    • with missing MSK prefix for QI_DATA files (i.e DETFOO instead of MSK_DETFOO)
  • ENH: Handle exception for corrupted bands (in Sentinel-2 and utils.read) (#34)
  • ENH: Add a STAC object that can be used to retrieve STAC Items from every Product (prod.stac.create_item()) (#29)
  • ENH: Add a get_mean_viewing_angles for Optical Products to fill STAC View Extension (#29)
  • ENH: Extending get_raw_band_paths to every product (#31)
  • ENH: Adding a is_ortho attribute corresponding to when the product is already orthorectified/geocoded, in order to avoid computing heavy processes without wanting it (i.e. footprint...)
  • ENH: Adding the instrument name of every constellation, under prod.instrument
  • ENH: Handling COSMO product with only the h5 file in it (if missing XML metadata file) (#36)

Optimizations

  • OPTIM: Retrieve name from filename if possible
  • OPTIM: Retrieve extent from metadata when possible (for VHR data)
  • OPTIM: Refactoring Landsat-XX products into LandsatProduct, this should bee invisible for user.

Bug Fixes

  • FIX: Fixing the band mapping of WorldView-2/3 Multi (8 bands)
  • FIX: Retrieval (if possible) of Sentinel-1 unique ID (was missing from the product name, as it is not in the product preview)
  • FIX: Fixing PAZ/TDX MTD regex
  • FIX: Optical products: Only set cloud cover and radiometry attributes if spectral bands are asked

Other

  • INTERNAL: File spot_6 and spot_7 are removed, replaced by a unique spot file. This shouldn't affect the user.
  • INTERNAL: Some refactoring in VHR files
  • WARNINGS: Filter warnings from __init__
  • CI: Do not process two times the zipped Sentinel-1 in end-to-end tests and manage when the runner kills SNAP
  • DOC: Adding a STAC notebook
  • DOC: Various updates