Delta Rs Versions Save

A native Rust library for Delta Lake, with bindings into Python

python-v0.17.4

3 days ago

New features

Bug Fixes

Other Changes

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.17.3...python-v0.17.4

python-v0.17.3

1 week ago

New features

  • feat(rust): advance state in post commit by @ion-elgreco in #2396
  • feat: cdf reader for delta tables by @hntd187 in #2048
  • feat(python, rust): add OBJECT_STORE_CONCURRENCY_LIMIT setting for ObjectStoreFactory by @zZKato in #2458

Bug Fixes

  • fix(rust): handle 429 from GCS by @adriangb in #2454

Other changes

  • chore(rust): bump arrow v51 and datafusion v37.1 by @lasantosr in #2395

New Contributors

  • @adriangb made their first contribution in #2454
  • @zZKato made their first contribution in #2458

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.17.2...python-v0.17.3

rust-v0.17.3

1 week ago

rust-v0.17.3 (2024-05-01)

Full Changelog

Implemented enhancements:

  • Limit concurrent ObjectStore access to avoid resource limitations in constrained environments #2457
  • How to get a DataFrame in Rust? #2404
  • Allow checkpoint creation when partion column is "timestampNtz " #2381
  • is there a way to make writing timestamp_ntz optional #2339
  • Update arrow dependency #2328
  • Release GIL in deltalake.write_deltalake #2234
  • Unable to retrieve custom metadata from tables in rust #2153
  • Refactor commit interface to be a Builder #2131

Fixed bugs:

  • Handle rate limiting during write contention #2451
  • regression : delta.logRetentionDuration don't seems to be respected #2447
  • Issue writing to mounted storage in AKS using delta-rs library #2445
  • TableMerger - when_matched_delete() fails when Column names contain special characters #2438
  • Generic DeltaTable error: External error: Arrow error: Invalid argument error: arguments need to have the same data type - while merge data in to delta table #2423
  • Merge on predicate throw error on date colum: Unable to convert expression to string #2420
  • Writing Tables with Append mode errors if the schema metadata is different #2419
  • Logstore issues on AWS Lambda #2410
  • Datafusion timestamp type doesn't respect delta lake schema #2408
  • Compacting produces smaller row groups than expected #2386
  • ValueError: Partition value cannot be parsed from string. #2380
  • Very slow s3 connection after 0.16.1 #2377
  • Merge update+insert truncates a delta table if the table is big enough #2362
  • Do not add readerFeatures or writerFeatures keys under checkpoint files if minReaderVersion or minWriterVersion do not satisfy the requirements #2360
  • Create empty table failed on rust engine #2354
  • Getting error message when running in lambda: message: "Too many open files" #2353
  • Temporary files filling up _delta_log folder - increasing table load time #2351
  • compact fails with merged schemas #2347
  • Cannot merge into table partitioned by date type column on 0.16.3 #2344
  • Merge breaks using logical datatype decimal128 #2343
  • Decimal types are not checked against max precision/scale at table creation #2331
  • Merge update+insert truncates a delta table #2320
  • Extract add.stats_parsed with wrong type #2312
  • Process fails without error message when executing merge #2310
  • delta_rs don't seems to respect the row group size #2309
  • Auth error when running inside VS Code #2306
  • Unable to read deltatables with binary columns: Binary is not supported by JSON #2302
  • Schema evolution not coercing with Large arrow types #2298
  • Panic in deltalake_core::kernel::snapshot::log_segment::list_log_files_with_checkpoint::{{closure}} #2290
  • Checkpoint does not preserve reader and writer features for the table protocol. #2288
  • Z-Order with larger dataset resulting in memory error #2284
  • Successful writes return error when using concurrent writers #2279
  • Rust writer should raise when decimal types are incompatible (currently writers and puts table in invalid state) #2275
  • Generic DeltaTable error: Version mismatch with new schema merge functionality in AWS S3 #2262
  • DeltaTable is not resilient to corrupted checkpoint state #2258
  • Inconsistent units of time #2256
  • Partition column comparison is an assertion rather than if block with raise exception #2242
  • Unable to merge column names starting from numbers #2230
  • Merging to a table with multiple distinct partitions in parallel fails #2227
  • cleanup_metadata not respecting custom logRetentionDuration #2180
  • Merge predicate fails with a field with a space #2167
  • When_matched_update causes records to be lost with explicit predicate #2158
  • Merge execution time grows exponetially with the number of column #2107
  • _internal.DeltaError when merging #2084

python-v0.17.2

2 weeks ago

What's Changed

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.17.1...python-v0.17.2

python-v0.17.1

3 weeks ago

Bug Fixes

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.17.0...python-v0.17.1

python-v0.17.0

3 weeks ago

New features

Bug Fixes

Other Changes

New Contributors

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.16.4...python-v0.17.0

python-v0.16.4

1 month ago

Bug Fixes

Other Changes

New Contributors

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.16.3...python-v0.16.4

python-v0.16.3

1 month ago

New features

Bug Fixes

Other Changes

New Contributors

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.16.2...python-v0.16.3

python-v0.16.2

1 month ago

Bug Fixes

Other Changes

New Contributors

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.16.1...python-v0.16.2

python-v0.16.1

1 month ago

New features

Bug Fixes

Other Changes

New Contributors

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.16.0...python-v0.16.1