ParquetSharp Versions Save

ParquetSharp is a .NET library for reading and writing Apache Parquet files.

12.0.1

10 months ago

This version of ParquetSharp is identical to 12.0.1-beta1, which has been promoted to stable.

What's Changed

Fix using internal F# types with the row oriented API by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/332
Use ISO 8601 date format for Date.ToString by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/330
Allow setting process-wide default writer properties by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/337
Don't retry writing in Dispose after encountering an error in ParquetRowWriter by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/344
Defer checking decimal precision and type length until creating converter by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/348
Add WriteRowSpan method to ParquetRowWriter by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/349
Upgrade to Arrow 12.0.1 by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/356

Misc

Update benchmark code and results by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/341
Update nested data documentation by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/342
remove deprecated ubuntu-18.04 runners by @ljubon in https://github.com/G-Research/ParquetSharp/pull/346
Document lack of Int96 timestamp support by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/331

Full Changelog: https://github.com/G-Research/ParquetSharp/compare/10.0.1...12.0.1

12.0.1-beta1

11 months ago

What's Changed

Fix using internal F# types with the row oriented API by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/332
Use ISO 8601 date format for Date.ToString by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/330
Allow setting process-wide default writer properties by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/337
Don't retry writing in Dispose after encountering an error in ParquetRowWriter by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/344
Defer checking decimal precision and type length until creating converter by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/348
Add WriteRowSpan method to ParquetRowWriter by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/349
Upgrade to Arrow 12.0.1 by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/356

Misc

Update benchmark code and results by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/341
Update nested data documentation by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/342
remove deprecated ubuntu-18.04 runners by @ljubon in https://github.com/G-Research/ParquetSharp/pull/346
Document lack of Int96 timestamp support by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/331

Full Changelog: https://github.com/G-Research/ParquetSharp/compare/10.0.1...12.0.1-beta1

10.0.1

1 year ago

This version of ParquetSharp is identical to 10.0.1-beta1, which has been promoted to stable.

This version upgrades Arrow to 10.0.1, which writes Parquet files with the version 2.4 format by default, and dictionary encoded data will use the RleDictionary encoding rather than PlainDictionary. Version 1 files can continue to be written by specifying the file format version in the WriterProperties.

What's Changed

Integrate Slack Github Action by @pavlovic-ivan in https://github.com/G-Research/ParquetSharp/pull/308
GitHub actions updates to fix deprecation warnings by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/310
Support round-tripping nested values by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/301
Avoid use of ManagedOutputStream and RandomAccessFile after GC by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/313
Support reading decimal data with int32 or int64 physical type by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/315
Convert paths to extend length format on Windows and add documentation on long path support by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/311
Allow creating a ParquetFileReader/Writer directly from a .NET stream by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/320
Support of optional nested-nested types reading by @oldukhno in https://github.com/G-Research/ParquetSharp/pull/319
Upgrade to Arrow 10.0.1 by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/321
Change version to 10.0.1 to match Arrow by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/323

Incompatible API Changes

LogicalColumnStream.SchemaNodesPath has been replaced by LogicalColumnStream.GetSchemaNodesPath

New Contributors

@pavlovic-ivan made their first contribution in https://github.com/G-Research/ParquetSharp/pull/308
@oldukhno made their first contribution in https://github.com/G-Research/ParquetSharp/pull/319

Full Changelog: https://github.com/G-Research/ParquetSharp/compare/8.0.0...10.0.1

10.0.1-beta1

1 year ago

What's Changed

Integrate Slack Github Action by @pavlovic-ivan in https://github.com/G-Research/ParquetSharp/pull/308
GitHub actions updates to fix deprecation warnings by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/310
Support round-tripping nested values by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/301
Avoid use of ManagedOutputStream and RandomAccessFile after GC by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/313
Support reading decimal data with int32 or int64 physical type by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/315
Convert paths to extend length format on Windows and add documentation on long path support by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/311
Allow creating a ParquetFileReader/Writer directly from a .NET stream by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/320
Support of optional nested-nested types reading by @oldukhno in https://github.com/G-Research/ParquetSharp/pull/319
Upgrade to Arrow 10.0.1 by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/321
Change version to 10.0.1 to match Arrow by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/323

Incompatible API Changes

LogicalColumnStream.SchemaNodesPath has been replaced by LogicalColumnStream.GetSchemaNodesPath

New Contributors

@pavlovic-ivan made their first contribution in https://github.com/G-Research/ParquetSharp/pull/308
@oldukhno made their first contribution in https://github.com/G-Research/ParquetSharp/pull/319

Full Changelog: https://github.com/G-Research/ParquetSharp/compare/8.0.0...10.0.1-beta1

8.0.0

1 year ago

This version of ParquetSharp is identical to 8.0.0-beta2, which has simply been promoted to stable.

What's Changed

Improve error message when getting a LogicalColumnReader or Writer with the wrong type by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/277
Use fixed culture when testing OverflowException by @Nisden in https://github.com/G-Research/ParquetSharp/pull/280
Fix reading string array values by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/282
Allow updating key-value metadata after ParquetFileWriter creation by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/279
Upgrade to Arrow 8.0.0 by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/283
Add support for .NET 6 by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/285
Allow specifying column definitions for a ParquetRowWriter by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/284
Added support for force_set_converted_type by @Nisden in https://github.com/G-Research/ParquetSharp/pull/281
Added switch to force DateTimeKind.Unspecified by @Nisden in https://github.com/G-Research/ParquetSharp/pull/288
Dispose of most uses of LogicalType and Node by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/290
Handle reading and writing string values without definition levels by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/293
Read whole leaf-level arrays at once when possible, rather than growing lists by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/295
Allow NullLogicalType given a PhysicalType by @jgiannuzzi, @msumner91, @adamreeve, and @mharward-gr in https://github.com/G-Research/ParquetSharp/pull/302

Misc

Update MacOS versions used in CI by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/294
Include runner's image identity into the cache key by @marcin-krystianc in https://github.com/G-Research/ParquetSharp/pull/289
Bump version to 8.0.0-beta2 by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/303
Promote 8.0.0-beta2 to stable release by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/307

New Contributors

@Nisden made their first contribution in https://github.com/G-Research/ParquetSharp/pull/280
@marcin-krystianc made their first contribution in https://github.com/G-Research/ParquetSharp/pull/289
@msumner91 made their first contribution in https://github.com/G-Research/ParquetSharp/pull/302
@mharward-gr made their first contribution in https://github.com/G-Research/ParquetSharp/pull/302

Full Changelog: https://github.com/G-Research/ParquetSharp/compare/7.0.0...8.0.0

8.0.0-beta2

1 year ago

What's Changed

Added support for force_set_converted_type by @Nisden in https://github.com/G-Research/ParquetSharp/pull/281
Added switch to force DateTimeKind.Unspecified by @Nisden in https://github.com/G-Research/ParquetSharp/pull/288
Dispose of most uses of LogicalType and Node by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/290
Handle reading and writing string values without definition levels by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/293
Read whole leaf-level arrays at once when possible, rather than growing lists by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/295
Allow NullLogicalType given a PhysicalType by @jgiannuzzi, @msumner91, @adamreeve, and @mharward-gr in https://github.com/G-Research/ParquetSharp/pull/302

Misc

Update MacOS versions used in CI by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/294
Include runner's image identity into the cache key by @marcin-krystianc in https://github.com/G-Research/ParquetSharp/pull/289
Bump version to 8.0.0-beta2 by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/303

New Contributors

@marcin-krystianc made their first contribution in https://github.com/G-Research/ParquetSharp/pull/289
@msumner91 made their first contribution in https://github.com/G-Research/ParquetSharp/pull/302
@mharward-gr made their first contribution in https://github.com/G-Research/ParquetSharp/pull/302

Full Changelog: https://github.com/G-Research/ParquetSharp/compare/8.0.0-beta1...8.0.0-beta2

8.0.0-beta1

1 year ago

What's Changed

Improve error message when getting a LogicalColumnReader or Writer with the wrong type by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/277
Use fixed culture when testing OverflowException by @Nisden in https://github.com/G-Research/ParquetSharp/pull/280
Fix reading string array values by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/282
Allow updating key-value metadata after ParquetFileWriter creation by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/279
Upgrade to Arrow 8.0.0 by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/283
Add support for .NET 6 by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/285
Allow specifying column definitions for a ParquetRowWriter by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/284

New Contributors

@Nisden made their first contribution in https://github.com/G-Research/ParquetSharp/pull/280

Full Changelog: https://github.com/G-Research/ParquetSharp/compare/7.0.0...8.0.0-beta1

7.0.0

2 years ago

What's Changed

Upgrade to Arrow 7.0.0 by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/268
Change DateTimeNanos.ToString format and allow overriding the format by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/262
Read DateTime values with Kind set based on isAdjustedToUtc by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/261
Handle partial stream reads in ManagedRandomAccessFile by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/264
Fix vcpkg binary caching in CI by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/270
Build linux-x64 in a CentOS 7 container for backwards compatibility by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/274
Promote 7.0.0-beta1 to stable release by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/276

Full Changelog: https://github.com/G-Research/ParquetSharp/compare/6.0.1...7.0.0

7.0.0-beta1

2 years ago

What's Changed

Upgrade to Arrow 7.0.0 by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/268
Change DateTimeNanos.ToString format and allow overriding the format by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/262
Read DateTime values with Kind set based on isAdjustedToUtc by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/261
Handle partial stream reads in ManagedRandomAccessFile by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/264

Full Changelog: https://github.com/G-Research/ParquetSharp/compare/6.0.1...7.0.0-beta1

6.0.1

2 years ago

What's Changed

Update to Arrow 6.0.1 by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/247
Fix GetEnumerator by @philjdf in https://github.com/G-Research/ParquetSharp/pull/244
Support Windows Server 2022 CI builds by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/258
Restructure and expand documentation by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/257
Switch to using a vcpkg manifest to decouple Arrow version from vcpkg version by @adamreeve in https://github.com/G-Research/ParquetSharp/pull/265
Promote 6.0.1-beta1 to stable release by @jgiannuzzi in https://github.com/G-Research/ParquetSharp/pull/267

Incompatible API changes

ParquetSharp. LogicalColumnReader.HasNext has been made abstract

Full Changelog: https://github.com/G-Research/ParquetSharp/compare/5.0.0...6.0.1