Icelake Versions Save

Pure Rust Iceberg Implementation

v0.0.10

8 months ago

What's Changed

feat: add interface of transform function by @ZENOTME in https://github.com/icelake-io/icelake/pull/130
chore: use arrow instead of arrow_array and arrow_schema by @ZENOTME in https://github.com/icelake-io/icelake/pull/131
feat: Add integration tests by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/118
Remove unused code. by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/134
refactor: refine StructValue by @ZENOTME in https://github.com/icelake-io/icelake/pull/135
refactor: refactor Struct and StructValue by @ZENOTME in https://github.com/icelake-io/icelake/pull/136
refactor: Use libtest-mimic to run integration tests. by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/137
chore: allow depending on arrow v45 and parquet v45 by @wangrunji0408 in https://github.com/icelake-io/icelake/pull/139
fix: add optional case in list, map,parse,serde by @ZENOTME in https://github.com/icelake-io/icelake/pull/140
fix: Fix possible empty snapshot by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/144
feat: support convert arrow value and type to AnyValue, Any by @ZENOTME in https://github.com/icelake-io/icelake/pull/141
feat: Expose methods for ser/de data file by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/145
feat: DataFile from/to json by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/146
chore: fix chore in arrow module by @ZENOTME in https://github.com/icelake-io/icelake/pull/147
feat: Introduce table config by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/148
test: More data types tests by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/150
test: Add test for timestamp datatype test by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/151
fix: manifest_file,manifest_list schema by @ZENOTME in https://github.com/icelake-io/icelake/pull/152
feat: support partition writer by @ZENOTME in https://github.com/icelake-io/icelake/pull/153
refactor: unify data file writer config by @ZENOTME in https://github.com/icelake-io/icelake/pull/155
feat: support more array cast and more test by @ZENOTME in https://github.com/icelake-io/icelake/pull/156
test: Use testcontainers to simplify tests. by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/157
refactor: support nest field look up by @ZENOTME in https://github.com/icelake-io/icelake/pull/158
refactor: support to select column by source coulumn id by @ZENOTME in https://github.com/icelake-io/icelake/pull/159
chore: allow depending on arrow and parquet v46 by @wangrunji0408 in https://github.com/icelake-io/icelake/pull/162
require Send for TransformFunction by @wangrunji0408 in https://github.com/icelake-io/icelake/pull/163
chore: Upgrade arrow to 46 by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/164
chore: Fix arrow version by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/165
feat: support temporal partition function by @ZENOTME in https://github.com/icelake-io/icelake/pull/161
fix: remove redundant println! by @ZENOTME in https://github.com/icelake-io/icelake/pull/166
fix: Use separate arrow crates by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/169
refactor: refactor the writer test framework by @ZENOTME in https://github.com/icelake-io/icelake/pull/170
feat: Define catalog api by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/171
feat: support bucket/truncate transform by @ZENOTME in https://github.com/icelake-io/icelake/pull/173
feat: Implement rest catalog tables and add test framework by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/179
feat: support Decimal in partition value by @ZENOTME in https://github.com/icelake-io/icelake/pull/178
feat: Introduce load table api by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/181
refactor: Remove unnecessary codes by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/183
chore: prepare version 0.0.10. by @RinChanNOWWW in https://github.com/icelake-io/icelake/pull/186

New Contributors

@wangrunji0408 made their first contribution in https://github.com/icelake-io/icelake/pull/139
@RinChanNOWWW made their first contribution in https://github.com/icelake-io/icelake/pull/186

Full Changelog: https://github.com/icelake-io/icelake/compare/v0.0.9...v0.0.10

v0.0.9

8 months ago

What's Changed

feat: Ser/de for schema by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/95
feat: Ser/de partition spec by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/101
feat: support serialize AnyValue to Json by @ZENOTME in https://github.com/icelake-io/icelake/pull/103
Complete avro types conversion by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/105
Add manifest writer tests by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/104
feat: support manifest list writer v2 by @ZENOTME in https://github.com/icelake-io/icelake/pull/107
Add support for serializing table metadata. by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/108
chore: bump arrow-*,parquet version range up to 45 by @ZENOTME in https://github.com/icelake-io/icelake/pull/109
fix: fix the serialize way of struct value by @ZENOTME in https://github.com/icelake-io/icelake/pull/110
refactor: replace lazy static to once cell by @PsiACE in https://github.com/icelake-io/icelake/pull/112
feat: add metadata for manifest list writer by @ZENOTME in https://github.com/icelake-io/icelake/pull/111
feat: add partition field in data file by @ZENOTME in https://github.com/icelake-io/icelake/pull/115
Add transaction api to support add data files. by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/113
Refactor to make it a workspace by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/117
feat: replace f32,f64 with ordered_float by @ZENOTME in https://github.com/icelake-io/icelake/pull/119
Generate icelake rust open api client by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/121
refactor: return Error instaed of ArrowError for to_arrow by @ZENOTME in https://github.com/icelake-io/icelake/pull/123
refactor: change the task writer to support more writer later by @ZENOTME in https://github.com/icelake-io/icelake/pull/124
fix: return Vec<Datafile> instead of HashMap by @ZENOTME in https://github.com/icelake-io/icelake/pull/126
ci: Make build happy by @Xuanwo in https://github.com/icelake-io/icelake/pull/127
feat: OpenDAL is known to work on 0.39 by @Xuanwo in https://github.com/icelake-io/icelake/pull/128
Bump to version 0.0.9 by @Xuanwo in https://github.com/icelake-io/icelake/pull/129

New Contributors

@PsiACE made their first contribution in https://github.com/icelake-io/icelake/pull/112

Full Changelog: https://github.com/icelake-io/icelake/compare/v0.0.8...v0.0.9

v0.0.8

10 months ago

What's Changed

feat: Initial check in of mainfest writer. by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/91
chore: allow depending on arrow version 43.0.0. by @youngsofun in https://github.com/icelake-io/icelake/pull/92
Bump to version 0.0.8 by @Xuanwo in https://github.com/icelake-io/icelake/pull/93

New Contributors

@youngsofun made their first contribution in https://github.com/icelake-io/icelake/pull/92

Full Changelog: https://github.com/icelake-io/icelake/compare/v0.0.7...v0.0.8

v0.0.7

10 months ago

What's Changed

feat: support data file location generator by @ZENOTME in https://github.com/icelake-io/icelake/pull/60
docs: Add docs for all public APIs by @Xuanwo in https://github.com/icelake-io/icelake/pull/64
deps: Allow deps within range by @Xuanwo in https://github.com/icelake-io/icelake/pull/65
feat: Add PrimitiveValue by @Xuanwo in https://github.com/icelake-io/icelake/pull/66
feat(in_memory): Add {Any,Struct,List,Map}Value by @Xuanwo in https://github.com/icelake-io/icelake/pull/67
feat: Add default to in-memory structs by @Xuanwo in https://github.com/icelake-io/icelake/pull/68
feat: Implement parser for initial-default and write-default by @Xuanwo in https://github.com/icelake-io/icelake/pull/69
chore: Enable all-features for docs.rs by @Xuanwo in https://github.com/icelake-io/icelake/pull/70
docs: Make rustdoc happy by @Xuanwo in https://github.com/icelake-io/icelake/pull/71
feat: implement data writer by @ZENOTME in https://github.com/icelake-io/icelake/pull/72
feat: support to track written size by @ZENOTME in https://github.com/icelake-io/icelake/pull/73
feat: support task writer by @ZENOTME in https://github.com/icelake-io/icelake/pull/75
feat: support open_with_operator by @ZENOTME in https://github.com/icelake-io/icelake/pull/76
fix: Rename some data structures. by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/77
fix: uncorrect partition write predict by @ZENOTME in https://github.com/icelake-io/icelake/pull/78
fix: make sure the version is valid when the table open by @ZENOTME in https://github.com/icelake-io/icelake/pull/81
fix: switch any::Result to crate::error::Result in table.rs by @ZENOTME in https://github.com/icelake-io/icelake/pull/82
ci: Add partition table testdata by @Xuanwo in https://github.com/icelake-io/icelake/pull/84
refactor: Use table format version in manifest by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/85
feat: make task_writer to use imutable reference by @ZENOTME in https://github.com/icelake-io/icelake/pull/87
feat: Convert iceberg schema to avro schema by @liurenjie1024 in https://github.com/icelake-io/icelake/pull/88
Bump to version 0.0.7 by @Xuanwo in https://github.com/icelake-io/icelake/pull/89

New Contributors

@liurenjie1024 made their first contribution in https://github.com/icelake-io/icelake/pull/77

Full Changelog: https://github.com/icelake-io/icelake/compare/v0.0.6...v0.0.7

v0.0.6

10 months ago

What's Changed

feat: Add support for writing parquet by @ZENOTME in https://github.com/icelake-io/icelake/pull/50
feat(io/parquet): Polish API by @Xuanwo in https://github.com/icelake-io/icelake/pull/53
feat: Add icelake own error type by @Xuanwo in https://github.com/icelake-io/icelake/pull/54
chore: Promote parquet as a mod by @Xuanwo in https://github.com/icelake-io/icelake/pull/55
feat: Implement parquet stream by @Xuanwo in https://github.com/icelake-io/icelake/pull/59
chore: add .vscode in .gitignore by @ZENOTME in https://github.com/icelake-io/icelake/pull/61
Bump to version 0.0.6 by @Xuanwo in https://github.com/icelake-io/icelake/pull/62

New Contributors

@ZENOTME made their first contribution in https://github.com/icelake-io/icelake/pull/50

Full Changelog: https://github.com/icelake-io/icelake/compare/v0.0.5...v0.0.6

v0.0.5

10 months ago

What's Changed

refactor: Use arrow schema to reduce dependence tree size by @Xuanwo in https://github.com/icelake-io/icelake/pull/45
feat: Add test data generate tools by @Xuanwo in https://github.com/icelake-io/icelake/pull/46
Bump to version 0.0.5 by @Xuanwo in https://github.com/icelake-io/icelake/pull/48

Full Changelog: https://github.com/icelake-io/icelake/compare/v0.0.4...v0.0.5

v0.0.4

10 months ago

What's Changed

feat: Implement convert to Arrow Schema by @Xuanwo in https://github.com/icelake-io/icelake/pull/42
ci: Cover all features and targets by @Xuanwo in https://github.com/icelake-io/icelake/pull/43
Bump to version 0.0.4 by @Xuanwo in https://github.com/icelake-io/icelake/pull/44

Full Changelog: https://github.com/icelake-io/icelake/compare/v0.0.3...v0.0.4

v0.0.3

10 months ago

What's Changed

feat: Implement table load without version hint by @Xuanwo in https://github.com/icelake-io/icelake/pull/34
Bump to version 0.0.3 by @Xuanwo in https://github.com/icelake-io/icelake/pull/35

Full Changelog: https://github.com/icelake-io/icelake/compare/v0.0.2...v0.0.3

v0.0.2

10 months ago

What's Changed

feat: add open table interface and add an example to read table by @xudong963 in https://github.com/icelake-io/icelake/pull/27
chore: remove Cargo.lock by @TennyZhuang in https://github.com/icelake-io/icelake/pull/30
feat: Add current data files support by @Xuanwo in https://github.com/icelake-io/icelake/pull/31
Bump to version 0.0.2 by @Xuanwo in https://github.com/icelake-io/icelake/pull/32

New Contributors

@xudong963 made their first contribution in https://github.com/icelake-io/icelake/pull/27
@TennyZhuang made their first contribution in https://github.com/icelake-io/icelake/pull/30

Full Changelog: https://github.com/icelake-io/icelake/compare/v0.0.1...v0.0.2

v0.0.1

11 months ago

What's Changed

feat: Add project layout by @Xuanwo in https://github.com/icelake-io/icelake/pull/2
feat: Add iceberg schema data types by @Xuanwo in https://github.com/icelake-io/icelake/pull/3
feat: Add Schema V2 types by @Xuanwo in https://github.com/icelake-io/icelake/pull/5
feat: Implement parse json into SchemaV2 by @Xuanwo in https://github.com/icelake-io/icelake/pull/6
refactor: Use move instead of clone from reference by @Xuanwo in https://github.com/icelake-io/icelake/pull/7
feat: Add Transform, Partition and Sorting by @Xuanwo in https://github.com/icelake-io/icelake/pull/8
feat: Implement parse partition spec by @Xuanwo in https://github.com/icelake-io/icelake/pull/9
feat: Implement parse from sort_order by @Xuanwo in https://github.com/icelake-io/icelake/pull/10
feat: Add manifest related types by @Xuanwo in https://github.com/icelake-io/icelake/pull/11
feat: Implement manifest parse by @Xuanwo in https://github.com/icelake-io/icelake/pull/12
refactor: Remove version suffix by @Xuanwo in https://github.com/icelake-io/icelake/pull/13
refactor: Re-organize the project layout by @Xuanwo in https://github.com/icelake-io/icelake/pull/14
fix: parse_manifest_list should return lists instead by @Xuanwo in https://github.com/icelake-io/icelake/pull/15
feat: Add in-memory layout for snapshot by @Xuanwo in https://github.com/icelake-io/icelake/pull/16
feat: Implement parse_snapshot by @Xuanwo in https://github.com/icelake-io/icelake/pull/17
feat: Implement version_hint for Table by @Xuanwo in https://github.com/icelake-io/icelake/pull/18
feat: Add in-memory types for table metadata by @Xuanwo in https://github.com/icelake-io/icelake/pull/19
feat: Implement parse of table metadata by @Xuanwo in https://github.com/icelake-io/icelake/pull/20
feat: Load table metadata from storage by @Xuanwo in https://github.com/icelake-io/icelake/pull/21
ci: Add publish CI by @Xuanwo in https://github.com/icelake-io/icelake/pull/23
Bump to version 0.0.1 by @Xuanwo in https://github.com/icelake-io/icelake/pull/24

New Contributors

@Xuanwo made their first contribution in https://github.com/icelake-io/icelake/pull/2

Full Changelog: https://github.com/icelake-io/icelake/commits/v0.0.1