Opendal Versions Save

Apache OpenDAL: access data freely.

v0.45.1

2 months ago

Pacakages

Name Version
core 0.45.1
bin/oay 0.41.1
bin/oli 0.41.1
bin/ofs 0.0.2
bindings/python 0.45.1
bindings/nodejs 0.45.1
bindings/c 0.44.3
bindings/zig 0.0.0
bindings/dotnet 0.1.1
bindings/haskell 0.44.3
bindings/java 0.45.1
bindings/lua 0.1.1
bindings/ruby 0.1.1
bindings/swift 0.0.0
bindings/ocaml 0.0.0
bindings/php 0.1.1
bindings/cpp 0.44.3
bindings/go 0.0.0
integrations/object_store 0.43.0
integrations/dav-server 0.0.1

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Full Changelog: https://github.com/apache/opendal/compare/v0.45.0...v0.45.1

v0.45.0

3 months ago

Upgrade to v0.45

Core

Public API

BlockingLayer is not enabled by default

To further enhance the optionality of tokio, we have introduced a new feature called layers-blocking. The default usage of the blocking layer has been disabled. To utilize the BlockingLayer, please enable the layers-blocking feature.

TimeoutLayer deprecated with_speed

The with_speed API has been deprecated. Please use with_io_timeout instead.

Raw API

No raw API changes.

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Full Changelog: https://github.com/apache/opendal/compare/v0.44.2...v0.45.0

v0.44.2

3 months ago

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Full Changelog: https://github.com/apache/opendal/compare/v0.44.1...v0.44.2

v0.44.1

4 months ago

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.44.0...v0.44.1

v0.44.0

4 months ago

Upgrade Note

Rust core

Public API

Moka Service Configuration

  • The thread_pool_enabled option has been removed.

List Prefix Supported

After RFC: List Prefix landed, we have changed the behavior of list a path without /. OpenDAL used to return NotADirectory error, but now we will return the list of entries that start with given prefix instead.

Nodejs binding

Public API

Now, the list operation returns Array<Entry> instead of a lister. Also, we removed scan, you can use list('some/path', {recursive: true})/listSync('some/path', {recursive: true}) instead of scan('some/path')/scanSync('some/path').

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.43.0...v0.44.0

v0.43.0

5 months ago

Upgrade Note

Rust Core

Public API

List Recursive

After RFC-3526: List Recursive landed, we have changed the list API to accept recursive instead of delimiter:

Users will need to change the following usage:

  • op.list_with(path).delimiter("") -> op.list_with(path).recursive(true)
  • op.list_with(path).delimiter("/") -> op.list_with(path).recursive(false)

delimiter other than "" and "/" is not supported anymore.

Stat a dir path

After RFC: List Prefix landed, we have changed the behavior of stat a dir path:

Here are the behavior list:

Case Path Result
stat existing dir abc/ Metadata with dir mode
stat existing file abc/def_file Metadata with file mode
stat dir without / abc/def_dir Error NotFound or metadata with dir mode
stat file with / abc/def_file/ Error NotFound
stat not existing path xyz Error NotFound

Services like s3, azblob can handle stat("abc/") correctly by check if there are objects with prefix abc/.

Raw API

Lister Align

We changed our internal lister implementation to align with the list public API for better performance and readability.

  • trait Page => List
  • struct Pager => Lister
  • trait BlockingPage => BlockingList
  • struct BlockingPager => BlockingLister

Every call to next will return an entry instead a page of entries. Also, we changed our async list api into poll based instead of async_trait.

Java binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

Node.js binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

Python binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

C binding

There are no API changes.

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.42.0...v0.43.0

v0.42.0

6 months ago

Upgrade Note

Rust Core

MSRV Changed

OpenDAL bumps it's MSRV to 1.67.0.

S3 Service Configuration

  • The enable_exact_buf_write option has been deprecated and is superseded by BufferedWriter, introduced in version 0.40.

Oss Service Configuration

  • The write_min_size option has been deprecated and replaced by BufferedWriter, also introduced in version 0.40.
  • A new setting, allow_anonymous, has been added. Since v0.41, OSS will now return an error if credential loading fails. Enabling allow_anonymous to fallback to request without credentials.

Ghac Service Configuration

  • The enable_create_simulation option has been removed. We add this option to allow ghac simulate create empty file, but it's could result in unexpected behavior when users create a file with content length 1. So we remove it.

Wasabi Service Removed

wasabi service native support has been removed. Users who want to access wasabi can use our s3 service instead.

Java binding

There are no API changes.

Node.js binding

There are no API changes.

Python binding

Breaking change for layers

Operator and BlockingOperator won't accept layers anymore. Instead, we provide a layer API:

op = opendal.Operator("memory").layer(opendal.layers.RetryLayer())

We removed not used layers ConcurrentLimitLayer and ImmutableIndexLayer along with this change.

File and AsyncFile

OpenDAL removes Reader and AsyncReader classes, instead, we provide file-like object File and AsyncFile.

Open a file for reading in blocking way:

with op.open(filename, "rb") as r:
    content = r.read()

Open a file for reading in async way:

async with await op.open(filename, "rb") as r:
    content = await r.read()

Breaking change for Errors

We remove the old error classes and provide a couple of Exception based class for the error handling.

  1. opendal.Error is based class for all the exceptions now.
  2. opendal.exceptions.Unexpected is added.
  3. opendal.exceptions.Unsupported is added.
  4. opendal.exceptions.ConfigInvalid is added.
  5. opendal.exceptions.NotFound is added.
  6. opendal.exceptions.PermissionDenied is added.
  7. opendal.exceptions.IsADirectory is added.
  8. opendal.exceptions.NotADirectory is added.
  9. opendal.exceptions.AlreadyExists is added.
  10. opendal.exceptions.IsSameFile is added.
  11. opendal.exceptions.ConditionNotMatch is added.
  12. opendal.exceptions.ContentTruncated is added.
  13. opendal.exceptions.ContentIncomplete is added.
  14. opendal.exceptions.InvalidInput is added.

C binding

The naming convention for C binding has been altered.

Struct Naming

Renaming certain struct names for consistency.

  • opendal_operator_ptr => opendal_operator
  • opendal_blocking_lister => opendal_lister
  • opendal_list_entry => opendal_entry

API Naming

We've eliminated the blocking_ prefix from our API because C binding doesn't currently support async. In the future, we plan to introduce APIs such as opendal_operator_async_write.

  • opendal_operator_blocking_write => opendal_operator_write
  • opendal_operator_blocking_read => opendal_operator_read

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.41.0...v0.42.0

v0.41.0

7 months ago

Upgrade Note

Rust Core

There are no public API and raw API changes.

Java binding

Breaking change for constructing operators

PR-3166 changes the API for constructing operators:

Previous:

new BlockingOperator(scheme, config);
new Operator(scheme, config);

Current:

BlockingOperator.of(scheme, config);
Operator.of(scheme, config);

Now, there is no public constructor for operators, but only factory methods. In this way, the APIs are free to do arbitrary verifications and preparations before constructing operators.

Node.js binding

There are no API changes.

Python binding

There are no API changes.

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.40.0...v0.41.0

v0.40.0

7 months ago

Checkout our OwO #1 to know more about this release!

Upgrade Note

Public API

RFC-2578 Merge Append Into Write

RFC-2578 merges append into write and removes append API.

  • For writing a file at once, please use op.write() for convenience.
  • For appending a file, please use op.write_with().append(true) instead of op.append().

The same rule applies to writer() and writer_with().

RFC-2774 Lister API

RFC-2774 proposes a new lister API to replace current list and scan. And we add a new API list to return entries directly.

  • For listing a directory at once, please use list() for convenience.
  • For listing a directory recursively, please use list_with().delimiter("") or lister_with().delimiter("") instead of scan().
  • For listing in streaming, please use lister() or lister_with() instead.

RFC-2779 List With Metakey

RFC-2779 proposes a new op.list_with().metakey() API to allow list with metakey and removes op.metadata(&entry) API.

Please use op.list_with().metakey() instead of op.metadata(&entry), for example:

// Before
let entries: Vec<Entry> = op.list("dir/").await?;
for entry in entris {
  let meta = op.metadata(&entry, Metakey::ContentLength | Metakey::ContentType).await?;
  println!("{} {}", entry.name(), entry.metadata().content_length());
}

// After
let entries: Vec<Entry> = op
  .list_with("dir/")
  .metakey(Metakey::ContentLength | Metakey::ContentType).await?;
for entry in entris {
  println!("{} {}", entry.name(), entry.metadata().content_length());
}

RFC-2852: Native Capability

RFC-2852 proposes new native_capability and full_capability API to allow users to check if the underlying service supports a capability natively.

  • native_capability returns true if the capability is supported natively.
  • full_capability returns true if the capability is supported, maybe via a layer.

Most of time, you can use full_capability to replace capability call. But if to check if the capability is supported natively for better performance design, please use native_capability instead.

Buffered Writer

OpenDAL v0.40 added buffered writer support!

Users don't need to specify the content_length() for writer anymore!

- let mut w = op.writer_with("path/to/file").content_length(1024).await?;
+ let mut w = op.writer_with("path/to/file").await?;

Users can specify the buffer() to control the size we call underlying storage:

let mut w = op.writer_with("path/to/file").buffer(8 * 1024 * 1024).await?;

If buffer is not specified, we will call underlying storage everytime we call write. Otherwise, we will make sure to call underlying storage when buffer is full or close is called.

Raw API

RFC-3017 Remove Write Copy From

RFC-3017 removes copy_from API from the oio::Write trait. Users who implements services and layers by hand should remove this API.

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.39.0...v0.40.0

v0.39.0

9 months ago

Upgrade to v0.39

Public API

Service S3 Role Arn Behavior

In PR #2687, OpenDAL changed the behavior when role_arn has been specified.

OpenDAL used to override role_arn simply. But since this version, OpenDAL will make sure to use assume_role with specified role_arn and external_id (if supplied).

RetryLayer supports RetryInterceptor

In PR #2666, RetryLayer supports RetryInterceptor. To implement this change, RetryLayer changed it's in-memory layout by adding a new generic parameter I to RetryLayer<I>.

Users who stores RetryLayer in struct or enum will need to change the type if they don't want to use default behavior.

Raw API

In PR #2698, OpenDAL re-org the internal structure of opendal::raw::oio and changed some APIs name.

What's Changed

Added

Changed

Fixed

Docs

CI

Chore

New Contributors

Full Changelog: https://github.com/apache/incubator-opendal/compare/v0.38.1...v0.39.0