Amazon S3 Tar Tool Versions Save

The storage class for concat-in-memory is working again

Full Changelog: https://github.com/awslabs/amazon-s3-tar-tool/compare/v1.0.22...v1.0.23

Added the ability to tag the final object created. This is useful to apply bucket policies that can move the object to the desired storage class. This helps reduce costs, doing a multipart upload to the desired StorageClass has a higher cost.

The syntax for tagging is the same as the AWS CLI

--tagging '{"TagSet": [{ "Key": "to-storage-class", "Value": "GDA" }]}'

Full Changelog: https://github.com/awslabs/amazon-s3-tar-tool/compare/v1.0.21...v1.0.22

Fix a logging verbose issue that must have been introduced when updating cli.v2

Full Changelog: https://github.com/awslabs/amazon-s3-tar-tool/compare/v1.0.19...v1.0.21

If you have aws cli installed on your system you can pass the same profiles using the --profile flag.

Full Changelog: https://github.com/awslabs/amazon-s3-tar-tool/compare/v1.0.18...v1.0.19

Created a new --max-part-size that allows for users to limit the part size of the MPU. This is for concat in memory mode. If a object is larger than the specified value, this object will be the whole part.

Added option to urldecode filenames that were url encoded by S3 Inventory

Fixed the new concat-in-memory to respect the Storage Class

This update introduces in-memory tarball generation. The goal is to optimize s3tar's performance for small objects. The new --concat-in-memory flag optimizes the process, altering the core design to download and build MultiPart objects in-memory when enabled. This modification brings about a notable improvement in both performance and pricing, prioritizing GET requests over PUT requests. Importantly, the default behavior of s3tar remains unaffected to ensure continuity with existing workflows. Users that are creating tarballs of extensive small files, numbering in the hundreds of thousands or millions, are recommended to leverage the --concat-in-memory flag for enhanced efficiency.

refactor small files and error handling/printing

Fixed a bug that would terminate the program before it could reach the cleanup function.