Amazon S3 Tar Tool Versions Save

refactor small files and error handling/printing

Fixed a bug that would terminate the program before it could reach the cleanup function.

Updates to retry logic

Added a --max-attempts flag to specify the number of max attempts the AWS Go SDK will retry AWS Operations. Default is 10, zero is unlimited.

• introduced --size-limit flag, it breaks up Tar's by the size limit, for example if the user wants 1GB tars.
• customer user-agent, helpful for S3 Server Access Logs
• refactor

• Created a flag to specify the storage class the final object will have
• Created an API that calls the older methods. This will simplify adding test cases so we can mock easier. Might deprecate the older methods in a new version, TBD
• Fixed a manifest bug where it would break if the final tar was in another bucket

• upgraded aws-sdk-go-v2 since they merged my pull request #2083 to fix the NXDOMAIN issue
• fixed bug that prevented the use of a manifest
• added tests to main (need to expand to cover the other actions)

Fixed a bug that would keep retrying when a user entered the wrong region. This should fail immediately with a 'no such host' error

Added the ability to generate a toc for an existing tarball. The previous archive must not have compression. for now the tar must be in the local system to generate the toc.

command:

s3tar --generate-toc -f my-previous-archive.tar -C /home/user/my-previous-archive.toc.csv

Then upload the tar and the toc to Amazon S3. Once the tar is in Amazon S3 you can extract it by passing the --external-toc flag with the location:

s3tar --region us-west-2
--external-toc s3://bucket/prefix/my-previous-archive.toc.csv
-xvf s3://bucket/prefix/my-previous-archive.tar
-C s3://bucket/prefix-dest/

This release introduces a breaking change in List, pin your versions if using the Go code.

Fixed an issue when users wanted to tar a whole bucket, the empty prefix was causing issues.

added the ability to generate a toc for an existing tarball. The previous archive must not have compression. for now the tar must be in the local system to generate the toc.

command: s3tar --generate-toc -f my-previous-archive.tar -C /home/user/my-previous-archive.toc.csv

Then upload the tar and the toc to Amazon S3. Once the tar is in Amazon S3 you can extract it by passing the --external-toc flag with the location:

s3tar --region us-west-2
--external-toc s3://bucket/prefix/my-previous-archive.toc.csv
-xvf s3://bucket/prefix/my-previous-archive.tar
-C s3://bucket/prefix-dest/

This release introduces a breaking change in List, pin your versions if using the Go code.