Bita Versions Save

This release focus on moving functionality from the cli tool (bita) to the library (bitar). Also tidy up the library interface, mainly around clone related functionality. This to simplify for 3rd party applications to integrate bitar and also allow for more code reuse between tool(s) and library (see related issue #13).

bitar

• Allow for any AsyncSeek+AsyncWrite as clone output
• Allow for any AsyncSeek+AsyncRead as archive reader
• Allow for in-place update of any AsyncSeek+AsyncWrite+AsyncRead type
• Add examples for how to implement archive cloning

Fix building without default-tls backend. To build with only rustls as tls backend use: cargo build --no-default-features --features rustls-tls

• Make number of chunks simultaneously processed dependent on the number of CPU cores available. Should result in lower memory usage on less performant systems. Can be overridden by cli option --buffered-chunks <COUNT>.
• Handle http redirects, failed previously.
• Use the system native tls as default. Rustls can be selected by feature flag (rustls-tls).

• The clone command can now use the output file as seed (using --seed-output). If so the output file is first scanned for chunks matching the archive, then found chunks are reorganized in place in the output file. After this other given seeds are processed and finally, if any chunks are still missing, they are now fetched from remote. This also let us avoid write operations when a chunk is already in place in the output.
• Minor chunking performance improvements and various clean ups

• New default rolling hash, RollSum, based on bup's implementation
• Add support for fixed size chunking, can be used instead of rolling hash
• New sub-command diff for displaying difference/similarities in data between two files
• clone sub-command improvements:
  • Post-verify the output file by checksum using the new --verify-output parameter
  • On http/connection errors bita can now retry and continue download without losing its progress
    • --http-retry-count - Set number of retries on http error
    • --http-retry-delay - Set time to wait between retries
    • --http-timeout - Set http transfer/connection timeout
• Simplified build by removing the last c-dependencies (libcurl and openssl) and replaced with native rust crates (hyper, rustls)

• New default chunk compression: brotli Lzma and zstd has been placed behind feature flags and will be deprecated.
• Now builds and runs on Windows
• Fix incorrect use of compression when the compressed chunk got the exact size of the uncompressed chunk.

• Add clone cli option --verify-header for checking the archive header integrity before a clone