A fast, auto-optimizing image server designed for high throughput and caching; Now that is hot.
This version adds some more in-depth timing metrics around the image upload system, and also adds concurrent uploads to the primary upload endpoint.
Lust 2 is out! After several months of mostly just bug fixes and small bits of maintenance, I have completely reworked the system and it's better than ever!
The config system has been completely reworked to create a more readable setup for developers and allows for more fine-tuning of how buckets are served and how the general global handler behaves. You can see an example of some of the new config here which I think is a massive improvement overall.
Backends have been completely reworked and the SQL and Redis backends have been removed.
This decision was made after running lust for several months and largely coming to the conclusion that they struggle to serve the image blobs as well as their wide column DB and blob store counterparts, it was also a lot of work maintaining them so, for now, I've made the decision to remove them. That being said adding a new backend is just the case of adding a new backend that implements the StorageBackend trait in the backends folder of the repo.
S3 compatible blob storage added! This probably should have been added sooner but here it is anyway. This allows you to turn any blob store (s3 compatible) into your own CDN system potentially, isn't that cool!?
Scylla DB backend: To be precise this does include Cassandra in its support bubble however, this is optimised for Scylla and even on a small machine 6 core machine I was able to serve around 12,000 images a second without caching, which I believe you could push even more with enough tuning.
Caching has been updated so now you can set both a local bucket cache and also a global cache allowing you to fine-tune what proportion of things get cached across buckets but also makes it simple if you just want a quick setup.
You can also now limit by memory usage! This allows you to set a limit in Mega bytes and the system will aim to cache up to that many images' worth of data. Note: This is an approximation system, please do not use it as a hard cap as the system does not control how much the allocator may return rather than the minimum amount of bytes required
I've moved Lust's backend over to poem-openapi (my beloved) which allows us to produce full documented OpenAPI schemas and intern documentation which you can find being served at the /ui endpoint of the server.
This should be your main reference as it will be kept in line with each version change.
A lot's changed between the time I first started Lust and now, and after 12 months of additional learning around the performance areas of Rust I've been able to massively increase Lust's throughput by around 10x in some tests, although your milage will vary depending on your setup, so I advise you do your own benchmarking to see 😄
There are now unit tests, this should hopefully help maintain the code quality but also prevent any accidental breaking changes from being brought in.
Webp encoders will no longer panic, this will be a welcome change to most people who have experienced Lust panicking with webp encoding if libwebp returns a bad code. (Man I hate C)
New logo 🎉
Resizing now maintains aspect ratio
Selectable resizing filter algorithms, the default is still the nearest neighbour (same as lust v1) so this shouldn't cause any unexpected difference in images produced.
Entity listing endpoints are gone - This was a massive pain to maintain compatibility with backends and also was a massive performance bottleneck.
Each sizing preset is tagged with an ID which intern allows for deterministic storage paths. Useful now that file listing endpoints no longer exist.
contains fixed docker images
This fixes docker images panicking due to the logger not being able to detect a time zone.
This fixes the build issues due to some cache handler changes.
This version bump fixes the WebP in-backward compatible changes and fixes unreachable panics in WebP lib by converting un-supported types.
This version adds the ability to see how long exactly each format takes to convert and allows parallel execution of format conversion, this should see significant performance improvements for multi-format conversion, if you have a single format setup this wont make any difference really.
This update brings some heavy optimizations to allow servers to make use of all their cores when encoding WebP, during testing, it was found that libwebp using its default single-threaded system was up to 10x slower than the png and JPEG encoders sometimes taking upwards of 800ms to encode a single 200KB image.
This update brings a custom encoder that allows you to customize the WebPencoding for better finer grain control and most notably offers multi-threaded encoding which offers significant performance increases on lower frequency but larger core count servers.
webp_quality Between 0 and 100, For lossy, 0 gives the smallest size and 100 the largest. If using lossless this does nothing (same as before).webp_compression with lossless encoding is the ratio of compression to speed. If using lossy encoding this does nothing - (float: 0.0 - 100.0 inclusive)webp_method The quality/speed trade-off (0=fast, 6=slower-better)webp_threading A bool signaling if multi-threading encoding should be attempted (false=disabled, true=enabled)Adds support for Redis-backed storage.