LANraragi release jumpscare! awooooooooooooo

Happy Halloween! 🎃

I wanted to hold off this release until it had a big feature deserving of the major version update, but since it's been so long... A terrifyingly huge pile of QoL stuff has built up in the meantime! I think you'll find something to enjoy in here.

There's also been some community activity recently, including:

• A fresh Nix package! Consider checking it out if you're a reproducible builds enjoyer™️.
• A new react-based PWA client!

It's not a spooky release without some cold sweat, right? So here's a:

🛑🪟 WARNING FOR OLD WINDOWS VERSIONS 🛑🪟

This new update jumps the Windows Linux distro by quite a few software versions (See "Add AVIF/JPEGXL support to Windows" below), including non-reversible updates to Redis 7 for the database.

I've received spare reports of large databases on WSL1 breaking with the new Redis version and being unable to save changes, so please make a backup of your database folders before updating in case you need to roll back!
In case you updated anyway and hosed yourself, WSL2 should work and allow you to access the app for backup.json generation.

---

All set? Let's get to the actual changes:

🧛 (#789) "Save archive title" is now a global preference for all plugins

With this change, plugins are no longer responsible for handling whether they should update the Archive title or not; They can always just hand off a title suggestion, and it'll only be applied if you enabled this global preference in Plugin Configuration.

It defaults to ON, so you should very much make sure all your plugin settings are set correctly upon updating.

🍬 Archive overlay improvements

The reader overlay now features a few additional shortcuts if you're logged in, including a Delete button and a quick Category UI so you can see at a glance which Categories a file belongs to, and add/remove them as you please.

It now also shows the Archive name on top!

🦇 Add AVIF/JPEGXL support to Windows builds (#626 / #792 / #805)

Through the arcane magicks of... Alpine Linux updates!

The Horror!

Windows was previously stuck on Alpine 3.12 due to musl's DNS changes breaking downloader support -- As a result they were missing out on recent ImageMagick updates, including the stuff needed for avif/jxl.

3.18 fixed the mess, so I'm quite pleased to finally be rid of dual-legacy Docker image building for the time being.
^{(the legacy Dockerfile will remain maintained for a bit longer for the poor chaps stuck on aging Synology hardware... update already lads I'm all for reducing ewaste but surely there's a more up to date community linux or something)}

🍡 JPEGXL thumbnail support (#825 - @polak14)

JPEGXLBROS WE'RE SO BACK _{unless you're using Chrome because Google decided your meme format didn't have a future}

This adds an optional toggle in settings to use JPEGXL instead of JPEG for thumbnail generation. Your existing thumbnails won't be regenerated however, so you should use the "Regenerate all thumbnails" option if you want to switch everything to JXL.

JXL brings slight size improvements, and has less artifacting if you're using high quality thumbnails:

du --apparent-size  
1404    ./jpg sample (HQ thumbnails)
1152    ./jxl sample 
21.88% diff

1180    ./jpg scale (LQ thumbnails)
906     ./jxl scale
30.24% diff

Considering the wins, I might use JXL in "Resize images in Reader" later down the line?
I guess browser support would have to get a bit closer(doesn't even work in Firefox stable smh my head), but with Apple supporting the format it'll probably happen at some point.

💀 Plugin for Koushoku/Anchira YAML files (#833 - @siliconfeces)

This plugin adds support for the info.yaml files you can find embedded in files coming from KSK(rip) or Anchira.
^{We had a full KSK plugin set with downloader and everything, but the website died before we could ship it lmao}

This also adds a YAML parsing library(YAML::Syck) to the stock codebase modules, so writing future plugins that parse yaml should be easier.

🧟 LRR now supports OPDS-PSE 1.2 (#791 - @kuroneko995)

OPDS Page Streaming Extension 1.2 adds support for tracking the last time reading progress was updated, so that clients can either use the server progress or their own.

As with everything OPDS, this is quite niche and only Panels for iOS supports it... But as a bonus for API clients, LRR archive objects now expose a lastreadtime value you can use.

👻 Tankoubons are (almost) here! (#519 / #818 - @EfronC)

This update introduces a new way to classify your files, Tankoubons.

Much like Categories, Tanks allow you to regroup multiple archive IDs...
Except that unlike Categories, Tanks have a specific order and will hide their contents from the Archive list and search, effectively allowing you to only show one entry for multiple files, with chapters. Sounds pretty neat! Except...it's not actually finished! 🫠

0.9.0 only features the API endpoints for tankoubons, allowing API clients to get a headstart on implementing adding/removing archives to/from tanks. The web UI and implementation in search will come in a future release.

🎃 Other changes

• (#864) Make stats page faster when file access is slow (@siliconfeces)

  📈 Historically, the stats page used to do filesystem operations to count your total archive count and content folder size. 📉 This could be slow if you were using software forbidden by god cloud-based filesystems. 📊 Stats now exclusively use the database to compute their information - It might not be filesystem-accurate at all times as a result, but with time I've changed my mind on this and think it's probably better to be database-accurate so we're consistent.

• Dev environment changes (@siliconfeces)

  💎 The Redis hostname can now be overridden by the environment variable LRR_REDIS_ADDRESS 🐳 A docker-compose file has been added and documented if you want to setup an environment where the Redis database is in a separate container. it's the docker way™️....

• (#811) Fix broken encoding for titles (@siliconfeces)

  🀄 get_title didn't handle text encoding properly, which caused non-trivial characters getting mangled in the "add to/remove from category" toasts from last release.

• (#589) Update themes to use webkit-line-clamp

  〰️ This adds basic support for two-line titles in thumbnail view. 🏬 If one wants unlimited lines, it's now as easy as removing -webkit-line-clamp: 2; from div.id2 a. Use table view if you want more tbh

• (#840) Implement title similarity scanning for the Hentag plugin (@siliconfeces)

  🔎 This plugin will now try to find the most similar title to your archive's, instead of relying on exact matches.
  📜 This has been added to a core function (LANraragi::Utils::String::most_similar($title_hint, @titles)), so feel free to reuse this for your own plugins!

• (#866 - #823) Remove loading='lazy' from Reader (@siliconfeces / @URenko)

  🐌 Lazyloading images was pretty much broken in different ways in both FF and Chrome, so it's been removed for now.

• (#874) Add missing package-lock.json file and switch to npm ci

  🏗️ This enables easily reproducible builds. I don't know how those newfangled npm things work, okay? 📦 I'm not quite sure the builds are perfectly reproducible considering cpan installs can use different/later versions of Perl packages depending on the options you give it...ah well

• Update secret generation so it doesn't only rely on machine hostname

  🧙 The mojo secret is used to authenticate login cookies - Using the hostname alone was a bit weak, so it's now coupled with a randomly-generated string.
  ㊙️ The secret string is generated and stored in the app root folder -- If LRR_DATA_DIRECTORY is declared however, it'll be stored there.

• Miscellaneous bugfixes and improvements

  💎 (#769) Add missing search cache invalidation to set_isnew
  💎 (#788) Add some more information to the epub warning 💎 (#792) Eval thumbnail minion job and fail it properly if thumbnail fails to gen 💎 (#798) Don't add non-ascii artists to EH URL searches 💎 (#807) Use File::Temp rather than homebrew stuff (@siliconfeces / Additional fixes in #827 - @polak14) 💎 (#812) Fix "Uncaught TypeError" when attempting to navigate past last page in infinite scroll mode (@siliconfeces) 💎 (#822) avoid await at top level (@URenko) 💎 (#824) add magazines to koromo (@polak14) 💎 (#834) Fix Batch tagging overrides not accepting utf8 text 💎 (#856) Make permission fixing optional (but default) (@siliconfeces) 💎 (#868) Reuse generate_archive_list when possible to simplify codebase (@siliconfeces) 💎 (#871) Remove duplicate tags when updating DB (@siliconfeces) 💎 (#872) ajax cache to reduce server load (@ndbiaw) 💎 (#878) Fix Redis password authentication (@TomaSajt) 💎 Add trace-level logging to backup generation 💎 make sure cpan packages are installed before running tests 💎 Fix LOADING redis check being bypassed by settings migration 💎 Fix thumbnail extraction not accounting for covers properly when used from the API 💎 Add a script for npm run get_version because perl oneliners are hell to maintain 💎 Move appendonly set to redis.conf instead of app boot 💎 Delete BlacklistMigrate plugin as we're long past 0.8.0 now 💎 Handle error codes better with nH parsing 💎 Update to Mojolicious 9.34

• API Changes

  🔑 Archive objects now expose a lastreadtime value. 🔑 Updating progress will also update lastreadtime, and return the value in the JSON response. 🔑 /api/tankoubons endpoints are available and documented. 🔑 /api/archives/:id/tankoubons has been added to know which tankoubons a given archive ID belongs to.

Full Changelog: https://github.com/Difegue/LANraragi/compare/v.0.8.90...v.0.9.0

Conjured up in a handful of days in the studio, “Spaceboy” was a liberating track for Bowie, who rode its beat and reveled in the trash. This chaos is killing me! he screamed, sounding delighted to die, mocking his past selves with “do you like girls or boys? It’s confusing these days. [...] “Spaceboy,” one of the last great Bowie pop moments, never quite seemed his own property; it was fluid, a coalescing held together by a beat that seemed to invade it.”

Happy new year! We back on the grind.
I'm still mostly working on non-LRR things at the moment(and playing Pizza Tower, go play pizza tower) but for some reason there's been an insane influx of pull requests recently -- Is this the power of Holopin? Just look at all those contributor avatars down there yooooooooo

As such, this is a plugin-focused release, with a few other nice bonuses. Roll it in!

🛑 Important changes

• If you're running from source, database_config is now officially in use, and all your non-archive configuration keys will be moved to it the next time you start the server. As a reminder, this DB lives in Redis index 2 by default and can be changed in lrr.conf.
  • See the architecture documentation for more info.

🧩 A morbillion new plugins

Plugins are a nice and easy way to contribute since they're isolated from the rest of the app, so I tend to be more lenient with code review on those!

My outmost thanks go out to everyone who added support for their meme format/websites in this release:

• (#705) ComicInfo.xml support (@Gin-no-kami)

• (#595) hentag.com metadata fetching + info.json (@siliconfeces)

• (#749) More tags from Chaika(@kagoromo)

• Chaika api.json embedded file support (@siliconfeces)

• (#764) Handle multiple artists in koromo info.json files (@siliconfeces)

(ok I guess comicinfo.xml isn't really a meme format considering how prevalent it is)

I've started drafting out some design concepts for an eventual plugin git repository so I can be even more lenient with plugins in the future -- Maybe that'll be the big feature to land in 0.9.0.

Other changes

• (#586) Page count search has been added

  🔎 You can now search for galleries with a specific number of pages with pages:20, or with a page range: pages:>20, pages:<=30.
  🤔 This first version isn't indexed so it might run a bit slow - let me know how it runs for you!

• (#762) Increase search speed when sorting by non-title tags (@djacks6278)

  🏎️ As we keep carving away at the old shit search code, sorting should now be around 20x faster for non-title sorts.

• (#682) Progress tracking and "Set this page as thumbnail" now works with infinite scrolling

  ♾️ If you open the Reader in infinite scrolling mode, it'll also now automagically scroll to the page you last read.
  ⌛ Although due to the way infinite scrolling currently works, you might have to wait for all the images to be loaded for that scroll to happen...

• (#780) Category messages will now give you the name of the category/archive instead of the IDs (@siliconfeces)

  🗂️ heh

• (#759) Added an option in settings to overwrite existing archives when uploading a new one (@qhua948)

  ⬆️ This only applies to remote downloads and the 'add archive' page. 🛑 This will delete metadata for old files when they're replaced! Use with caution.

• (#742) Filter more credit pages & move covers to start (@Asinin3)

  🛺 The reader will now move the following pages to the end when reading: end_card_save_file, notes, artist_info, credit, 999nhnl 🆙 Likewise, images starting by cover will now be moved to the start by default.

• Miscellaneous bugfixes and improvements

  💎 (#686) Add top level subfolder option for Folder2Cat script 💎 (#731) Fix tooltips in the carousel (@pastalian) 💎 (#732) Fix full page refreshes occasionally breaking the carousel 💎 (#755) Fix resetting the database not triggering a search index rebuild 💎 (#760) Fix source: tags not being indexed for search 💎 (#752 / #778) Fix some old code that was spewing out warnings (@siliconfeces) 💎 Perl signatures are now allowed in the codebase 👍👍🆒 💎 Some documentation fixes (@GoGim1 , @ForsakenRei)

Full Changelog: https://github.com/Difegue/LANraragi/compare/v.0.8.81...v.0.8.90

“Hearts” began as a group improvisation in the Leon sessions (Garson started things off by playing a hook on piano, while Eno’s contribution was to loop a French radio broadcast and blast it every four bars) and it shows in the song’s structure, as “Hearts” can seem like a welded-together collection of pieces.

woops lol

This is a hotfix update for v.0.8.8 - I'd recommend checking the release notes for that if you haven't already.
I swear this is the last 2022 release for real trust me bro

• Miscellaneous bugfixes and improvements

  💎 Fixed OPDS API outputting invalid XML and not formatting its timestamps to the expected format 💎 Fixed newly added archives not being in the search index unless at least one plugin was ran 💎 Fixed Linux::Inotify2 not installing anymore and leading the filewatcher to use its default mode 💎 Updates Linux::Inotify2 to 2.3 on non-Windows versions 💎 Fix being unable to search with non-Latin1 characters

Full Changelog: https://github.com/Difegue/LANraragi/compare/v.0.8.8...v.0.8.81

The track’s harmonic stasis and ominous mood better suited the sequence David Lynch used it for on Lost Highway—scoring a driver’s-eye shot of a sped-up stream of highway center lines, a loop of ceaseless, violent motion. “Deranged” also improved in concert, once the song was prised loose from its album mix and given fresh, bloody life by the Outside and Earthling tour bands.

Happy Holidays from your favorite degeneracy library!
I am bringing you the long-awaited gift of indexes to cap off the year.

🛑 Important changes

If you don't want to read the rest of the changelog:

• If you're running from source, LRR now requires two supplementary Redis databases, using indexes 2 and 3 by default. You can change those numbers in lrr.conf. (Only database_search is used for now, config will come into use next release)
• If you update a large instance, search will not work for a bit while the indexes are built -- You can follow the status of the stat_indexes Minion job to see when you can start searching again.
• The /api/archives/random endpoint has slightly changed, which means third-party clients might have broken random functionality until you update.
• OPDS-PSE support is in, but won't work for archives that don't use JPEG for their images.

🔍 Search has been rewritten (#555, #444, #563, #657 and more!)

Search has always been the biggest performance bottleneck of the entire server due to using a rather dumb algorithm that always did multiple database passes... Through the incredible invention of search indexes ^{but for redis}, I'm happy to announce that search is now up to 10 times faster for the most used scenarios.

https://user-images.githubusercontent.com/8237712/208326844-4534508b-0b66-4fe5-a9bf-7bc15b61a7fa.mp4

Some notes on the search rewrite:

• Spaces are no longer a delimiter for different tags in a search prompt, commas should now be used instead. This matches what the autocompletion box does and how metadata as a whole is entered into the app, so I don't think this will cause too much pain.
• Search now requires the index tables to be built (does so on app start, and then updates them as tags/titles/etc are modified), so large instances might take a few mins to get a usable search again after updating.
• Title searches will work the same as before (quotes or $ for an exact match, ? * _ % as wildcards. )
• For tags, if you don't use exact matches, tag searches will be wildcarded on both ends (like previous behavior), namespace:tag searches will only be wildcarded at the end (namespace stays intact). You can use *namespace:tag if you want the old behavior for some weird reason.

This was one of the biggest jobs on the list, and while it took a few days to get it right I hope you'll enjoy it -- I do think it's quite a gamechanger.
This rewrite doesn't optimize everything (large intersections and sorting by anything else than title will still be a bit slow), so there's certainly still room for improvement.

The new system makes heavy use of Redis sets and sorted sets, which are all lodged in a separate, third database.
See the DB Architecture document. (the config keys moving to a fourth database is a pending change that will land in a future release.)

📚 OPDS PSE Compliance

I've dusted off the OPDS API to make it use the new Search, and took the opportunity to implement a few niceties.

LANraragi now supports OPDS PSE 1.1, which means you can use compatible OPDS clients to read archives page-by-page from the server and update your reading progression. (If server-side progress tracking is enabled)
As a known caveat, OPDS-PSE will currently not work if your archive contains non-JPEG image files.

Considering the surprisingly complete state of third-party LRR clients, I don't think many people use OPDS at all, but it's nice to be a good citizen to standards nonetheless.
And if you can't sideload DuReader, that means you can now use Panels on iOS! At least until Apple finally caves in and allows sideloading on their devices. ^{god bless the EU}

Other changes

• (#646) New EH layout compatibility

  🐼 Both ex and e-h have recently updated their layout and search API - While the old plugin still works fine on the new layouts, it still got a small update to benefit from the changes.
  🔍 You can now search by gID -- this will fall back to title name if no gId found or search failed. (Thanks @polak14 !) 🍪 The igneous cookie is now supported, which fixes the problem of using ex in non European and American regions. (Thanks @zizzdog !)

• (#709) Carousel now auto-reloads with every new search + has a nicer empty view

  🎠 whee 🆕 I couldn't really afford to make the carousel auto-reload before since it relies on Search, which had...yeah.

• Add basic Redis password support

  💍 You can now set redis_password in lrr.conf in case your Redis server is password-protected.

• Windows fixes

  🪟 The WSL detection should now work regardless of your system language. 🛠️ The Windows app should now detect "file not found" WSL failures better. 🔧 The repair button is also now always available from Settings.

• Miscellaneous bugfixes and improvements

  💎 Login now redirects you to the page you tried to access beforehand 💎 Update FontAwesome to v6 💎 Update Mojolicious to v9.30 💎 (#396) Allow most common http methods when CORS is enabled 💎 Fix not being able to redownload an URL after having deleted it from the server without a restart

• API Changes

  🔑 /api/search/random now uses the same object syntax as /api/search (arcid instead of id, newonly instead of isnew) 🔑 ?key= can be used again to authenticate API requests. (Mostly just for OPDS, please don't actually use this)

Full Changelog: https://github.com/Difegue/LANraragi/compare/v.0.8.7...v.0.8.8

Inspired by his and Eno’s trip to the Gugging Psychiatric Clinic in 1994, [...] “Deranged” seems mainly Eno’s work, though one ancestor was Bowie and Nile Rodgers’ “Real Cool World” (there’s also an echo of “Billie Jean” in its opening four-note synth hook)

https://user-images.githubusercontent.com/8237712/195881965-116f40c0-7800-41f3-9c00-8c255e2a676c.mp4

If you can believe it, it's a bugfix update once again!
This is a small update to mostly fix the Windows build, but I've had enough time to throw in some extra bonuses as well.
It's a busy end of the year so LRR is probably going to stay on the backburner for the time being; Although as always, if you want to pitch in a PR for Hacktoberfest, I'll gladly take the time to review it! 🧑‍💻

🪟 Windows installs have been fixed..again! (#673)

Not so much fixed as adapted for Windows 11; The WSL detection wasn't working properly and generating a ton of support requests as a result.
I've also fixed some leftover issues with DistroInstaller, so hopefully this is a smooth experience again now.

🐧 The Docker image has been updated to s6-overlay 3.x (#684)

Thanks to @Guerra24 for this!
This is mostly an internal change that won't affect endusers, but it took a bit of work to make that update so I thought I'd shout it out anyways. _{(totally not trying to make those release notes at least 50 lines long)}

I've said this back in 0.8.6 already but:
⚠️ If you're encountering issues while running on old Linux versions (seen it happen on Synology devices), I recommend you build a custom Docker image yourself using the legacy Dockerfile as a base.

🖼️ Add tentative JPEGXL support (#665)

Much like HEIF exactly 10 releases prior (wow!), jpegxl (.jxl) files are now recognized as valid images by the server.

If you want thumbnail support to work, you'll likely have to install libjxl-dev or similar on your Linux machine.
Said lib isn't out in any stable form of Debian yet, but I'm sure Arch users or similar can get it already.

Windows users don't have that library since it hasn't been backported to Alpine 3.12, so no JXL thumbnails for you! 🤷

Other changes

• (#672) Add category operation to batch tagging

  😎 You can now serial-add multiple archives to a category through a batch operation. Simple yet effective!
  🤔 The main category UI already allowed you to do this so I'm duplicating a bit, but it makes sense to have that in the batch UI as well.

• (#671) Update the filemap whenever an ID gets modified

  🔢 The ID update code from 0.8.6 overall worked well, but it was forgetting to update the internal filemap, which would lead to duplicates and other weirdness when running clean database operations.

• Miscellaneous bugfixes and improvements

  💎 (#678) Make URL downloads use unlimited response size 💎 step down to x86-64 level 1 for the ffi docker build so that my E350 CPU can still run the Docker image 💎 (#662) Add some extra filtering to F! search so it performs better (@polak14) 💎 (#698) Remove the maxlength parameter on tagrules

Full Changelog: https://github.com/Difegue/LANraragi/compare/v.0.8.6...v.0.8.7

"[Outside] is only symbolically anguished. I think we are in for a very good time when we get to the next millennium." -- Bowie, press conference, 1995.

Hej!
Here we are again with another update where I procrastinated on doing major improvements but external contributions piled up so I hafta make a release. What can I say?

🖼️ Very fast thumbnails generating at incredible hihg speed

Thumbnail generation code has been improved with this one simple trick (CPU vendors hate him!), which leads to some very noticeable gains on hi-res images as long as you're thumbnailing JPEG images (aka 95% of images for manga)

time convert problem.jpg -quality 50 -thumbnail 500x out.jpg
real    0m3.704s
user    0m3.077s
sys     0m2.052s

time convert -define jpeg:size=500x problem.jpg -quality 50 -thumbnail 500x out.jpg
real    0m0.685s
user    0m0.631s
sys     0m0.111s

Thumbnails for non-cover images will now also render at a lower quality to gain even more speed, as said thumbnails are generated at a much higher volume than covers. (Setting a random page as your thumbnail will still re-generate a HQ version of the thumbnail.)

And if you want to be all-hq-all-the-time, there's a new option you can use for that:

🪟 Windows installs have been fixed (#654)

0.8.5 was not kind to Windows users due to bugs in the way I integrated the DistroInstaller external tool, and the Alpine upgrade essentially breaking outbound network requests most of the time.
Windows builds will now use Alpine 3.12 again as a base -- If you're encountering the same issues while running on Linux (seen it happen on Synology devices), I recommend you build a custom Docker image yourself using the legacy Dockerfile as a base.

As a bonus, the Windows GUI tool will now try to be helpful in case your WSL distro is broken, instead of just telling you to reinstall like a scrub:

🐧 Linuxbrew installs work again!

One of the few advantages of having a package manager containing LRR is that I sometimes get surprise updates, such as restored Linux support! This is 0.8.5 only for now, but should be up to date again soon.

☁️ Cloudflare bypass for NH + New metadata plugin for Hitomi.la (#631)

The implementation of Cloudflare by NH breaks the plugin from 0.8.5, so courtesy of @Pheromir , there's now a built-in login plugin for NH that allows you to key in CF bypass cookies.

And if you don't want to use NH anymore _{because it's garbage} there's also a new alternative in the form of a hitomi.la metadata parser, thanks to @doublewelp !

🐳 Docker container improvements (#637 / #659)

To answer a long-standing request from users living in countries with nice and friendly governments ( :^) ), the LRR Docker container will now respect the HTTP_PROXY, http_proxy, HTTPS_PROXY, https_proxy, NO_PROXY and no_proxy environment variables, so you can have outbound network requests (metadata and downloads) go through a proxy of your choice. (Thanks @MistrPokr !)

And for reverse proxies, the container will now also pick up the X-Forwarded-For and X-Forwarded-Proto headers!
You can now also pass a TZ environment variable to the container to change its timezone. (Thanks @xiazeyu !)

Toasts and Dialogs have been updated

This is another big @xiazeyu change -- Toasts and dialogs across the app now use more modern libraries, which brings a few new features like visible expiration timers for toasts and themable dialogs.

Other changes

• (#646) Sort key/order is now remembered when you reload the Archive Index

  😎 Withdrawing essential features for years is a good way to drive up user engagement!

• (#652) Fix sorting issue where uppercase characters would always end up in front

  ♻️ Another bug for the wall of shame. I think this one's been in here for years at this point?
  🔍 To fix other sorting issues, files that contain "credit" are now always pushed to the end of the list. Know your place scanlator scum

• (#614) Fix the New Archive Carousel loading as many thumbnails as computationally possible (@xiazeyu)

  🎠 The carousel now uses virtualization, which will make it only load 10 or so thumbs at a time.

• (#644) Added "Synology eCryptFS Compatibility Mode)

  🔏 This is basically what it says on the tin -- eCryptFS limits number of characters for filenames to 143 bytes, and this mode enforces that.

• (#635) Changing a file externally will now update its ID

  🔢 The design for LRR hinges entirely on generating IDs out of hashes of your files, so actively modifying a file would lead to its ID changing, which would then cause multiple issues on the database side, where it'd keep the old ID. 🍩 Which worked fine for most cases, but could lead to data loss in some circumstances! I've updated the filewatcher to check for ID changes and update the DB accordingly.
  🧹 This check is also made when using the database cleanup function.

• (#651) Fix the first tag from a plugin run being undeletable in the edit metadata form

  🫠 This is because plugin runs have a leading space, which was then misinterpreted by tagger.js when we try and remove the tag -- effectively not deleting it from the backing textfield. Woops!

• Miscellaneous bugfixes and improvements

  💎 (#618) Fixed Fullscreen hooks not being removed if Escape is used to exit fullscreen instead of F 💎 (#645) Fix downloads keeping query parameters from their URL in the source: tag 💎 (#660) Fix thumbhashes not being restored from backup JSONs
  💎 (#624) Fix various garbled non-ascii character bugs in logging and plugin results (@uparrows) 💎 Fix dateadded toggle not hiding datemodified toggle in settings
  💎 Use jpeg:size for the page resizing feature in Reader as well to gain a bit more extra speed
  💎 (#630) Add logging for failed/successful logins to the admin panel to help with fail2ban scenarios 💎 (#655) Camelcase GitHub everywhere (it's not even hacktoberfest yet!)

Full Changelog: https://github.com/Difegue/LANraragi/compare/v.0.8.5...v.0.8.6

Bowie has called Buddha of Suburbia one of his favorite records. Maybe he said that as a bit of mischief, touting his most obscure record as one of his best, like a hipster connoisseur of his own work (and he was). But Buddha did seem to have resonance for Bowie; something about its creation had felt right with him.

Well, it's been a little while! I wanted to delay a new release until search improvements were done ^{but I haven't worked on them at all lmao}

There's been a sizeable number of bug fixes and external contributions however, so I think it's worth making a small release.
Thanks to everyone who's contributed to this one. 🙏

⚠️ Update your Tsukihi extension if you have it installed

Get 1.2 here!
API changes in this build will break download notifications unless you update.
Nothing too major since downloads themselves will still work. Thanks to @xiazeyu for saving me an hour of work by PR'ing a fix. 😅

The Windows installer now supports Windows 10 1809 again!

https://user-images.githubusercontent.com/8237712/165130010-f6668093-f591-4d9a-9937-839a89115729.mp4

(This video shows a work-in-progress MSIX port, but the regular MSI installer now uses the same tooling.)

I wasn't expecting so many people to still be on old-ass builds LTSC versions of Windows when I removed support for it to make the installer code cleaner, but thanks to @Guerra24's efforts, we now have a better infrastructure for the Windows helper side of things, which allows me to bring 1809 support back.

No more mucking about with an old 0.7.9 MSI 👏 (I'd like to thank the random individual who wrote that guide on 4chan to help fill the gap)
The Karen bootloader has also been updated to use more recent Windows APIs, with hopefully more to come.

The webreader now supports Full Screen mode!

The reader improvements simply don't stop coming even when I'm not working on it; Thanks to @djacks6278 , you can now use the reader in fullscreen: Both reading directions, as well as infinite scrolling and double-page work in this mode just fine.

He also finally killed my old-ass canvas-based code for double page view, so you might just get faster reading as a bonus now.

Changelog

• Double page load speedup & Fullscreen w/ mousewheel (#599)

  🎉 No more canvas: double-page views should load faster 🎊 Press F to go into fullscreen mode ⌨️ All keyboard shortcuts work as-is in fullscreen 🖱️ Mouse scrolling will either snap from page to page, or scroll smoothly if you're in infinite scrolling mode.

• Stop using File::Temp for dynamically resized files

  🖼️ Resized files were never deleted and took space in /tmp, which causes issues for Docker containers -- This moves resized files to the main LRR temp folder, whose autoclean routines should fix this issue.

• (#586) Calculate pagecounts for incoming archives

  🧮 Since the switch to the new libarchive bindings, it's possible to get the file count (and size) fast enough to allow me to finally know the pagecount of every archive as soon as they come in, instead of waiting for them to be extracted. 🕑 This won't apply retroactively since it'd end up super messy code-wise, but if you want to force it, you can use the brand new...

• (#590) Added a button to force rescan the archive directory

  🗺️ While the existing filemap mechanism in shinobu is fairly robust and tracks files in the content folder well, misses can sometimes be unavoidable, and until now the only solution was to hit the ol' redis CLI and delete the filemap yourself. 🦩 The button basically is just a shortcut to that, with a matching API endpoint.

• (#498) Support for external JSON metadata files (eze plugin only)

  🥲 The eze plugin is now able to pull metadata from JSON files located in your content folder, if they have the same name as your archive. 🥳 Thanks to @xiazeyu for this!

• (#601) Fix timestamp tags replacing existing tags instead of simply adding to them

  💦 If you were using the new built-in date_added functionality, the timestamp tag would overwrite whatever tags were saved previously.
  ⬇️ In practice, this resulted in source: tags from URL downloads being skipped over, since those were applied before the timestamp. Woops!

• API Changes

  🔑 Add a public API endpoint for getting (simplified) Minion job status You can now use /api/minion/:jobid to know if a job succeeded or failed, without any extra info (and therefore without the need for an API key.) 🔑 (#590) Add /api/shinobu/rescan for resetting the filemap

• Miscellaneous bugfixes and improvements

  💎 (#593) Fix eze plugin adding a space to source: tags 💎 (#587) Add lowercase info.json support to koromo plugin 💎 (#611) Wrap compute_id call in eval to handle null files in FolderToCat 💎 (#606) Add support for parsing timestamp, category, uploader tags to the eze plugin 💎 Fix opds not using date_added tags 💎 (#607) Handle schemes in source: tags properly 💎 Add support for source: tags to the nH plugin 💎 Update Docker container base to Alpine 3.14 💎 Update marked.js

Full Changelog: https://github.com/Difegue/LANraragi/compare/v.0.8.4...v.0.8.5

Bowie closed out “Sex and the Church” with a callback to glam (the rave-up ending of “Jean Genie”) and some moans, but the whole production had a cold, disassociated feel, the sound of a virtual reality sex program punched up by a Philip K. Dick character.

Happy New Year to all! May your manga collections be fruitful and your TempleOS installations blessed.

This release brings in more of the planned User Survey Requests features, namely everything related to Thumbnails.
Next release should be focused on tackling the BIG ONE, aka search speed.

ppc64 and s309x have been dropped from the Docker images

I don't think anyone ever pulled those architectures (since they're mainframe-oriented) and they were starting to take a toll on github build times (to the point I'd often hit the 6h limit for cacheless builds), so they're gone!

I apologize to all the PowerPC enjoyers.

Reader improvements New Year's Bundle

In this version, I'm finally adding in the feature that will make all panda pundits praise me and potentially pledge some pennies:
You can now directly show the thumbnails overlay when opening the Reader.

wow!

The thumbnail overlay previously had performance issues due to showing the full-size images; I've fixed this by extending the existing thumbnail APIs to be able to create/return thumbnails for all pages in an archive, and not only the cover.
This makes overall perf much nicer, althrough if you're hosting on a CPU that doesn't have many cores/threads, the thumbnails might take a while to appear on first reads now.

But as a result, you can now also set any page to be the cover thumbnail for an archive!
This replaces the old "regenerate thumbnail" button in the Reader, since you can just set page 1 for the same effect.

I've also improved back button functionality: The Reader no longer clutters browser history with an URL for each page, and going back using the return button will now bring you back to whatever search you'd performed before reading.

Changelog

• New Year's Reader Fun Pack™️

  🥳 (#514) Add an option to show the tags/thumbnail overlay first when opening the Reader 🎉 (#517) Add button to set the current page as the cover thumbnail 🎊 (#520) Use the new thumbnail API in the Reader overlay ✨ Remove pushstate/popstate functionality from Reader 😳 (#546) Use http referrer headers to be able to redirect the user to his previous search when leaving the Reader

• (#567) Hiding the carousel now sets a pref so it doesn't re-open on page reloads

  🧧 Response to the new carousel has been quite positive, but as with every new feature, some people just want to disable it. 😭 And that's perfectly fine! I forgot to make collapsing the carousel a saved change in the previous release, so it should be less egregious from now on.

• (#554) Fix PDFs not working if your Ghostscript install was too recent

  🧧 gs 9.50 marked the flags I was using to pull pdf pages as insecure, which required some command line updates. 🎍 This wouldn't have been too big an issue except I uh, shipped 9.50 in the Docker image most people use, so sorry about that.

• Update koromo plugin so it can also accept metadata files from [REDACTED] compilation torrents

  👻 gee I truly wonder what that mystery site could be

• (#565) Fix Tag Rules only accepting ASCII characters

  🥲 Another day, another disappointment. 🥳 And by disappointment I mean I fucked up encoding again! I have failed you once more and deeply apologize.

• Add option to use file modified time for the builtin date_added feature

  ⏱ I planned to remove the old date plugin after fully integrating it to the main app, but since the feature doesn't apply retroactively, I decided to leave it in for the time being, in case you want to batch tag dates for all your stuff.

• (#572) Fix gif thumbnails, which used to not work due to different file naming by imagemagick

  🎉 who the heck uses gif for their covers? ah dingus it doesn't matter there ya go fixed

• Add support for default values in plugin parameters

  🎍 A surprisingly easy addition that was missing before -- default values will only apply if the user never saved preferences for your plugin, so you shouldn't see any changes if you're updating. 🎊 The Regex plugin should also now execute before all the others in autoplugin if it's enabled, in order to try and get better base tags before querying external services. (And it has save title set to true by default now!)

• API Changes

  🔑 (#517) Add no_fallback to GET /api/archives/:id/thumbnail to get JSON in case a thumbnail is being generated instead of a fallback image 🔑 The pages/extract endpoint will now add the job ID for the matching Minion job ID. 🔑 (#517) Add PUT /api/archives/:id/thumbnail?page=xx to set a defined page as the cover thumbnail for an archive 🔑 (#517) Add a page parameter to GET /api/archives/:id/thumbnail to get a thumbnail for any page of a given archive

• Miscellaneous bugfixes and improvements

  💎 Fix toasts being under the overlays 💎 (#550/#551/#552/#553) Various CSS fixes by @CirnoT (Thanks!) 💎 Fix is_file_in_archive and matching plugins to work with the new extraction logic from 0.8.3 💎 (#559) Add a default robots.txt file 💎 (#531) Disable usage of tagger.js on mobile browsers 💎 (#536) Add missing useragent parameter to MEMS plugin 💎 (#563/#566) Add warning text to a few plugins to clarify potential issues 💎 Add default value for isnewin database 💎 Fix timestamp tags so it works properly with web uploads 💎 (#574) Disable header setting in infinite scrolling mode

Full Changelog: https://github.com/Difegue/LANraragi/compare/v.0.8.3...v.0.8.4

Bowie had been writing the Black Tie White Noise material throughout late 1991 and 1992. The first track that emerged from a desultory series of sessions was “Real Cool World,” a song written for Cool World, Ralph Bakshi’s disastrous animated film, a crass rip-off of Who Framed Roger Rabbit?, complete with a cartoon temptress and human-toon interactions.

This is a quick hotfix update for 0.8.2. Ain't much else going on lmao

The Windows version is now available in the MS Store

Due to the Microsoft Store allowing unpackaged Win32 apps now, I've thrown the LRR installer in there so you don't have to visit Github to quickly install it. Store updates will always lag behind regular GH releases by a few days due to Microsoft validation, so this is more of a gimmick than anything 😅

Changelog

• Miscellaneous bugfixes and improvements

  💎 Documentation/screenshots update 💎 Fix plugin sideloading being broken since the theme change 💎 (#542) Get rid of sanitize_filename for archive extraction 💎 (#545) Fix progress values not being shown properly in new Index 💎 Refine empty file detection for image serving a bit further to avoid issues with first files 💎 Fix thumbnail generation potentially grabbing the wrong image

The Walkers recorded “Nite Flights” in February 1978 at Scorpio Sound, UK. Bowie’s version was cut ca. summer/autumn 1992 at the Power Station and/or Mountain Studios, Montreux. A remix was released as a UK promo 12″ single (Arista HOME 1) and later included on the reissued Black Tie White Noise.

Holy cungadero, this update is big!
I've started addressing the User Survey Requests and got most of the small stuff out of the way, alongside completing my draft work on the new Reader model.

Very fast Reader extracting at incredible hihg speed

This is the big one; This new release should bring much-desired speed improvements to Reading, which was the top ask alongside Search speed improvements. (Search will come later it's slightly harder)

• /api/extract (now dubbed /api/pages but I'm leaving the old endpoint in for backwards compatibility) now fires a background Minion job to extract the archive, and uses at the same time a brand new Libarchive interface to return the list of files, directly streamed off the Archive.
• /api/page now checks if the desired page exists (as it mightve been extracted by the background job already), and if it doesn't, extracts it on the fly.

There are some timing considerations with this new method (If the background job is currently extracting a page and we want to serve it, we have to make sure it's fully extracted before serving), but overall things seem to work well! This should bring awesome performance improvements to most third-party clients as well.

Archive Index Revamp

I have taken into account all the requests for search-aware randomness, and rebuilt the Archive Index to introduce a new view:

The interface has been simplified in a few ways:

• Thumbnail view now has a proper sorting UI instead of reusing the one from Compact/Table view.
• The options modal has been removed; All settings are now directly integrated in the view itself to simplify operation
• The carousel view on top replaces the old new/untagged search filters, and introduces a new display mode that shows a selection of random archives from the current search. It's collapsible in case you don't really care and just want to use the Index as before.

I wasn't expecting search-aware random to be that big but in practice I find this implementation quite useful: I've been finding old stuff I hadn't read in ages again thanks to it. I hope this satisfies your demands 🙇

date_added is now a default tag

date_added tags benefit from quite a lot of integration in third-party clients and there's been demands to handle date stuff better, so as of this release, all new archives will now have the tag by default.

As a bonus, date-related tags (date_added and timestamp) will now be formatted correctly in the UI:

Theme preference is now stored server-side

I've removed the old common modal that allowed changing the theme from anywhere in the app due to low usage.
Theme settings are now applied to all users and can be found in the Configuration page, which hopefully will make them a bit more discoverable.

Changelog

• (#399) Switch Reader to a streaming model 🚀

• (#527) Add a swiper.js-powered carousel view containing random picks from the current search

  🌮 There's no longer a popup for options, as those are toggled via buttons on top of the table, Reader-style. 🪲 Debug mode will now show a sticky toast instead of a tiny link at the bottom of the page.

• (#516) Add date_added by default to new archives and format them correctly in the UI

• (#189) Detailed tag stats

  📈 The statistics page now features a table displaying all your tags, alongside their total count. 🔎 Tags are all clickable and can be searched for directly from this UI.

• (#326) (#513) Add Batch Tag Rules, Batch Delete and Batch Clear New

  🦇 Batch tagging received a pretty big revamp in this release and is now Batch Operations! 🆕 New archives are now marked as such in the batch window.

• API Changes

  🔑 (#515) Added /api/search/random ✨ 🔑 Added file extension info to archive endpoints in order to detect PDFs and other formats easily

• Miscellaneous bugfixes and improvements

  💎 Fix tag edit field not being fully clickable 💎 Fix Tag Cloud being wrongly populated 💎 Community-contributed regex fix for F* search 💎 CSS fixes for mobile 💎 Use flavor icons for spinners across the app 💎 (#510) Fix total count in search API 💎 Show new flag in batch tagging 💎 Update Reader to (almost) entirely rely on API endpoints 💎 Fix launcher issue that could prevent the WSL version from starting 💎 Deleting an archive from the Index will now reload your search instead of refreshing the entire page 💎 Docker builds now use a custom-built libffi to sidestep a bug with the new libarchive bindings (@Guerra24)

For the rest of the Eighties, when Walker was nowhere to be found, Bowie endured his own public set of lost years, reduced to making records for the sake of it, losing himself, trying to purge his way back with Tin Machine. Finally, in 1992, looking for some anchorage, casting about for fresh influences or just any means to move ahead, he finally decided to take Walker on. He covered “Nite Flights.”