Greyhole Versions Save

• Improvement: Added note about mounting shares locally, at the end of the initial setup wizard
• Improvement: use greyhole --config instead of grep-ing the greyhole.conf file manually, in init scripts
• Added note about 30-days trial period for the Admin web UI to the LICENSE file
• Bugfix: don't show unfinished (written, but not closed) task, in Greyhole Admin | Status
• New: show Queue stats in Greyhole Admin | Status (equivalent of greyhole --view-queue)
• New: start fsck from new Greyhole Admin | Actions page
• New: button to cancel an ongoing fsck in Greyhole Admin | Status
• New: Last fsck Report tab in Greyhole Admin | Status (which uses tabs now to display the various sections)
• Change dark mode theme in Greyhole Admin
• New: Greyhole Admin | Status | Past Tasks: browse past (already processes) tasks, with pagination & search
• New: Start (and stop) balance from Greyhole Admin | Actions
• New: Emptying Trash from Greyhole Admin | Actions
• Code refactoring for tabs in Greyhole Admin
• New: Pause/Resume in Greyhole Admin | Actions; also show Paused status in Status page, and --status output
• New: toggle to follow (tail) log in Greyhole Admin | Status; will refresh the "Recent log entries" every 10 secs
• Removed space before ,
• gh-disk-usage.log is not used anymore
• Bugfix: fsck for all shares: email report was listing all the shares multiple times
• WebUI: Better detect what is the currently ongoing task
• Added to WebUI: remove a drive from the storage pool
• WeUI: load all data using AJAX, and reload when changing tabs, to ensure all presented data is up to date (in Status and Storage pool tabs - others are static config/actions)
• WebUI: use Tab objects to define and print tabs (nav & content); refactored tabs IDs everywhere
• Samba VFS module for Samba 4.13 Fixes #247
• WebUI: New Trash Manager page Fixes #245

• Improvement: when a file is modified, don't wake all storage pool drives unless necessary
• Improvement: Less log spam when postponing a task for a frozen dir
• Bugfix: webapp (to visualize used space) was missing some includes
• Bugfix: quiet PHP warning about file_put_contents() when trying to save a metafile eg. when drive is full; but log a warning
• Removing deprecated db_engine from example .conf and man page
• Removed underline from '=' character, in greyhole.conf man page
• Allow disk-usage web-app to run from any dir
• Document 'check_storage_pool_schedule' config
• WIP: New web UI to monitor / control Greyhole; UI only for now, nothing is connected (changing values does nothing!) Ref: #8
• Better organization of code in new web UI; split init, functions, styles, and javascript into different files
• New web UI: save changed (config) values in /etc/greyhole.conf, using the new "--config name value" parameter of the greyhole CLI
• Dark/Light modes
• New Web UI: added Status section
• Committed test code by error; small UI improvements
• New Web UI: add new drive to storage pool (using modal); needs more validation checks
• New Web UI: multiple UI tweaks
• New web UI: various improvements, mostly in code, but some in UX: allow using web UI without DB connected; num_copies=max is not the same as num_copies=# (where # is the number of storage pool drives); added favicon
• New web UI: Samba config ([global] only), and users (view and add only); missing: restart Samba when needed
• New web UI: enable/disable Greyhole for specific Samba shares; restart Samba when needed (prompt)
• Bugfix: use utf8mb4 charset for MySQL session; fixes issues with non-ASCII characters in du_stats
• New web UI: create new Samba share
• New web UI: disk usage browser (treemap & bar graphs)
• New web UI: use menu to show different sections; split code in different view files; other small improvements
• Small updates to README and example .conf
• New web UI: Added help text to Greyhole config; updating browser history & page title when navigating (tabs) using Javascript
• New web UI: New LICENSE (donation-ware), and related UI elements
• Merge pull request #243 from gboudreau/dev/web-ui
• Update INSTALL
• Update README.md
• Balance Status graphic in web UI (Status page), when --balance is ongoing
• Bugfix: get_debug_bt() was modifying the actual function arguments (objects and array), replacing them with string representations!
• Bugfix: don't insert into settings table if it already contains something (an empty array, for example)
• Improvement: better log, when a file is kept open by another process
• Bugfix: Stop loading Gone-FSCKed-Drives setting from DB over and over for no reason. Same for Gone-OK-Drives. Fixes: #244
• Bugfix: allow .deb to be installed in a Ubuntu Docker container, where modprobe is not installed, and /sys/modules/ are read-only
• Bugfix: status table was not logging executed tasks, because the action column definition was wrong
• Web UI: new wizard for initial setup
• Including new web UI in release
• Web UI: Bugfix: show Samba config options in red when invalid (wide links off, or unix extensions on & allow insecure wide links off)

• Bugfix: status table should be MYISAM, not InnoDB Fixes #241
• Improvement: Renamed --going and --gone to --remove Will prompt the user to know if we should (or not) try to copy files from the specified drive. Fixes #88
• Improvement: Renamed --replace to --replaced
• Improvement: --remove is now executed asynchronously (using the daemon) An email report is sent once it's done. Ref: #58
• Improvement: --remove-share is now executed asynchronously (using the daemon) An email report is sent once it's done. Fixes #58
• Improvement: When executing --remove, check the copied files MD5 Fixes #242
• Bugfix: quiet PHP warning about file_put_contents() when trying to save a metafile eg. when drive is full
• Bugfix: --balance-status triggered division-by-zero for drive groups that were empty

• Build script: make sure code is up to date with remote, before creating packages!
• arm64: install the Samba VFS modules in the correct directory

• Bugfix: there was a 50% chance that Samba spooled tasks with the same modified-time timestamp would be incorrectly sorted, resulting in close-write-open ordering, instead of open-write-close. This resulted in a stuck daemon. Fixes #238
• Bugfix: Missing 4.12 support in .deb postinst script
• Bugfix: when a CRITICAL error occurs during the initialize phase (eg. during config file parsing), don't log, but DO exit(1) Fixes #239
• New: Support for arm64 (aarch64) libdir; added .deb package for arm64
• Improvement: Simplified spooled tasks sorting code
• Improvement: Added some steps and notes in INSTALL file, to help users manually install Greyhole

1040a1041

• Bugfix: round free space number, in output of dfree Fixes #235

• greyhole-dfree was broken; missing required classes Fixes #235

• Crashfix: renaming (moving) a folder that was ignored Fixes #234

• Small fix for MySQL 8 Fixes #232
• Prevent crash when trying to send a inexistant fsck log file by email
• Bugfix: free space calculation was not working, in docker, if multiple mounts (-v) were defined for a storage pool drive, and 1+ folders inside it

New features

• New command: greyhole --mv (beta; requires more testing) - Allows you to move data from one Greyhole share to another without copying the data through the Samba shares
• Detect and fix bitrot/silent errors; requires a new cron job; see near the end of USAGE file Fixes #199
• New command: greyhole --print-fsck : Print the fsck report for the last completed fsck task. This will print the same content that is sent by email when the --email-report option is used.
• Refactored FSCK report code; new layout for the 'all shares' fsck that runs weekly

Improvements

• greyhole-dfree (used by Samba to calculate free space) should take into account the num_copies option of the share; i.e. real free space = 1 GB, num_copies = 4 => available free space on Samba share = 250 MB; also added -h parameter to greyhole-dfree executable Fixes #156
• Save recent log entries in DB; use that in greyhole --status Fixes #114; Fixes #102
• Allow restarting Samba & Greyhole when running in Docker
• Mention allow insecure wide links = yes as an alternative to unix extensions = no in USAGE file Ref: #192

Bugfixes

• Correctly attribute write/close tasks; using the file handle (fd) was not enough; using the full path now
• When a write task is logged, and no opened-for-writing task is in the DB, create it! Fixes issue where Greyhole would think a file is closed without being written to, when that is not the case.
• When two spooled files (created by Samba VFS module) have the exact same modification time (to the microsecond!), ensure they are executed in the correct order: open, writes, close
• When running --fsck --disk-usage-report, correctly update 'parents' du_stats, when deleting the checked folder's du_stats
• --iostat was missing storage pool drives on un-numbered partitions (eg. whole-drive XFS partition on /dev/sdb); also added timestamps to output, updates are more frequent (5s instead of 10s), and sorted the storage pool drives in alphabetical order
• Correctly balance storage pool drives, when drive_selection_groups are configured: will balance the drives in each group amongst themselves only, instead of trying to balance all the drives together. Fixes #198
• greyhole --process-spool logged warnings because Metastore backups config was uninitialized
• If the Samba VFS symlink is missing/not working, and the expected target is also missing, download it from Github
• greyhole --logs should not watch both .log and .err, as .log will always contains all the same logs as .err
• Install script: ensure gnupg package is installed; it is required for apt-get add Fixes #227
• build_vfs script: try to install the required dependencies, before trying to build Ref: #223
• lsof file-locked check was not working in Docker (Alpine Linux), because -M flag is not available

Various code improvement & refactoring

• Removing old files; shellcheck; fixed white-space in shell scripts (was a mix of tabs and spaces)
• Updated .patch to Samba 4.11 & 4.12 VFS module (was not working on Alpine 3.11 & Alpine Edge - 3.12)
• WIP: VFS module for Samba 4.13 (untested)
• Simplified manual INSTALL instructions about Samba VFS module; just need the correct binary in the LIB dir; daemon will take care of created the required symlink pointing there
• Moved build-related files into build/ sub-directory
• Create new Docker images during build
• Faster re-building Samba VFS modules, on build servers