💫 The easy way to synchronize the DNS configuration of two Pi-hole 5.x instances.
Fixes bug that prevented properly calculating hashes of the DNS CNAME file, resulting in always considering there are changes to sync (#431) by @alexperrault
Full Changelog: https://github.com/vmstan/gravity-sync/compare/v4.0.6...v4.0.7
Updates to respect configured options (https://github.com/vmstan/gravity-sync/pull/413) by @emroch
Full Changelog: https://github.com/vmstan/gravity-sync/compare/v4.0.4...v4.0.5
Gravity Sync will now sync static DHCP assignments. This is handled by management of the 04-pihole-static-dhcp.conf file. There is no additional work required for Gravity Sync to sync these static DHCP assignments, if you're using your Pi-hole as a DHCP server this file will exist and Gravity Sync will see it and act accordingly.
See the link above for more information on scoping for these changes.
🐞 #339 Fixes bug in config script where a inputting a blank value wasn't properly handled reported by @chris2fourlaw 🪲 #335 Fixes bug in importer where non-standard SSH keyfiles were ignored reported by @zzen
🪵 The hash log for CNAME replication has been renamed. After upgrading, your first replication event will include CNAMEs even if nothing has changed. It should not indicate any problems.
A couple more bug fixes and a tweak to automation:
gravity-sync auto function, with the help of @greggitterPlease see https://github.com/vmstan/gravity-sync/wiki/Automation for more.
A few minor issues fixed up, as reported by early adopters. Please see the linked issues for more information.
gravity-sync config commandThanks to @jackc94 @ksdmg @bakerboy448 for your feedback!
Full Changelog: https://github.com/vmstan/gravity-sync/compare/v4.0.0...v4.0.1
Gravity Sync 4 is an extensive rewrite of the application. There is not a single function, variable, or chunk of code that has gone untouched.
In the past I've always tried to make sure that anyone who wanted to update, would not have to do much more than occasionally reapply automation jobs to take advantage of new features. The effect of this made me reliant on flawed or shortsighted decisions that I'd made two years ago, and forced building all new features on top of (or within) the structure of those decisions.
Gravity Sync 4 is a fresh start.
gravity-sync dev will toggle you back over.The Gravity Sync installer has been rewritten to accommodate for both new installations and upgrades. (Upgrading only after a successful migration to 4.0, see below.) Previously it would detect existing installs and refuse to complete. There is also a dedicated update script that can be used to fix a broken deployment by executing bash /etc/gravity-sync/.gs/update.sh but should only be required if for some reason gravity-sync update has failed.
The architecture and installation have been simplified so that Gravity Sync is installed on the pair of Pi-hole boxes that you want to use it. There is no more primary or secondary Pi-hole, and both sides run replication jobs as peers. When either side detects changes to your Pi-hole configuration, they are sent to the peer.
Automation jobs are set to run at rotating-randomized times to avoid collisions, and to prevent continuous re-syncing the hashes that were previously only saved to one instance are now also sent to the peer at the end of a successful replication event.
With automation fully enabled, changes to either Pi-hole will usually result in both sides reflecting the change within 5 minutes. However if you're making large changes you want to monitor, or just feeling impatient, you can still manually run a replication job anytime.
If you only want to run Gravity Sync on one Pi-hole, that's still fine too. An optional "peerless" mode will detect the absence of a configuration at the remote site and bypass the hash sync. A warning will appear in the Gravity Sync console output to configure at the remote side for full functional, but can be ignored. (This also should allow folks who replicate between more than two Pi-hole instances to have multiples syncing to a single source of truth.)
Gravity Sync previously required itself to live in the user's $HOME directory. Alternatively, if it was installed as the root user, Gravity Sync could live in any self contained folder on the system.
In 4.0, Gravity Sync now lives in /usr/local/bin (along side pihole) and the configuration files and job logs are contained in the /etc/gravity-sync folder. This change to the configuration folder should make containerizing Gravity Sync easier in the future. (hint, hint)
The main script has been renamed from gravity-sync.sh to gravity-sync and due to it's relocation, has the nice effect of letting you execute just gravity-sync from anywhere in the system. Said another way, you no longer have to run ./gravity-sync.sh from the Gravity Sync folder in $HOME anytime you want to manually run it.
Along with the removal of the dedicated backup and restore features that happened in 3.5, in 4.0 the backup functions are resimplified in that backup files are no longer timestamped, so jobs that no longer complete successfully will not continue to write new backup files each time, until the hard drive has been filled.
/tmp folders on both the local and remote Pi-hole..gsb appended to them during processing. This replaces the various .backup, .pull, .push files of previous versions.The amount of write activity, network traffic, and overall execution time has been reduced by cutting down on the number of backup data copies that are created, especially during push operations.
The dependency on a standalone copy of SQLite3 has been removed. Gravity Sync now uses the integrated SQLite3 instance that is built into Pi-hole's FTL component (pihole-FTL sql) which also removes impediments for folks wanting to install Gravity Sync on systems that didn't have the ability to also install SQLite3.
All cron related automation functions have been removed. The systemd based replication function that was introduced in 3.7 comes forward, with further enhancements. This means packages like 'cronie' are no longer needed on systems that lacked a built in crontab.
In addition to everything above there are a few new functions.
gravity-sync purge now uninstalls and completely removes Gravity Sync from the system. It previously only reset the installation to new.gravity-sync disable will fully stop and remove all automation jobs in systemd.gravity-sync monitor will let you watch the real time status of replication jobs.If DNS Records and CNAME Records are detected at either side, they are replicated. Previously these functions required you to opt-in to replicating these, but this was done because their functionality was added after the initial release of Gravity Sync in 2020.
There are two ways to upgrade to Gravity Sync 4.
curl -sSL http://gravity.vmstan.com | bash
Using the curl to bash one liner listed above, you will deploy a new install of Gravity Sync 4 to both of your existing Pi-hole. Any current Gravity Sync configuration will be ignored.
You will need to generate a new configuration file using gravity-sync config but because the configuration workflow has been completely redone, in most cases you will only need to provide the IP address & username/password for your remote Pi-hole. Gravity Sync will automatically detect if you have a containerized instance of Pi-hole and prompt for the name of the instance(s). Once this is provided it will automatically detect the bind mounts in use for Pi-hole's two /etc directories. It's possible to have your new configuration generated in less than a minute.
Gravity Sync now uses a dedicated SSH key for connection to remote Pi-hold instances. This key is stored as /etc/gravity-sync/gravity-sync.rsa and should be used only for Gravity Sync. The configuration utility will create the new key file. Existing default id_rsa key associations to remote servers can be safely deleted/removed unless they're associated with some other process.
From your existing Gravity Sync folder, running any version 3.1 to 3.7, run ./gravity-sync.sh update and the version 4.0 code will temporarily come into the legacy folder. At this point ./gravity-sync.sh is the migration tool to move you to the version 4.0 format.
Run ./gravity-sync.sh again and a new Gravity Sync 4 installation will be laid down for you and your existing configuration, logs, SSH key, and hashes will be migrated to the fresh install. Legacy backup files are not retained. At the end of a successful script execution the legacy Gravity Sync folder will be deleted, so you may need to run a cd ~ to get yourself out of the now deleted folder.
Many of what were termed "Hidden Figures" as advanced configuration options have been removed, as they're no longer supported or the logic behind them being necessary have changed. So only the configuration settings relevant to 4.0 will be transferred.
Your automation settings will be defaulted to nothing. You will want to run a gravity-sync compare to make sure your migration was successful.
At this point you will want to perform a fresh install of Gravity Sync on your other Pi-hole to enable peer mode and hash sharing. Once that is successful enable automation on both nodes using gravity-sync auto.
This release features a different method of managing automated replication. Crontab has been deprecated in favor of scheduled tasks using systemd timers. To switch to the new replication method, just run ./gravity-sync.sh auto again, and it will remove existing crontab settings in favor of systemd.
Using systemd allows more options in timing replication jobs, but this especially useful for viewing the logs of previous automation jobs via standard journalctl -u gravity-sync commands. Where as previously automation jobs would run at 5, 10, 15 or 30 minute intervals within an hour, new automation tasks within systemd are configured by default to run every 5-10 minutes after being started. (This is 5 minutes + a random timer < 5 minutes.) Replications will automatically attempt for the first time 2 minutes after the system is powered on. This bit of randomization should allow users with multiple secondary Pi-hole syncing from a single primary, the ability to stagger load on the primary.
/etc/systemd/system/gravity-sync.timer file and adjust the values for OnBootSec, OnUnitInactive, and RandomizedDelaySec accordingly. There is no method within the Gravity Sync interface to adjust these, at present.templates/gravity-sync.timer file are not be persistent after updating Gravity Sync, but the timers applied to your individual system in /etc/systemd/system/gravity-sync.timer will not change unless you run ./gravity-sync auto again after updating.journalctl -u gravity-sync More how-to on the advanced uses of journalctl can be found here.Previously, running ./gravity-sync.sh auto and setting the value to 0 would disable the cronjob. Running it again would then prompt for a new value. Now the AUTOMATE function will only reapply the latest automation job settings. If you would like to temporarily pause automated replication, you can use systemctl stop gravity-sync.timer and then systemctl start gravity-sync.timer to begin again.
systemctl disable gravity-sync for this task. Running ./gravity-sync.sh auto again will enable and start replication again../gravity-sync.sh cron which was previously used to display the output of the previous cronjob.cat logs/gravity-sync.cron to view the output of the previous cronjob../gravity-sync.sh auto