A standalone system that can perform backup and restore functions for a wide variety of pluggable data systems
This version of SHIELD will be unable to perform backups if the core has ever been restarted. This does not cause any permanent damage, but this version will not be able to do backups.
The meta SHIELD plugin has two additional (optional) fields. The core CA certificate and skip ssl validation fields allow us to now properly handle tls.
Properly Handle 301's from amazon during purge tasks
Switching bespoke Vault for vaultkv
New Features
metashield
plugin for backing up and restoring your shield core(s). This is a standardized way
for SHIELD to backup/restore itself or another SHIELD instance.This release fixes issues with failed 8.5.0 → 8.6.0 migrations
that left the interim jobs_new
table around.
This release chiefly introduces logic to existing and new database schema migrations to fix fallout from the 8.6.0 release, in which we mistakenly modified a historic migration to do something.
All database upgrade paths, including pre-8.6.0 → present, (failing) 8.6.0 deployment → present, and brand new deployments, should all work now.
fs
plugin is now quieter by default, and will only turn on
per-file debug logging if asked to do so via its own
configuration. This should greatly speed up backup operations
on busy SHIELDs, since it reduces the database lock contention.The front-end configuration wizard now properly looks up plugin metadata for an agent. Previously, there was a Javascript variable shadowing bug that caused the front-end to return any arbitrary plugin metadata as the "correct" one.
Task cancelation had a n inverted boolean assertion on global tenant-iness that has been fixed.
Do not use this version. This version has a database schema migration error that will be fixed in an upcoming patch release.
Helm Support! This version of SHIELD ships with OCI Docker images that can be used in the new (Beta!) helm chart for SHIELD. See https://github.com/shieldproject/helm for more details, and to give it a spin yourself.
The SHIELD Core can now be configured almost entirely through environment variables, for ease of configuration in Docker, Compose, and even Kubernetes.
We have a new Prometheus-compatible metrics exporter, accessible
at /metrics
, and governed by a separate set of HTTP Basic Auth
credentials.
The fabrics
array in the configuration file has been replaced with
the legacy-agents
hash.
Previously, configuration for this would look like
fabrics:
- name: legacy
ssh-key: |
-----BEGIN RSA PRIVATE KEY-----
kEy
-----END RSA PRIVATE KEY-----
Now, that should be configured like the following:
legacy-agents:
enabled: true
private-key: |
-----BEGIN RSA PRIVATE KEY-----
kEy
-----END RSA PRIVATE KEY-----
Agent SSH is now constrained to a more secure set of message authentication codes (MACs). Specifically, we got rid of one embarassing 96-bit MAC algorithm. Ooof!
Several quality-of-life improvements were made to the web UI and message bus / websocket implementations. In general, the web interface is easier to use and more robust now.
Old task logs and purged archives will now be removed from the database after a minimum retention period has passed. If you've been with us since the 0.x days, this update is for you, and we're sorry it's taken us so long to do this type of cleanup.
The SHIELD IP Address (which gets less and less relevant every day) is no longer reported via the API / web UI.
Uncompressed backups can now properly be restored.
The healthy
and paused
fields of the Jobs table now no
longer allows NULL values, landing us squarely back in the
territory of booleanitude -- things are either true or false;
there is no maybe.
Negative daily storage increases now properly convert to kilo-, mega-, and giga- units, to help humans understand magnitude.
The api.session.timeout
value is now interpreted properly as
seconds, not hours. This effectively means that sessions now
expire when they ought to, not several orders of magnitude
later.
The shield status
command is twice as pretty and three times
as useful.
Cloud Storage detail pages in the web interface now show a timeline similar to the one shown for Data Systems, so that SHIELD operators have an easier time of troubleshooting failing storage configurations.
The SHIELD CLI now displays task+log data for the last
test-store task of a given store (for shield store X
and
shield global-store Y
), to assist in troubleshooting failing
storage configurations.
Operators now have a web interface and some CLI commands for inspecting the state of SHIELD Data Fixups, and re-running them (if / when necessary).
The mongo
target plugin can now have options applied
individually to just mongodump
or mongorestore
.
Passwords and RSA private keys are now properly obscured in the web interface detail pages for both systems and cloud storage. People without rights to see such credentials will still see the "REDACTED" string instead; but people with the required privilege will instead see the blurred-out obscured text that they can hover over to reveal.
The shield tasks
command can now filter down to only tasks
that involve a particular tenant or global cloud storage system.
Data Fixups will now be properly skipped if they've already been applied. Additionally, names / dates / summaries will be updated every time the SHIELD Core boots up, to catch typos and mispellings there.
The Data System detail page in the web interface no longer has a
race condition between the start of an AJAX call for the plugin
configuration details and a shield:navigate
away from the
page. Other such race conditions involving AJAX should now also
be fixed.
Switching between tenants (with differing levels of access) now properly re-renders the sidebar to show your new privileges.
The shield job X
command now populates the "Status" and
"Agent" fields, using data in the API responses.
etcd
plugin for backing up and restoring your etcd key-value stores. It supports single- and multi-node clusters and can authenticate via roles and X.509 certificates. If you want, you can restrict the backup to a subset of the etcd tree (via a prefix setting). It also supports additive restore for situations that need it. You can also do a clean restore of the etcd cluster.cancel
, task
, restore-archive
and purge-archive
commands in the SHIELD CLI now properly support short UUIDs,
like all other commands.Submit buttons on forms now (a) disable themselves when clicked and (b) change their text to indicate an ongoing operation. This greatly increases the usability of the web UI. See #505
The web UI for rekeying SHIELD Core now correctly identifies when the operator would like to rotate the fixed key. Also, the error messaging for an incorrect current master password is better now, and by default, the "rotate fixed key" checkbox on the rekeying form is off. See #546
The default password for the failsafe account has been changed
from shield
to password
, for more continuity across various
packaging formats. See #531
The shield tasks
command (and the backing API) can now filter
tasks based on their task type (i.e. "backup", or "restore")
See #523
The Encryption
column of the system detail page's backup jobs
table now always shows something. For jobs that do not used
the fixed key, the new tag is randomized
. See #536
SHIELD now tracks when it last checked each agent separately from when it last "saw" the agent. Last Seen now means the point in time when the agent last connected to the SHIELD core, and Last Checked is when the core last connected to the agent for metadata retrieval.
SHIELD now allows agents to change their IP address; only the agent name is unchangeable. Previously, attempts to change an agents registered IP address (without changing its name) would fail.
Both the SHIELD Agent and the SHIELD CLI now trust the system X.509 CA Certificate Stores. See #555 and #556
The MotD separator no longer displays if the MotD is empty or not specified. See #530
The Ad Hoc Backup and Restore wizards now handle the "empty" state more gracefully, and instead of showing an empty table when there are no data systems, they warn you that you have no systems to backup or restore. See #532 and #533
Stores (global and tenant-specific) can now be properly deleted via the web UI and CLI.
When editing targets and stores on the webui changes are now persisted when editing again without a refresh.
The "Agents of SHIELD" admin page no longer gets stuck in a loading loop whenever websocket events are seen.
Global Storage Configuration details are now properly loaded when accessing a global store detail page from the admin panel. Previously, the details would only load if you visited the global store from the "Storage Systems" (non-admin) top-level, which only works if you actually have a tenant. See #535
shield annotate-archive
now works with short uuids
Orphaned archives from a target deletion no longer prevent users from loggin into SHIELD
The mongo
plugin now handles hosts with embedded ports,
allowing operators to configure multi-node mongo across
different ports on the same node, without --port
getting in
the way.
The mongo
plugin now allows you to authenticate against a
database other than "admin" (which remains the default).