Docker Compose project for the MusicBrainz Server with replication, search, and development setup
This release upgrades both the PostgreSQL version to 16 and the MusicBrainz database schema version to 29. It requires to run commands taking down all services during the upgrade.
Docker images for the services db
and musicbrainz
are now prebuilt too. It shorten the build time by half an hour to just a few minutes. Running any command that requires Perl in the service musicbrainz
should now be prefixed with carton exec --
.
Disk space requirement has been slightly readjusted: 250 GB (or 100 GB without indexed search), system included.
If your musicbrainz-docker
is earlier than v-2023-05-16-mbdb28
, you must first update to it (see its release notes) and follow the upgrade instructions below, or remove everything using sudo docker-compose down --rmi local --volumes
and reinstall.
Assuming your checked out version of musicbrainz-docker
is v-2023-05-16-mbdb28
or later, then:
Ensure you’ve replicated up to the most recent replication packet available with the old schema 28. If you’re not sure, run:
sudo docker-compose exec musicbrainz bash -c ./admin/replication/LoadReplicationChanges
and see what it tells you; if you’re ready to upgrade, it should say “This replication packet matches schema sequence #29, but the database is currently at #28.”
Turn off cron jobs:
admin/configure rm replication-cron
sudo docker-compose up -d musicbrainz
Switch to the new code with:
git fetch --tags origin
git checkout v-2024-05-13-mbdb29-pg16
Upgrade the PostgreSQL data format:
./admin/upgrade-to-postgres16
This step is expected to take some time, especially the last vacuumdb
commands.
(It took ~20 minutes with 4GB of shared buffers in test.)
It should say “Upgrade complete!” at the end.
Upgrade the MusicBrainz database schema:
mkdir -p local/compose/ && cp -a admin/lib/upgrade-db-schema/musicbrainz-stopped.yml local/compose/
admin/configure add local-compose-musicbrainz-stopped
sudo docker-compose up -d musicbrainz
sudo docker-compose exec musicbrainz upgrade-db-schema.sh
This step is expected to take a very long time, especially the last “Vacuuming DB”. (It took more than 2h with 4GB of shared buffers in test.) At the end it should say “Done” followed with a message that you can safely ignore as handled already: “UPDATE THE DB_SCHEMA_SEQUENCE IN DBDefs.pm TO 29 !”
The following lines are expected and can be safely ignored:
NOTICE: constraint "allowed_collection_entity_type" of relation "editor_collection_type" does not exist, skipping
NOTICE: constraint "label_code_length" of relation "label" does not exist, skipping
If the last command quickly stops with the following error, it means that previous step 4 failed for some reason:
ERROR: relation "edit_note_change" already exists
Please report this issue to us with the full log of the previous step(s) if possible. Then run the following commands instead:
sudo docker-compose stop
sudo docker-compose rm db
sudo docker-compose build
sudo docker-compose up -d musicbrainz
sudo docker-compose exec musicbrainz upgrade-db-schema.sh
Load the first replication packet available with the new schema 29:
sudo docker-compose exec musicbrainz bash -c 'carton exec -- ./admin/replication/LoadReplicationChanges --limit=1'
and see what it tells you; if the upgrade, it should say “This packet was produced (or begins) at 2024-05-13 21:29:12.739887+00”
The following lines are expected and can be safely ignored (for the first packet only):
The current row in musicbrainz.replication_control with key id='1' contains a different value in column current_schema_sequence (29) than the replication packet suggests it should have as the old value (28). at /musicbrainz-server/admin/replication/ProcessReplicationChanges line 497.
WARNING: amqp could not commit tx mode on broker 1, reply_type=2, library_errno=4
Turn cron jobs back on:
admin/configure rm local-compose-musicbrainz-stopped
admin/configure add replication-cron
Restart all services:
sudo docker-compose up -d musicbrainz
musicbrainz
servicemusicbrainz
service (see https://github.com/metabrainz/musicbrainz-docker/pull/276/commits/98732f4ad27baf98c150412c8a05288dc6247070):
carton exec --
See the above step 6. for example.db
servicev-2024-05-13-schema-change
(release notes)db
and musicbrainz
services)
It shorten installation time by half an hour to just a few minutes.This release can be safely skipped if you don’t use search indexes.
Just follow the new release notes for v-2024-04-09
instead if you didn’t already update to it.
Otherwise, that is assuming your checked out version of musicbrainz-docker
is exactly v-2024-04-09
, just run:
git fetch --tags origin && \
git checkout v-2024-04-10-hotfix && \
sudo docker-compose up --build -d
Assuming your checked out version of musicbrainz-docker
is v-2023-11-13
or later, then just run:
git fetch --tags origin && \
git checkout v-2024-04-10-hotfix && \
sudo docker-compose up --build -d
Changes to the development setup require to remove dependencies that will be cleanly reinstalled on startup:
sudo docker-compose exec musicbrainz bash -c 'rm -fr node_modules perl_modules .cpanm' && \
sudo docker-compose restart musicbrainz
In case your mirror is older than PostgreSQL 12 migration in 2020, if some update scripts didn’t work for some reason back in that time, you might have the following error in the file mirror.log
or mirror.log.1
in the service musicbrainz
since yesterday:
ERROR: index row requires xxxxx bytes, maximum size is 8191
Then open a PostgreSQL invite, for example as follows:
sudo docker-compose exec musicbrainz bash -c admin/psql
Finally, run the following SQL instructions:
SET search_path = musicbrainz, public;
DROP INDEX IF EXISTS track_idx_musicbrainz_collate;
DROP INDEX IF EXISTS track_idx_name;
DROP INDEX IF EXISTS track_idx_txt;
See MBS-13536 for more info.
Upgrade MusicBrainz Server to v-2024-04-09
Make Postgres, RabbitMQ, and Redis hosts configurable via Docker environment variables
Upgrade the version of Perl to 5.38 in development setup only
Make bash history persistent for the musicbrainz service in development setup only
Assuming your checked out version of musicbrainz-docker
is v-2023-11-13
or later, then just run:
git fetch --tags origin && \
git checkout v-2024-03-11 && \
sudo docker-compose up --build -d
v-2024-03-11
Assuming your checked out version of musicbrainz-docker
is v-2023-11-13
or later, then just run:
git fetch --tags origin && \
git checkout v-2024-02-16 && \
sudo docker-compose up --build -d
db
image build failingv-2024-02-16
Assuming your checked out version of musicbrainz-docker
is v-2023-11-13
or later, then just run:
git fetch --tags origin && \
git checkout v-2024-01-08 && \
sudo docker-compose up --build -d
v-2024-01-08
Assuming your checked out version of musicbrainz-docker
is v-2023-11-13
or later, then just run:
git fetch --tags origin && \
git checkout v-2023-12-20 && \
sudo docker-compose up --build -d
v-2023-12-20
Assuming your checked out version of musicbrainz-docker
is v-2023-11-13
or later, then just run:
git fetch --tags origin && \
git checkout v-2023-11-27 && \
sudo docker-compose up --build -d
v-2023-11-27
Assuming your checked out version of musicbrainz-docker
is v-2023-05-16-mbdb28
or later, then run:
git fetch --tags origin && \
git checkout v-2023-11-13 && \
sudo docker-compose up --build -d
sudo docker-compose exec musicbrainz bash -c 'rm -fr node_modules perl_modules .cpanm' && \
sudo docker-compose rm --stop musicbrainz && \
sudo docker-compose up --build -d musicbrainz
In case you have any error related to the collation in the database, or preventively, run last:
sudo docker-compose exec musicbrainz bash -c './admin/RebuildIndexesUsingCollations.pl'
v-2023-11-13
musicbrainz
to Ubuntu Jammy
musicbrainz
, to better follow the repository musicbrainz-server
Assuming your checked out version of musicbrainz-docker
is v-2023-05-16-mbdb28
or later, then just run:
git fetch --tags origin && \
git checkout v-2023-10-24-hotfix && \
sudo docker-compose up --build -d
db
(#259)
(not affecting existing instances)v-2023-10-24-hotfix
(affecting existing mirror instances only).