Easy data migration for MongoDB using Python.
_ _ _ _
___(_) | _(_) | ___ _ __
/ __| | |/ / | |/ _ \| '_ \
| (__| | <| | | (_) | |_) |
\___|_|_|\_\_|_|\___/| .__/
|_|
Cikilop v.1.1 is a simple and easy to use data migration tool for MongoDB that encourages you to write your migration scripts in Python. All you need is docker (or python 3.6).
What you need: Prepare your migration scripts, prepare your config, run cikilop. That's it.
1. Build your migration script with only two functions:
def success(client):
col = client["exampledb"]["examplecol"]
col.insert_one({"_id": 1})
def fail(client):
col = client["exampledb"]["examplecol"]
col.remove({"_id": 1})
Both functions must have an input parameter client
, which cikilop ships to you. Indeed, it is regular pymongo
MongoClient object (pymongo.MongoClient
).
2. You need to name your script as 01-xxxx.py
, it must start with a number with dash. Cikilop sorts your migration files using that number.
Caveat: When applying the migration, scripts run with ascending order (e.g. 01-mig.py
, 02-mig.py
, 03-mig.py
).
When reverting the migration, scripts run with descending order (e.g. 03-mig.py
, 02-mig.py
, 01-mig.py
).
3. Prepare your simple configuration file, Its name must be config.json. You must define configuration for each environment. There is an example for local environment above:
{
"local": {
"migrations_db_name" : "migrations_db",
"migrations_coll_name" : "migration_list",
"mongo_uri" : "mongodb://username:[email protected]:27017..."
}
}
Note: The name you set for migrations_coll_name and migrations_db_name are used by cikilop itself. It creates a collection in specified db and stores the migration records in it.
4. Run it!
Use the wrapper script (it also uses docker):
curl https://raw.githubusercontent.com/skynyrd/cikilop/master/ciki.sh --output ciki.sh && chmod +x ciki.sh
sudo ./ciki.sh -e ENVIRONMENT -c ABSOLUTE_CONFIG_FILE_PATH -m ABSOLUTE_MIGRATIONS_FOLDER_PATH
e.g.: sudo ./ciki.sh -e local -c $(pwd)/config.local.json -m $(pwd)/migrations
To revert, add -r true
or use docker directly:
docker run -e env=local -v ABSOLUTE_CONFIG_FILE_PATH:/app/src/config/config.json -v ABSOLUTE_MIGRATIONS_FOLDER_PATH:/app/src/migrations skynyrd/cikilop
e.g. docker run -e env=local -v $(pwd)/config.json:/app/src/config/config.json -v $(pwd)/migrations:/app/src/migrations skynyrd/cikilop
To revert, add --revert
to docker run
command
Thanks!
1.0.1 => 1.1