Development environment for Meedan Check, a collaborative media annotation platform
A collaborative media annotation platform.
This is a Docker Compose configuration that spins up the whole Check app locally. Tested on Linux and Mac OS X. The repo contains two Docker Compose files, one for development (docker-compose.yml
) and the other for testing (docker-test.yml
).
docker-compose
vm.max_map_count=262144
in /etc/sysctl.conf
git clone --recursive [email protected]:meedan/check.git && cd check
bin/first-build.sh
and wait (for about one hour this first time!!) for a string in the log that looks like web_1_88cd0bd245b7 | [21:07:07] [webpack:build:web:dev] Time: 83439ms
docker-compose exec api bash
bundle exec rails c
me = User.last; me.confirm; me.is_admin = true; me.save
Note 1: For security reasons, not all credentials and configuration values are provided by copying .example
files during the initial build. For Meedan members, you need to set your AWS_PROFILE
environment variable, login to AWS (aws sso login
) and then the script bin/first-build.sh
will retrieve and set the required values for you. If you're not a Meedan member, you need to set at least the google_client_id
and google_client_secret
values in check-api/config/config.yml
, here is how you can get those. Other optional features can be enabled by setting the required credentials, for example, the FACEBOOK APP ID is needed to get Facebook social metrics and to run the Facebook related tests.
Note 2: For performance reasons, some services (that are not needed to run the application with its basic functionality) are disabled by default (e.g., commented in the Docker Compose file). If you need those services, please uncomment them in docker-compose.yml
. If you may need to increase the amount of memory allocated for Docker in order for it to work.
Note 3: There is a seed file you can run to create a new user and fake data, or to add fake data to an existing user. You can do that by running docker compose run api bundle exec rake db:seed
. If you picked the option to create a new user it will print the user information when the script is done. If you picked the option to add data to a user, it will let you know it was successfully added when it's done.
Note 4: Some users on MacOS have reported memory issues when running Docker, so it's recommended to install OrbStack to manage those services. Installation of Check is the same, Orbstack will automatically pick up the docker containers and they can then be managed within the app.
web
) at http://localhost:3333
api
) at http://localhost:3000/api - use dev
as API keyapi
) at http://localhost:3000/admin
bot
) at check-bot/dist
, a ZIP file that should be deployed to AWS Lambdapender
) at http://localhost:3200/api-docs - use dev
as API keyalegre
) at http://localhost:3100
minio
) at http://localhost:9000 - use AKIAIOSFODNN7EXAMPLE
/ wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
to loginelasticsearch
) at http://localhost:9200
kibana
) at http://localhost:5601
postgres
) at localhost:5432
(use a standard Pg admin tool to connect)chromedriver
in test mode) at http://localhost:4444/wd/hub
localhost:5900
(use a standard VNC client to connect with password secret
)narcissus
) at http://localhost:8687
fetch
) at http://localhost:8687/about
search
) at http://localhost:8001
Start the app in test mode: docker-compose -f docker-compose.yml -f docker-test.yml up
Check web client: docker-compose exec web npm run test
Check browser extension: docker-compose -f docker-compose.yml -f docker-test.yml exec geckodriver bash -c "cd /home && chown -R root seluser" && docker-compose exec mark npm run test
Check Slack Bot: docker-compose exec check-slack-bot npm run test
Narcissus: docker-compose exec narcissus npm run test
Fetch: docker-compose exec fetch bundle exec rake test
Running a specific Check Web test: docker-compose exec web bash -c "cd test && rspec --example KEYWORD spec/integration_spec.rb"
Running a specific Check Mark test: docker-compose exec mark bash -c "cd test && rspec --example KEYWORD spec/app_spec.rb"
Running Check API or Pender tests:
check
directory run docker-compose up api pender
). They do not need to be run in a "test mode" container, allowing us to run unit tests while having a browser connected to the development server. These tests run against a different database than development, so your development data will be preserved. More on testing Rails with Minitest, the testing framework we use for most of our application, can be found here, including directions about managing the test database.check
directory, docker-compose exec <service name (pender, api)> bash
bin/rails test
bin/rails test <path/to/test/file/in/check-api>.rb:<test-line-number>
(e.g. bin/rails test test/controllers/graphql_controller_test.rb:18
)
test_helper.rb
when running individual tests frequently, as they will not be automatically disabled../bin/git-update.sh
./bin/tar-config.sh
docker-compose run api bash -c "touch tmp/restart.txt"
docker-compose run api bundle exec rails c d
docker-compose run api bundle exec rake lapis:graphql:schema
docker-compose run api bundle exec rake jsonapi:resources:update_schema
We have recently upgraded to Postgres version 11 from 9.5. This necessitates a migration of existing databases to the new version. The migration will create a new data volume, so make sure you have enough storage space for a second copy of your databases. To migrate run these commands:
docker-compose down
docker-compose -f docker-upgradedb.yml up
docker run --rm -v check_postgres11:/var/lib/postgresql/data -u postgres -it postgres:11 bash -c "echo host all all 0.0.0.0/0 md5 >> /var/lib/postgresql/data/pg_hba.conf"
docker-compose up --remove-orphans --abort-on-container-exit
This will leave behind your original data volume and which you can clean up by running docker volume rm check_postgres
.
container_name exited with code 137
, you will need to adjust your Docker settings, as per https://www.petefreitag.com/item/848.cfm
chromedriver
in test mode, like the following:docker-compose -f docker-compose.yml -f docker-test.yml up --abort-on-container-exit
Starting check_elasticsearch_1_2e69e84ccb56 ...
Starting check_chromedriver_1_6a1e9d8f5fd4 ... error
[..]
ERROR: for chromedriver Cannot start service chromedriver: network 16d99f6d3d81011870fece7c627230b9410bdb5d0abc2d10a32f54af9f37931f not found
ERROR: Encountered errors while bringing up the project.
try this: docker-compose -f docker-compose.yml -f docker-test.yml down