Dependencies
- DC/OS 1.9+
- Docker 1.11+
- ElasticSearch 2.4
- PostgreSQL 9.4+, PostGIS 2.0+
- Message Broker (RabbitMQ 3.6 or Amazon Web Services SQS)
- Vault 0.6.2+ (only if not using DC/OS Enterprise for secrets storage)
Deprecated
- The v4 version of the REST API is now deprecated. We recommend that you migrate all use of the Scale REST API to use the new v5 REST API. The changes between the v4 and v5 versions are:
- The /v4/sources/{id}/ API has been replaced by a v5 version. The new v5 version removes the "ingests" and "products" fields, which were expensive to include. To get the same information, users should now use /v5/sources/{id}/ingests/, /v5/sources/{id}/jobs/, and /v5/sources/{id}/products/. The new /v5/sources/{id}/ API also removes the ability to use the file name in the URL.
- The /v4/status/ API has been replaced by a v5 version. The new v5 version is designed to be a near-real time look at the scheduler status and updates approximately every 5 seconds.
- The /v4/nodes/ API has been replaced with a v5 version. The new v5 version removes the port, slave_id, is_paused_errors, and last_offer fields in the response.
- The /v4/nodes/{id}/ API has been replaced with a v5 version. The new v5 version removes the port, slave_id, is_paused_errors, last_offer, resources, disconnected, and job_exes_running fields in the response.
- The /v4/nodes/status/ API has been deprecated in favor of the new /v5/status/ API.
Known Issues
- A bug has been introduced where jobs may not complete or fail correctly, leaving them in the RUNNING state. This can occur due to a race condition where the completion or failure occurs before the backend update that sets the jobs to the RUNNING state. For a workaround, any stuck jobs can be canceled and requeued. A fix will be coming in the Scale v5.3.0 release.
New Features
- A new system task has been added that will perform needed database changes in the background, allowing Scale to run at the same time (Issue 928)
- Created the Scale message handler service to launch tasks that consume and process the messages in the asynchronous backend system. The number of message consumers can be controlled via the Scale REST API (Issue 959).
Enhancements/Updates
- Updated REST APIs for creating, importing, and editing job types to take secret setting values provided in the configuration JSON and store them in Vault for security (Issue 816)
- Updated trigger rules to allow matching on one tag within a list of data tags (match any) and allow a file to be excluded (not matched) with any tag within an exclusion list (Issue 88)
- Completed a major database refactor that split the job_exe table into there separate tables for future scalability (Issue 928)
- Created the first backend message command that creates job_exe_end models for jobs canceled off of the queue (Issue 962)
- Moved update of jobs to RUNNING status to the new backend messaging system and added node field to job model (Issue 965)
- Moved creation of job_exe_end models to the new backend messaging system (Issue 974)
- Created an index for the created field in the scale_file model (Issue 967)
- Improved performance for the product updates REST API (Issue 968)
- Updated recipe comparisons to detect changed jobs that supersede one another when a recipe definition is updated (Issue 656)
Bug Fixes
- Fixed a bug in retrieving secret settings values from Vault (Issue 938)
- Fixed a rare bug related to the scheduler database sync (Issue 953)
- Fixed bug where the job execution REST API encountered an error handling old configuration and resource JSON values in the database (Issue 968)
JSON Schema Changes
- The job execution configuration schema has been updated to version 2.0 to contain all configuration used to launch each execution task.
Starting with this release, changes to the database will occur in two steps:
Database Migrations
These are relatively quick database changes applied using Django migrations to alter tables, constraints, indexes, etc. The migrations are applied during the start up of the Scale scheduler.
- The configuration field in the job table is removed.
- The job_exe table is effectively split into three total tables. To achieve this two new tables are created: job_exe_end and job_exe_output. Also new fields job_type_id, exe_num are added to job_exe, field disk_in_scheduled is renamed to input_file_size, many fields in job_exe have been made nullable (they will be removed in the future), and a new multi-column index on (job_id, exe_num) is added to job_exe. The new index may take a little while to create depending on the size of your job_exe table.
- The queue table is deleted and re-created with a different column set. Then a custom migration runs to populate the new queue table will all currently queued jobs. The length of this migration will depend upon how many jobs are currently queued.
- A new field node_id is added to the job table.
- A new index is created on the scale_file created field.
Database Updates
These are long running database changes related to actual data row updates that run in the background in a Scale system task. These updates are performed while Scale is running so they don't increase down time when deploying a new Scale version.
- Fields from the job_exe table have been made nullable and moved to the new job_exe_end and job_exe_output tables. To update legacy data, the Scale database updater will grab old field values from the job_exe table, bulk create the correct corresponding rows for the job_exe_end and job_exe_output tables, and set the old fields in the job_exe table to null.