1.1.0 (2022-04-04)

New Features

Project & API Documentation

• Adding RTD Requirements file and yml
• Updated JDOX links so they don't produce 404 errors
• Updates to About page

Web Application

• Speed up Webapp loading
• Stripped code for creating mosaic preview images
• Ignoring unuseful pipeline products
• Updated feature description on EDB page
• Closed all open sessions to db explicitly
• Fix/Modify Integration Number slider
• Switch filesystem and astroquery.mast to point to OPS
• Updates to BANDIT CI
• Made plotting limits robust to missing data
• Removing reliance on jwst-dms-edb
• Updated MIRI Readout Patterns
• Updated "Filter By" Dropdown in Archive View
• Plot non-numerical mnemonic dataset

jwql Repository

• Removed python 3.7 support
• Added python 3.9 environment
• Updated README to reflect changes in environment files.

New Features

Project & API Documentation

• Included documentation describing the algorithms used for instrument monitors
• Included templates for bug reports and feature requests as well as a link to submit issues on the home page
• Included Documentation option on instrument drop down menu in the navigation bar

Web Application

• All relevant instruments (NIRCam, NIRISS, NIRSpec) are incorporated into the Bias Monitor
• The generalized pipeline function from pipeline_tools.py is used for image calibration
• For exposures containing multiple integrations, created thumbnail image for only a single integration
• Implemented tests for all Monitor webpages

jwql Repository

• Modified filesystem to reflect MAST-like organizational structure
• Implemented application security tools into GitHub Actions builds
• Switched filesystem to use latest DMS build
• Improved configuration file handling

Bug Fixes

Project & API Documentation

• Include API Docs for all modules, classes, and functions in ReadTheDocs

Web Application

• Included various fixes and improvements suggested during stakeholder demonstrations
• Fixed View Image, View Proposal, and View Header buttons
• Updated dashboard such that NULL Monitor Status values were fixed
• Updated options that users can select for general image queries
• Eliminated pop-up warning when viewing fits file headers in the web app
• Removed login button on homepage and supporting authentication code and replaced with SSO authentication

jwql Repository

• Fixed Dark Monitor failure due to dither point keyword values in data
• Addressed Dark Monitor failure with data including 64-bit integers
• Resolved the NIRSpec amp boundary issue
• Fixed missing reference file error handling with latest DMS build in Readnoise Monitor
• Ensured Dark, Bias, and Readnoise Monitors complete successfully with latest DMS build
• Included read permissions for database tables
• Performed flake8 sweep of the code repository
• Updated software to support the latest versions of bokeh and pysiaf
• Removed dependencies on the astroconda channel

New Features

Project & API Documentation

• Included badges for GitHub Actions in README.md
• Replaced Jenkins support with GitHub Actions for CI testing
• Removed Python 3.6 support

Web Application

• Allow local developers to bypass auth.mast authentication
• Added cosmic ray monitor, unit test code, and relevant database files
• Removed unsupported options including image anomalies, unlooked images and monitors that haven't been implemented
• Improved loading times for web app archive pages by generating a representative thumbnail with generate_proposal_thumbnail.py, using astroquery.Mast rather than scraping filesystem, adding optional checks that file paths exist
• Implemented JWQL Dashboard View

jwql Repository

• Updated the call to the JWST pipeline RSCD step
• Added GitHub Actions

Bug Fixes

Web Application

• Improved loading times of readnoise monitor webpage
• Fixed broken bokeh CDN links

jwql Repository

• Dark monitor exits gracefully when there are not enough files in the filesystem
• Fixed filename parsing for WFS&C and AMI products
• Adjusted generation of preview images such that images are created for all file types
• Update ops naming convention in log directory

New Features

Project & API Documentation

• Added project citation information to README, along with a Zenodo badge
• Added API Documentation for bokeh templating software

Web Application

• Reorganized and made aesthetic improvements to instrument landing pages to be more user-friendly
• Enabled more dynamic search options and aesthetic improvements for anomaly query webpage
• Added web app view for displaying a particular table of the jwqldb database
• Added webpage for displaying Bias Monitor results with bokeh plots

jwql Repository

• Changed utils.credentials.py to always authenticate a MAST user with a user-identified token in the config.json file, instead of using a cached token, which was sometimes causes errors
• Updated software to support the latest versions of django and bokeh
• Removed affected_tables column of monitor database table, as it stored redundant information
• Updated the Readnoise Monitor to work for all JWST instruments

Bug Fixes

Web Application

• Fixed bug in Readnoise Monitor webpage that was causing the web app and jwql database to hang

jwql Repository

• Fixed bug that was causing the test_amplifier_info() test in test_instrument_properties.py to fail; truth values were updated to reflect a change in the format of the returned dictionaries from the amplifier_info() function
• Fixed bug in get_header_info() that was causing test_data_containers.py to fail; the function expected the filename without the FITS extension, and the returned header info is in a dictionary (not a string)
• Fixed bug in test_utils.py, and changed utils.py to make it robust in matching upper and lowercase detector names
• Updated utils.instrument_properties fix MIRI amplifier bounds calculation when omitting reference pixels

New Features

Project & API Documentation

• Added API documentation for Bias, Bad Pixel, and Readnoise Monitors

Web Application

• Added webpage that describes how to use the JWQL web app API
• Added webpage that enables a user to query the jwqldb database contents
• Enabled more search options and aesthetic improvements for anomaly query webpage

jwql Repository

• Added support for Python versions 3.7 and 3.8
• Added unit tests for Readnoise Monitor
• Added unit tests for Bias Monitor
• Added unit tests for Bad Pixel Monitor
• Added unit tests for bokeh templating library
• Added webpage for displaying Readnoise Monitor results with bokeh plots
• Added webpage for displaying Bad Pixel Monitor results with bokeh plots

Bug Fixes

Project & API Documentation

• Updated broken links to JWST instrument JDox pages

Web Application

• Fixed various issues that was cuasing authentication through auth.mast to fail

jwql Repository

• Fixed bug that was causing Dark Monitor to crash on recently added apertures from pysiaf
• Fixed several bugs in bokeh templating library
• Fixed bug that was causing unit tests for permissions.py to fail
• Fixed bug that was causing most_recent_search function in Dark Monitor to fail

New Features

Project & API Documentation

• The jwql_web_app PowerPoint presentation has been updated
• The bokeh templating software now has full API documentation
• Updated README and About webpage to reflect changes to development team members

Web Application

• Added webpage to view FITS headers of each extension for a given dataset
• Added webpage for displaying Dark Monitor results with bokeh plots
• Added webpage for viewing contents of a given JWQL database table
• Added webpage for querying and displaying anomaly results
• Added slider bar for easily navigating through integrations in preview image display
• The list of anomalies one can flag in a preview image is now instrument specific

jwql Repository

• The jwql-3.6 conda environment now uses the astroconda channel instead of astroconda-dev
• Added Bias Monitor module, which currently monitors the bias levels for NIRCam
• Added Readnoise Monitor module, which monitors readnoise for all instruments except FGS
• Added Bad Pixel Monitor module, which tracks bad pixels for all instruments
• Cron job logs now include a print out of the complete conda environment being used

Bug Fixes

Project & API Documentation

• Fixed broken link to numpydoc docstring convention in Style Guide

New Features

Project & API Documentation

• Added slides from July 2019 TIPS presentation to presentations/ directory

jwql Repository

• Updated dark monitor to support all five JWST instruments, instead of only NIRCam
• Changed the jwql-3.5 and jwql-3.6 conda environments to be more simple and to work on Linux distributions
• Added library code for creating instrument monitoring bokeh plots with new bokeh templating software

Bug Fixes

Project & API Documentation

• Fixed various bugs that were causing the sphinx API documentation to crash on ReadTheDocs

New Features

Project & API Documentation

• Updated README to include instructions on package installation via pip.

Web Application

• Updated all webpages to conform to Web Application Accessibility Guidelines.
• Upgraded to django version 2.2.
• bokeh is now imported in base template so that the version being used is consistent across all HTML templates.

jwql Repository

• The jwql package is now available on PyPI (https://pypi.org/project/jwql/) and installable via pip.
• Updated Jenkins configuration file to include in-line comments and descriptions.
• Added utils function to validate the config.json file during import of jwql package.
• Added support for monitoring contents of the jwql central storage area in the filesystem monitor.

Bug Fixes

Web Application

• Fixed position error of JWQL version display in footer.

jwql Repository

• Fixed spelling error in dark monitor database column names.
• Fixed dark monitor to avoid processing files that are not in the filesystem.

New Features

Project & API Documentation

• Updated the notebook exemplifying how to perform an engineering database (EDB) telemetry query.
• Updated the README for the style_guide directory.

Web Application

• Added form on preview image pages to allow users to submit image anomalies.
• Added buttons for users to download the results of EDB telemetry queries as CSV files.
• Enabled users to search for or navigate to program numbers without requiring leading zeros (i.e. "756" is now treated equivalently to "00756").
• Enabled authentication for EDB queries via the web login (rather than requiring authentication information to be present in the configuration file).
• Added custom 404 pages.
• Added adaptive redirect feature so that users are not sent back to the homepage after login.
• Added more descriptive errors if a user tries to run the web application without filling out the proper fields in the configuration file.

jwql Repository

• Replaced all EDB interface code within jwql with the new jwedb package<https://github.com/spacetelescope/jwst-dms-edb>_.
• Fully incorporated Python 3.5 testing into the Jenkins test suite.

Bug Fixes

Web Application

• Fixed bug in which dashboard page would throw an error.
• Fixed incorrect dashboard axis labels.

New Features

Project & API Documentation

• Added guidelines to the style guide for logging the execution of instrument monitors
• Added example useage of logging in the example.py module

Web Application

• Modified various web app views to enable faster loading times
• Modified archive and preview image views to only display data for an authenticated user
• Added views for MIRI and NIRSpec Data Trending Monitors, which monitors the behavior of select MIRI and NIRSpec Engineering Database mnemonics over time

jwql Repository

• Added Dark Monitor module, which monitors the dark current and hot pixel populations for each JWST instrument
• Added software for producing MIRI and NIRSpec Data Trending Monitors (described above)
• Modified generate_preview_images module to support the creation of preview images for stage 3 data products
• Refactored monitor_filesystem to utilize PostgreSQL database tables to store archive filesystem statistics
• Configured codecov for the project. The project homepage can be found at https://codecov.io/gh/spacetelescope/jwql
• Modified logging_functions module to enable dev, test, and production logging environments
• Added convenience decorator to logging_functions module to time the execution of a function or method
• Modified monitor_cron_jobs module to make use of updated logging_functions

Bug Fixes

Web Application

• Fixed API views to only return the basenames of file paths, instead of full directory names

jwql Repository

• Fixed logging_functions module to properly parse new format of INSTALL_REQUIRES dependency in setup.py for logging system dependencies and their versions
• Fixed Jenkinsfile to not allow for one failed unit test in Jenkins builds