Feature Calculation Changes

• Remove feature SumVariance, rename SumVariance2 to SumVariance. SumVariance reflected the formula as is defined in the paper by Haralick et al [1]_. However, the variance is calculated by subtracting the entropy as opposed to subtracting the average, most likely due to a typo('f8' instead of 'f6'). SumVariance2 reflected the formula where the average is subtracted and is retrained as the only SumVariance. (#233)
• Redefine features Elongation and Flatness as the inverse of the original definition. This prevents a returned value of NaN when the shape is completely flat. (#234)
• In certain edge cases, the calculated maximum diameters may be too small when calculating using the python implementation. This is corrected by the C extension and a warning is now logged when calculating these features in python. N.B. As of this change, maximum diameter is not available for calculation in full-python mode (#257)
• For certain formulas, a NaN value is returned in some edge cases. Catch this and return a predefined value instead. Document this behaviour in the docstrings of the features affected. (#248)

New Features

• Add Region of Interest checks. (#223, #227)
• Add variable column support for batch input file (#228)
• Add Docker support (#236)

Bug fixes

• Instantiate output with input in commandlinebatch
• Correct Np when weighting is applied in GLRLM (#229)
• Update CSV generators to reflect variable number of columns for input CSV in batch processing (#246)
• Return corrected mask when it had to be resampled due to geometry mismatch errors (#260)

Requirements

• Remove tqdm requirement (#232)
• Reorganize requirements, with requirements only needed during development moved to requirements-dev.txt (#231)

Documentation

version 1.2.0 docs

• Update feature docstrings, making them more easily adaptable for article supplements (#233)
• Add FAQ concerning the cmatrices lib path (#233)
• Add developer install step to documentation (#245)
• Remove use of sudo (#233)
• Fix subclass name in feature class signature (section "Developers")
• Add subsection on customizing the extraction to the "Usage" section (#252)
• Remove SimpleITK installation workaround, this is no longer needed (#249)
• Add a changelog to keep track of changes and integrate this into the auto generated documentation (#255)

Examples

• Add pandas example, showing how to process PyRadiomics output/input using the pandas library (#228)

Internal API

• Add function to get or download test case (#235)
• Rewrite C Extension algorithm for GSLZM. Instead of searching over the image for the next voxel when growing a region, store all unprocessed voxels in a stack. This yields a significant increase in performance, especially in large ROIs. Requires slightly more memory (1 array, type integer, size equal to number of voxels in the ROI) (#257)
• Implement C extension for calculation of maximum diameters. (#257)

Cleanups

• Restructure repository (#254)
  • Move jupyter notebooks to separate root folder (root/notebooks)
  • Move example script to separate root folder (root/examples), with example settings in separate subfolder (root/examples/exampleSettings)
  • bin folder now only contains support scripts for the core code (i.e. generators for input files for batch processing and scripts to generate new baselines or to resample a mask to the image geometry)

Feature changes

• Correct error in formula for Compactness1. N.B. Baseline updated! #218
• Remove feature Roundness, as this feature is identical to feature sphericity, but uses different implementation for surface area calculation (all implemented in SimpleITK) #218

Documentation

Restructure documentation #216

• Split package documentation into "Pipeline Modules" (all non-feature-class modules) and "Feature Definitions" (feature class modules)
• Add developers section with documentation on how to implement new filters, feature and feature classes.
• Add FAQ section with some trouble shooting tips
• Rename some GLSZM features, this is to make them more consistent with GLRLM features, which are similar, but calculated on a different matrix
• Add documentation for Elongation and Flatness
• Document mathematical correlation between various Shape features.
• The version specific documentation is available here.

Bugfixes

• Add some missing python 3 compatibility lines to the supporting script addClassToBaseline and command line script pyradiomicsbatch #210 #214
• Fix bug when loading image as file path and mask as SimpleITK object. #211
• Add sphinx_rtd_theme to requirements (needed to build documentation). #222
• Change location of parameter schema files. These files are otherwise not included in the wheel distribution. #221

Enhancements

• Update logging with more extensive debug logging and more informative info log messages. #220
• Replace parameter verbose with output printing implemented in logging. Control verbosity level to output (stderr) by calling radiomics.setVerbosity(level), where level determines the verbosity level (as defined in python logging). This prints out the requested levels of the log messaging, where process reports with parameter verbose are now classified as INFO-level messages (i.e. specify INFO or DEBUG to enable these). N.B. parameter verbose is not longer supported and will throw an error if passed in the parameter file #220
• Add feature class and input image type checks in featureextractor when changing these settings. #213
• Remove usage of eval (replaced by implementations of getattr), this is a more secure approach. #216
• Change handling of cases where max(X) mod binwidth = 0 during image discretization. These used to be assigned to topmost bin, but this produces unexpected behaviour (i.e. in range 1, 2, 3, 4, 5 with binwidth 1, value 5 would be discretized to 4 in stead of 5). Value now assigned is topmost bin + 1 (in concordance with default behavior of numpy.digitize) #219
• Add forced 2D extraction (as alternative to resampling for handling anisotropy in voxels spacing) #215
• Enable specification of distances between neighbors for texture matrices. #215
• Change default value for voxelArrayShift (from 2000 to 0), this is to prevent unknowingly using a too large shift when not necessary. Document effect of this parameter in the first order formulas affected. #219

Style

• Remove unused variable (self.provenance_on in featureextractor, this value is now replaced by a customizable setting)
• Define default settings in featureextractor in a separate function. This is to ensure consistency in applied default settings, as well as make them easily available outside of featureextractor #224 216
• Update reference in documentation. #224

Implementation of C Extensions

Stats:

Profiling 5 testcases using python 2.7

• GLCM 6913 ms -> 3 ms
• GLSZM 12064 ms -> 58 ms
• GLRLM 1850 ms -> 10 ms
• Surface Area 3241 -> 1 ms

Code:

• _cmatrices: C implementation for matrix computation associated with GLCM, GLSZM and GLRLM features.
• _cshape: C implementation for Shape surface computation.
• tests/test_cmatrices: testing for matrix equality: Tests whether the python generated matrix is equal to the matrix generated by the C implementations (allows for machine precision errors). Also compares the python and C generated surface computations similarly.

Details

• Standardize function names for calculating matrices in python and with C extensions to _calculateMatrix and _calculateCMatrix, respectively.
• Add docstring to C modules.
• Use of C implementation optional. At initialization, the package tries to use C, but if loading has failed, or calculation is forced to python, python is used. Note that the import of _cmatrices is done after initialization of logger. This ensures error will be logged if import fails. This can be controlled through the enableCExtensions setting, which can be provided in a parameter file or as part of the kwargs dictionary in featureextractor.
• GLSZM: Use "char" datatype for mask. (It is signed char in GLSZM).
• C code is consistent with C89 convention. All variables (pointers for python objects) are initialized at top of each block.
• GLSZM: Use calloc and free for the temporary array holding the calculated zones.

Optimizations

• GLSZM:
  • Define temporary array for holding the calculated zones. During calculation, the matrix must be able to store all possible zones, ranging from zone size 1 to total number of voxels (Ns), for each gray level (Ng). In this case, the GLSZM would be initialized with size Ng * Ns, which is very memory intensive. Instead, use a temporary array of size (Ns * 2) + 1, which stores all calculated zones in pairs of 2 elements: the first element holds the gray level, the second the size of the calculated zone. The first element after the last zone is set to -1 to serve as a stop sign for the second function, which translates the temporary array into the final GLSZM, which can be directly initialized at optimum size.
  • Uses "while" loops. This allows to reduce the memory usage. We observed that with recursive functions it was 'unexpectedly' failing.
  • Optimized search that finds a new index to process in the region growing.

Associated issues: #106

Associated PRs: #158, #200, #202

Python 3 Compatibility:

• As of this version, PyRadiomics is compatible with python 2.7 and python >= 3.4. This is acchieved through use of third-party package six (added to requirements.txt). The changes to ensure this compatibility do not change the user interface.

Associated issues: #178, #204

Associated PRs: #188, #194, #196, #205

Normalization

• Add functionality to imageoperations for normalizing image intensity values. If enabled, the image is normalized by centering it at the mean with standard deviation. Normalization is based on all gray values in the image, not just those inside the segementation. It is enabled or disabled by setting the parameter normalize.
• Values can be scaled using parameter normalizeScale, which multiplies the normalized intensity values by the value of this paramter.
• Outliers can be removed by setting the removeOutliers parameter. If set, all values differing more than n * (standard deviation) from the mean are set to n * (standard deviation), where n is equal to the value of this parameter. This is done after normalizing and before applying the scale. If this last parameter is omitted, no outliers are removed.
• If enabled, normalization is applied after loading the image, before any resampling or application of filters.

Associated PRs: #209

Parameterization:

• Add parameter for controlling the additional output (enable/disable by setting additionalInfo parameter)
• Add parameter for controlling whether or not to calculate in full-python mode (enable/disable by setting enableCExtensions parameter).

Associated PRs: #190, #202

Documentation:

• Documentation on installation and usage is upgraded, with the addition of an embedded instruction video (in section "Usage", cued at the section on usage examples).
• Updated contact information to point to the google groups.
• Updated the classifiers in the setup script to reflect the more advanced status of Pyradiomics.
• The version specific documentation is available here.

Associated PRs: #187, #193

Continuous Testing:

Upgrade to include other python versions and platforms

• Tests are passed for Linux, Windows and Mac, using python 2.7, 3.4 and 3.5, in 32- and 64-bit builds.

Associated PRs: #183, #191, #199

Style:

• Testing output is upgraded to ensure unique feature names.
• getFeatureClasses() and getInputImageTypes() functions are moved from featureextractor to global radiomics namespace. This enumerates the possible feature classes and filters at initialization of the toolbox, and ensures feature classes are imported at initialization.

Associated PRs: #190, #195, #197, #198

Additional Functionality:

• Added 2 commandline scripts: pyradiomics and pyradiomicsbatch, which enable feature extraction directly from the commandline. For help on usage, run script with “-h” argument.

Changed Functionality:

• Different implementation of the various filters. No changes to calculation, but has a changed signature.

  N.B. This results in inputImages to be differently defined (different capitalization, e.g. "orginal" should now be "Original"). See documentation for definition of inputImages (featureextractor section).

Bugfixes:

• Bugfix of a numpy slicing error in application of wavelet filters. This error caused the derived image to be erroneously rotated and flipped, with misaligned mask as a result.

Documentation:

• Update on the documentation, reflecting recent changes in the code.
• Added additional references and updated installation and usage section.

• Initial release of PyRadiomics
• Full python calculation (C matrices branch not stable and reserved for later release)
• Published documentation at readthedocs