HiCExplorer Versions Save

Thanks @pavanvidem!

What's Changed

• Allow chicAggregateStatistic.py to to extract the aggregated data from the views.hdf based on differential.hdf or differential_target.bed. Now the BED may have the target name in the 4th column. In that case, the aggregation is done per target.
• Allow hicCorrectMatrix.py to write filtered out regions to a BED file

by @pavanvidem in https://github.com/deeptools/HiCExplorer/pull/875

New Contributors

• @pavanvidem made their first contribution in https://github.com/deeptools/HiCExplorer/pull/875

Full Changelog: https://github.com/deeptools/HiCExplorer/compare/3.7.3...3.7.4

• Maintenance update for HiCExplorer to keep up to date with APIs of dependencies
• Add the polarization ratio to the output of hicCompartmentalization. Thanks @contessoto.

• Fixing a bug in hicHyperoptDetectLoops concerning the validation of the loop locations
• Fixing a bug in hicValidateLocations concerning the computation of TADs
• Adding the -v option for the version for hicInfo
• Adding documentation concerning the native file formats of HiCExplorer
• Fixing the way A/B compartments are computed based on Lieberman-Aiden 2009. The intermediate Pearson matrix is not used anymore.

The promised functions for the strand information in 3.7 were not part of the release due to confusion in merging multiple branches. Therefore they are now included in the 3.7.1 release:

• hicAggregateContacts: Option to consider the strand orientation (#633)
• hicAverageRegions: Option to consider the strand orientation (#633)

Additionally, a small bug fix in hicInfo concerning the correct sum of the interaction matrix.

• A new TAD prediction method: hicTADClassifier and a method to train own classifiers hicTrainTADClassifier. Thanks @AlbertLidel for the design and implementation.
• New file formats for capture Hi-C modules. chicViewpoint, chicSignificantInteractions, chicAggregateStatistic and chicDifferentialTest use now HDF5 based files. A new export script chicExportData is provided to retrieve text files from these HDF5 files.
• Implementing a few feature requests:
  • hicPlotMatrix: TADs can be visualized with hicPlotMatrix
  • hicAdjustMatrix is able to remove intra- or inter-chromosomal contacts (#664, #704)
  • hicValidateLocations: An option to validate TADs and to use additional data stored in cool matrices
  • hicPCA: Adding a function to select which eigenvector should be used for the output (#669)
  • hicConvertFormat: Adding the function to export hicpro file format and to import 2D-text files.
  • hicFindRestSites: Support of multiple restriction cut sequences (#659)
  • hicPlotMatrix: Option for loop locations spanning more than one bin to define if the start, center or end should be used for plotting (#640)
  • hicInterIntraTAD: A new script to compute the ratio of inter and intra TAD. (#404)
  • hicAggregateContacts: Option to consider the strand orientation (#633)
  • hicAverageRegions: Option to consider the strand orientation (#633)
  • hicCompareMatrices: An option to not normalize the matrices before the computation. (#677, #645) Thanks @lldelisle
  • hicDifferentialTAD: Adding rank sum statistics to the output (#728, #727). Thanks @dawe
  • hicPlotDistVsCounts: Adding a function to plot the counts vs distance in TAD regions. (#696) Thanks @lldelisle
• Bug fixes:
  • hicCorrectMatrix: A bug that lead to wrong correction factors for the KR correction for cool files (#724)
  • hicDifferentialTAD: Solved multicore issue related to skipping data at the start and end of chromosomes (#725, #685)
  • hicHyperoptDetectLoops: Added an option to set if the chr prefix should be added or removed (#723)
  • hicPCA: Solving an issue if the region defined by the gene track is larger the region stored in the interaction matrix (#655, #710, #716, #719)
  • hicPCA: Fixing a bug where the masking of bins was automatically applied which lead to differing matrix dimensions for the e.g. the Pearson correlation matrices (#618)
  • hicBuildMatrix: Solving a bug if multiple restriction cut sites have the same dangling ends (#720)
  • hicBuildMatrix: Solving a bug that the parameter --removeSelfLigations was always set to true. Changed parameter name to --keepSelfLigations to keep the functionality. If the parameter is not set, the self ligations are removed.
  • hicBuildMatrix: If a region is specified, only the restrictionCutSite file information for that region is loaded to save memory (#646)
  • hicConvertFormat: Fixing a bug to copy the genome annotation information in the case of a cool to cool file conversion (#657)
  • hicCorrelate: Correcting the range of colors for the heatmap (#585)
  • hicCompartmentalization: Fixed index bug (#635, #637) Thanks @LeilyR
• Updating hicBuildMatrix to be able to work with biopython versions > 1.77. Thanks @lldelisle (#691)

• hicAggregateContacts, thanks @LeilyR:
• hicAggragateContact has been updated to be able to handle inter chromosomal contacts as well as intra chromosomal contacts
• Added scikit-learn to dependencies
• hicBuildMatrix: Fixing another bug concerning scaffolds without any restriction enzyme cut site
• Updated dependencies
• Adding default values to the documentation. Thanks @lldelisle
• hicTransform: Fixing a bug in case one of the intermediate matrices is empty
• Official Python 3.8 support: Manually setting 'fork' as the start method for multiprocessing because the default on macOS was set to 'spawn'

• Bug fix release:
  • Reads from scaffolds without any restriction enzym cut site are considered as 'same fragment'. An appearance of such a read will not lead to a crash anymore
  • Minor documentation updates

• Bug fix release:
  • enforcing version 15 of HiCMatrix. Version 14 had a bug concerning the application of the correction factors of cool files. See issue #595
  • Fixing a bug in hicDetectLoops in single-core mode. Thanks @alecw
  • Fixing a bug in hicDifferentialTAD concerning multiple chromosomes in the bed file. See issue #587
  • Updating dependencies to newest versions, except biopython. Forcing here <1.77 because their API change is breaking our source code. See issue #609
  • Fixing #596
• Changing internal structure of the docs. Navigation should be better now.

• patch hicCorrelate
• hicBuildMatrix: Better help text
• patch for hicPlotMatrix if matrix does not start at 0 and --region is used
• bug fix for remove option in hicAdjustMatrix

• Major update for hicDetectLoops: Results are now closer to HiCCUPS, it is faster and needs less memory.
• hicHyperoptDetectLoops: New tool to compute the best parameter setting if the number of expected loops is given and a protein boundary file (e.g. CTCF on mammals) is provided
• hicHyperoptDetectLoopsHiCCUPS: New tool to compute the best parameter setting for Juicers HiCCUPS. HiCCUPS and all its dependencies are not part of HiCExplorer and must be provided by the user. Number of expected loops and a protein boundary file (e.g. CTCF on mammals) must be provided.
• hicMergeDomains: New tool to compute a merge of TADs computed on different resolutions. Moreover it provides a cleaning of the boundaries with the help of a protein peak file, and the hierarchical dependencies of TADs can be plotted. This tool is the result of the Bachelor thesis from Sarah Domogalla (@SarahMaria27). Thanks for your contribution!
• hicDifferentialTAD: New tool to compute differential TADs between two samples
• Bug fix for hicFindTADs: The format of the intermediate z-score matrices are now depending on the format of the input Hi-C matrix
• Bug fix for chic*-modules: Fixate range is now correct applied.
• Bug fix for hicPlotMatrix:
  • multiple bigwig tracks in the vertical setting are now supported
  • correct plot of bigwig if the given matrix does not start at the beginning of the chromosome
  • new parameters 'increaseFigureWidth' and 'increaseFigureHeight' to add more space to the plot if multiple bigwigs are plotted. Adjust this parameter to correct the plot of the heatmap which may be not quadratic anymore.
  • restriction of the loop regions to the user given range. This effects especially SVGs that will now contain less objects as before.
• New feature for hicBuildMatrix:
  • multiple restriction cut sequences, dangling ends and restriction cut sites files are now supported
  • restriction cut sequences, dangling ends and restriction cut sites files are now mandatory parameters. This is now enforced to guarantee a correct detection of self ligations and self circles
• hicPrepareQCreport: New support for multiple dangling ends
• hicQuickQC: restriction cut sequences, dangling ends and restriction cut sites files are now mandatory parameters
• hicFindRestSite: gz support for fasta file
• hicAggregate: interactions between two bed files by comparing every row in the first bed file with its corresponding row in the second file. (issue #390)
• hicAdjustMatrix: fix #341, 454
  • fixed --action remove: it actually remove the part from the matrix previously was masking it
  • the case where the end of the chromosome need to be removed.
• New Azure testing for the general test cases, the trivial ones run on travis. Makes it faster, maybe around half an hour to 45 mins instead of multiple hours of CI