Cheng Li Pyramid Versions Save

New reranker implementation with sigmoid, weak monotonicity, and KL pre-training

Allow users to set minimum and maximum predicted set sizes

Add an option to disallow empty predictions

Fix an out-of-memory issue

• App3 is renamed to AppBRGB
• AppBRGB and AppBRLR use similar properties files and output structures
• Make it easier for different classifiers to share input data
• Split data into train/validation/calibration/test
• Add CTAT tuning and reporting
• Add reranker calibrator and prediction
• Support additional features in calibrator
• Support instance weights in classifier training

Fix the null pointer exception bug in gradient boosting training

App3: reduce label calibration memory usage

App3: cardinality based calibration

App3:

• boosting speedup via adaptive feature scan and minibatch life span
• allow users to set random seed in gradient boosting training
• fix a bug in unordered span near query for ngrams with duplicate words
• add an option to filter out ngrams with duplicate words
• allow users to specify whether the ngram matching is in order or out of order
• fix a bug that causes the slowness in categorical feature matching
• add an option to skip rare labels
• add an option to sort docs in report by confidence or mistake
• introduce a validation set for early stopping and probability calibration
• better probability calibration for individual label probabilities

GB regressor:

• add logger
• allow users to set report folder name
• add an option to skip PMML

Special thanks to Jinpeng Wang for helping with the span near query fix.

App3: reduce isotonic regression calibration memory usage