Ludwig Versions Save

Low-code framework for building custom LLMs, neural networks, and other AI models

v0.10.3

3 weeks ago

What's Changed

Replace Slack links with Discord links. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3988
Allow image bytes type during preprocessing by @vijayi1 in https://github.com/ludwig-ai/ludwig/pull/3971
Fix for 'upload_to_hf_hub()' path mismatch with 'save()' by @sanjaydasgupta in https://github.com/ludwig-ai/ludwig/pull/3977
Minor change to fix the incorrect response truncation by @amankhandelia in https://github.com/ludwig-ai/ludwig/pull/3986
Pin minimum transformers to 4.39 to reduce Llama/Gemma memory pressure by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3976
Actually add support for RSLoRA and DoRA by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3984

New Contributors

@amankhandelia made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3986

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.10.2...v0.10.3

v0.10.2

1 month ago

What's New

Add support for RSLoRA and DoRA by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3948 To enable, set the corresponding flag to true in config (can be used in conjunction):

    adapter:
        type:  lora
        use_rslora:  false
        use_dora: false

Add support for eval batch size tuning for LLMs on local backend by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3957 To enable, set "eval_batch_size" to "auto" in the trainer section:

    trainer:
        eval_batch_size:  auto

Enable loading model weights from training checkpoint by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3969 To enable, pass from_checkpoint=True to LudwigModel.load():

LudwigModel.load(model_dir, from_checkpoint=True)

Full Changelog

Save ludwig-config with model-weights in output directory by @sanjaydasgupta in https://github.com/ludwig-ai/ludwig/pull/3965
Add unit tests for image utils unet functions by @vijayi1 in https://github.com/ludwig-ai/ludwig/pull/3921
fix: Update imdb_genre_prediction dataset yaml to match dataset by @jeffreyftang in https://github.com/ludwig-ai/ludwig/pull/3944
Fix kube apt source by @noyoshi in https://github.com/ludwig-ai/ludwig/pull/3952
Temporarily disable expensive text metrics by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3954
[MAINTENANCE] Comment Out PyTorch Nightly Test by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3955
[BUGFIX] Fixing integration test failures. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3959
[MAINTENANCE] Use latest version of psutil library. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3956

New Contributors

@sanjaydasgupta made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3965

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.10.1...v0.10.2

v0.10.1

2 months ago

What's Changed

Fixed a critical bug in Gemma model fine-tuning that prevented the model from learning when to stop generation. This is accomplished by using eos token in target tensor for instruction-tuning by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3945

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.10.0...v0.10.1

v0.10.0

2 months ago

What's Changed

Add Phi-2 to model presets by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3912
Add default LoRA target modules for Phi-2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3911
Add support for prompt lookup decoding during generation by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3917
Pin pyarrow to < 15.0.0 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3918
Add unet encoder-decoder and image output feature by @vijayi1 in https://github.com/ludwig-ai/ludwig/pull/3913
fix: Add Nested quantization check by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3916
fix typo in save_dequantized_base_model log statement by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3923
Add example for base model dequantization/upscaling by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3924
fix: Always return a list of quantization bits values from get_quantization by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3926
fix: set use_reentrant to True to fix Mixtral-7b bug by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3928
Disabling AdaptionPrompt till PEFT is fixed. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3935
Add default LoRA target modules for Gemma by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3936
Pinning transformers to 4.38.1 or above in order to ensure support for Gemma by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3940
Ludwig release version change by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3941

New Contributors

@vijayi1 made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3913

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.9.3...v0.10.0

v0.9.3

3 months ago

What's Changed

[MAINTENANCE] Use Trusted Publishers credentials instead of User/Password for uploading releases to PyPi by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3892
Add support for official microsoft/phi-2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3880
Ensure correct padding token for Phi and Pythia models by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3899
Enable AdaLoRA tests for LLM adapter by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3896
Cast LLMEncoder output to torch.float32, freeze final layer at init. by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3900
Enable IA3 adapters in LLMEncoder by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3902
[Maintenance] Remove torch nightly pin by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3903
Pin deepspeed to < 0.13 and pandas to < 2.2.0 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3906
Add batch size tuning for LLMs by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3871

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.9.2...v0.9.3

v0.9.2

3 months ago

What's Changed

fix: Handle missing and unexpected keys during LLMEncoder state dict load by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3841
fix: Add name and description classmethods to IA3Config by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3844
Improve IA3 long description by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3845
fix: Handle missing and unexpected keys during LLMEncoder state dict load, part 2 by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3843
Update description for max_new_tokens to explain the dynamic setting behavior in our docs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3847
Add default LoRA target modules for Mixtral and Mixtral instruct by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3852
QOL: Fail config validation if a user tries to use ECD with a text output feature and an LLM encoder. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3792
Pin minimum transformers to 4.36 for Mixtral and Phi support by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3854
Revert hack that leads to OOM during fine-tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3858
Add support for exporting models to Carton by @VivekPanyam in https://github.com/ludwig-ai/ludwig/pull/3797
[Maintenance] Bump minimum tokenizers to 0.15 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3856
fix: correct typo in FeatureCollection by @dennisrall in https://github.com/ludwig-ai/ludwig/pull/3863
Convert test main script in algorithm_utils to unit test by @dennisrall in https://github.com/ludwig-ai/ludwig/pull/3864
Allow hyperopt config to be loaded from a file by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3865
fix: unify ludwig training set metadata and hf pad token by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3860
Add a utility to detect LLM usage in a config by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3869
Early stop training if model weights have nan or inf tensors by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3740
Scrub credentials from model_hyperparameters.json and description.json by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3866
[Maintenance] Bump minimum torch version to 2.0.0 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3873
[Maintenance] Fix docker images by pinning ray==2.3.1, daft==0.1.20, unpinning proto, and using torch 2.1.1. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3872
[BUGFIX] Guard against UnicodeEncodeError when saving validation results in Google Colab environment by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3875
Docker image fixes part 2: pin to torch==2.1.0, add dependency for urllib<2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3877
Add custom prepare_for_trianing logic to ECD model for LLM encoder adapter initialization by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3874
qol: Fix some lints. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3868
[Maintenance] Docker Image Fix part 3: fix torchaudio 2.1.0 dependencies by installing libsox-dev and update API by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3879
Add streaming support for zero shot inference by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3878
[Maintenance] Remove torchdata pin for nightly install by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3855
Add per-step token utilization to tensorboard and progress tracker. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3867
Set use_reentrant to False for gradient checkpointing by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3882
[BUGFIX] Pinning torch nightly to January 13, 2024 to avoid AttributeError by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3885

New Contributors

@VivekPanyam made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3797

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.9.1...v0.9.2

v0.9.1

4 months ago

What's Changed

fix: Handle missing and unexpected keys during LLMEncoder state dict load by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3841
fix: Add name and description classmethods to IA3Config by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3844
Improve IA3 long description by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3845
bump ludwig version by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3846

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.9...v0.9.1

v0.9

4 months ago

What's Changed

int: Rename original combiner_registry to combiner_config_registry, update decorator name by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3516
Add mechanic to override default values for generation during model.predict() by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3520
[feat] Support for numeric date feature inputs by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3517
Add new sythesized response column for text output features during postprocessing by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3521
Disable flaky twitter bots dataset loading test. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3439
Add test that verifies that the generation config passed in at model.predict() is used correctly. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3523
Move loss metric to same device as inputs by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3522
Add comment about batch size tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3526
Ensure user sets backend to local w/ quantization by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3524
README: Update LLM fine-tuning config by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3530
Revert "Ensure user sets backend to local w/ quantization (#3524)" by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3531
Improve observability during LLM inference by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3536
[bug] Pin pydantic to < 2.0 by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3537
[bug] Support preprocessing datetime.date date features by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3534
Remove obsolete prompt tuning example. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3540
Add Ludwig 0.8 notebook to the README by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3542
Add effective_batch_size to auto-adjust gradient accumulation by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3533
Refactor evaluation metrics to support decoded generated text metrics like BLEU and ROUGE. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3539
Fix sequence generator test. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3546
Revert "Add Cosine Annealing LR scheduler as a decay method (#3507)" by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3545
Set default max_sequence_length to None for LLM text input/output features by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3547
Add skip_all_evaluation as a mechanic to skip all evaluation. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3543
Roll-forward with fixes: Fix interaction between scheduler.step() and gradient accumulation steps, refactor schedulers to use LambdaLR, and add cosine annealing LR scheduler as a decay method. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3555
fix: Move model to the correct device for eval by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3554
Report loss in tqdm to avoid log spam by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3559
Wrap each metric update in try/except. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3562
Move DDP model to device if it hasn't been wrapped yet by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3566
ensure that there are enough colors to match the score index in visua… by @thelinuxkid in https://github.com/ludwig-ai/ludwig/pull/3560
Pin Transformers to 4.31.0 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3569
Add test to show global_max_sequence_length can never exceed an LLMs context length by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3548
WandB: Add metric logging support on eval end and epoch end by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3586
schema: Add prompt validation check by @ksbrar in https://github.com/ludwig-ai/ludwig/pull/3564
Unpin Transformers for CodeLlama support by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3592
Add support for Paged Optimizers (Adam, Adamw), 8-bit optimizers, and new optimizers: LARS, LAMB and LION by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3588
FIX: Failure in TabTransformer Combiner Unit test by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3596
fix: Move target tensor to model output device in check_module_parameters_updated by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3567
Allow user to specify huggingface link or local path to pretrained lora weights by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3572
Add codellama to tokenizer list for set_pad_token by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3598
Set default eval batch size to 2 for LLM fine-tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3599
[CI] Explicitly set eval batch size in determinism tests, introduce a new integration test group, and exclude slow tests. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3590
[CI] Run sudo apt-get update in GHAs. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3608
Store steps_per_epoch in Trainer by @hungcs in https://github.com/ludwig-ai/ludwig/pull/3601
Updated characters, underscore and comma preprocessors to be TorchScriptable. by @martindavis in https://github.com/ludwig-ai/ludwig/pull/3602
[CI] Deflake: Explicitly set eval batch size for mlflow test. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3612
Fix registration for char error rate. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3604
fix: Load 8-bit quantized models for eval after fine-tuning by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3606
Add Code Alpaca and Consumer Complaints Datasets by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3611
Add support for gradient checkpointing for LLM fine-tuning by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3613
Bump min support transformers to 4.33.0 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3616
[CI] Fix failing tests on master by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3617
Eliminate short-circuiting for loading from local by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3600
Refactor integration tests into matrix by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3618
fix: Check underlying model device type when moving 8-bit quantized models to GPU at eval by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3622
Fixed range validation for text generation penalty parameters by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3623
Update comment for predict to update Ludwig docs by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3535
Avoid deprecation warnings on pandas Series.fillna by @carlogrisetti in https://github.com/ludwig-ai/ludwig/pull/3631
QoL: Default to using fast tokenizer for Llama models by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3625
fixed typo in EfficientNet's model variant from v2_ to v2_s by @saad-palapa in https://github.com/ludwig-ai/ludwig/pull/3628
Add pytorch profiler and additional tensorboard logs for GPU memory usage. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3607
Pin minimum transformers version to 4.33.2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3637
Add function to free GPU memory by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3643
❗ Enable LLM fine-tuning tests when no quantization is specified by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3626
Add check to ensure selected backend works with quantization for LLMs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3646
[CI] Use a torch-nightly-compatible version of torchaudio by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3644
Set do_sample default to True by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3641
FIX: Failure in audio feature related test by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3651
Remove unnecessary peft config updating by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3642
FIX: docker build error for ludwig-gpu by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3658
Exclude getdaft on Windows by @carlogrisetti in https://github.com/ludwig-ai/ludwig/pull/3629
Add daft back for windows since the wheels are now officially published by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3663
fix: The final batch of an epoch is skipped when batch size is 1 by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3653
Place metric functions for BLEU and Rogue on correct devices when using multiple GPUs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3671
Remove duplicate metrics by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3670
Increment epochs based on last_batch() instead of at the end of the train loop. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3668
[FEATURE] Support Merging LoRA Weights Into Base Model (Issue-3603) by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3649
[FEATURE] Include Mistral-7B model in list of supported base models by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3674
[MAINTENANCE] Partially reconcile type hints, fix some warnings, and fix comments in parts of the codebase. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3673
Improve error message for when an LLM base model can't be loaded. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3675
Fix eos_token and pad_token issue by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3667
FIX: error with nightly CI tests for test_resize_image by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3678
[BUGFIX] Remove spurious test directory at the end of the test_llm.py::test_local_path_loading test run by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3680
Add per-device logging to tensorboard by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3677
Fix dynamic generation config load during model.predict by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3666
[CI] Ensure that mlflow callback cleans up background-saving threads on trainer teardown. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3683
fix: temporarily remove config validation check for backend by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3688
fix: Failing test for backend with quantization by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3689
[BUGFIX] Ensure that full base models and not only adapter weights get saved when merge_and_unload is set by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3679
Add Ludwig Star History to README by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3696
Use sphinx for all docstrings in api.py by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3693
Fix binary variables being visualized as 0 and 1 by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3691
[MAINTENANCE] Fix the linting warnings in two backend component classes. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3698
[BUGFIX] Pin deepspeed<0.11, skip Horovod tests by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3700
Unpin deepspeed following fix in v0.11.1 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3706
Move on_epoch_end and epoch increment to after run_evaluation loop. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3690
Remove model_load_path from experiment by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3707
[FEATURE] Allow typehints without the quotes. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3699
Add consumer complaints generation dataset by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3685
Set the metadata only during first training run by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3684
Add ability to upload Ludwig models to Predibase. by @martindavis in https://github.com/ludwig-ai/ludwig/pull/3687
Log additional per-GPU information in model metadata files and GPU utilization on tensorboard. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3712
QoL: Only log generation config being used once at inference time by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3715
[MAINTENANCE] Adding typehint annotations in backend and data components and fixing mypy errors. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3709
QoL: Limit top-level trainer logging messages such as saving model or resuming model training to main coordinator process by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3718
Add sample_size as a global preprocessing parameter by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3650
QOL: Update recommended vscode settings. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3717
Add new fine-tuning notebooks to README by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3722
Dynamically set max_new_tokens based on output feature length, GMSL and model window size by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3713
Fix issue while logging cuda device utilization to tensorboard by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3727
Pin PEFT to PEFT master for Mistral-7B support by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3725
Limit ray versions to keep 2.4 out of the picture. by @trevenrawr in https://github.com/ludwig-ai/ludwig/pull/3731
Set default text preprocessing lowercase behavior to False by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3721
Add functionality to allow direct importing of datasets from Hugging Face by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3664
[MAINTENACE] Improve typehints and use constants instead of common hardcoded string values. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3732
DRAFT: Call .finish() for WandB callback on trainer end. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3710
Fix calculation of dataset size by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3723
[BUGFIX] Make sure that the "hyperopt" directory in the local repository path is deleted after test run is completed by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3736
[BUGFIX] Making integration tests in the group "integration_tests_e" pass in Azure CI by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3737
[CI] Fix test broken by transformers v.4.34.1 by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3739
Raise error if output feature name is in the prompt template by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3743
Retry LLM model downloads from HF Hub with exponential backoff when there is a Read timeout by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3742
Simplify how we set pad token and pad token ID for huggingface tokenizers. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3735
Increment the epoch before running evaluation by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3729
Uncap the torchmetrics version, so we can get fixes in 1.0+. by @trevenrawr in https://github.com/ludwig-ai/ludwig/pull/3741
Fix Consumer Complaints Gen Dataset Config by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3746
[MAINTENANCE] Restrict "fsspec" to not include the latest version "2023.10.0" by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3752
[MAINTENACE] Fixing method overload violations; providing typehints and method argument documentation by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3753
Fix: Prevent memory from ballooning during post-training evaluation by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3756
Update base.py by @jetaudio in https://github.com/ludwig-ai/ludwig/pull/3750
Pin getdaft<0.2 in requirements by @jaychia in https://github.com/ludwig-ai/ludwig/pull/3757
QOL: Fail during preprocessing if max sequence lengths are shorter than the prompt template. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3719
Define initialize_llm() and generate() methods. Remove extra logging in llm.py by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3711
Update ludwig version to 0.9.dev. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3638
fix: Check device type in _initialize_llm before moving to cuda. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3762
[BUGFIX] Pin Transformers version to prevent tests from failing in PEFT by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3766
[MAINTENANCE] Update contributor instructions with additional steps that may be needed for installation to succeed. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3764
fix: adapter checkpoint loading on resume by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3769
fix LudwigProgressBar rendering on model resume by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3770
enh: Add on_checkpoint callback that triggers after every checkpoint even if the model was not evaluated by @jeffreyftang in https://github.com/ludwig-ai/ludwig/pull/3763
[BUGFIX] Un-pin "transformers" version and update file names for saved merged model directory, because issue with "transformers" impacting "peft" has been fixed. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3775
OnnxExporter by @skanjila in https://github.com/ludwig-ai/ludwig/pull/3761
[CI] Upgrade torch on CIs to 2.0.0, 2.1.1, and nightly by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3785
Cleanup: Use existing HFTokenizer to consolidate manual pad token setting. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3774
Update copyright by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3778
Add k eval predictions to mlflow for LLM fine-tuning by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3768
Add eval_steps as a flag by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3776
Fix MLFlow logging for sample outputs during evaluation. by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3786
Add NEFTune implementation for Noised Embedding Instruction Fine-Tuning support by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3744
[MAINTENANCE] Provide links to contributor guide in README.md by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3794
[Troubleshooting] [BUGFIX] Troubleshooting the "slow" tests group in Azure DevOps CI. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3748
feat: add support for tif files by @dennisrall in https://github.com/ludwig-ai/ludwig/pull/3779
QOL: Fix slow tests GHA by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3795
Added target_modules to LoRA config by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3801
[FEATURE] Generalize LoRA Projection Layers Tests by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3802
Add support for dequantizing 4-bit bitsandbytes base models into fp16 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3799
[Maintenance] Fix deprecation warning for CharErrorRate by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3804
Remove dicts from comparison metrics during visualization by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3796
Fix torch nightly tests: Temporarily add torchdata as a dependency for nightly installation by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3808
Log LLM test set evaluation outputs to CSV files under the model log directory, stamped by checkpoint number. by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3793
Introduce CODEOWNERS to ensure at least 1 approval by Ludwig maintainers / the Predibase team by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3809
Add utility function for uploading local folders to huggingface hub by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3805
Fix LLM integration tests that fail because of PEFT 0.7 release by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3810
Add AutoAugmentation to image classification training #3581 by @ethanreidel in https://github.com/ludwig-ai/ludwig/pull/3798
refactor get_total_expected_checkpoint into a function by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3816
Add support for IA3 PEFT Strategy by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3818
Update base LLM preset list to include CodeLlama, Mistral, and Zephyr by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3819
Requirements and import change (ray) by @SarthakNikhal in https://github.com/ludwig-ai/ludwig/pull/3800
Pin Transformers to < 4.36 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3820
Fix epoch computation by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3822
[MAINTENANCE] Support Adapter Model ".safetensors" File Format During HF Upload by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3823
Move reusable LLM model methods to utility functions by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3821
Remove quantization config before saving dequantized weights to disk by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3825
unpin transformers by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3826
[maintenance] small fix in log message for save_dequantized_base_model by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3829
Add support for Phi-1 and Phi 1.5 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3831
hack: workaround torch bug to unblock mixtral fine-tuning by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3830
Add LLM Text Encoder by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3828
pin transformers to patch commit by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3832
enh: Updating requirements to pin huggingface-hub>=0.19.0 by @brightsparc in https://github.com/ludwig-ai/ludwig/pull/3834
Add support for Phi 2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3838
Unpin transformers in favor of 4.36.2 by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3839
Update LLM trainer defaults by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3840
Update ludwig version to v0.9. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3842

New Contributors

@thelinuxkid made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3560
@saad-palapa made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3628
@alexsherstinsky made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3649
@trevenrawr made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3731
@jetaudio made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3750
@ethanreidel made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3798
@SarthakNikhal made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3800

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.8...v0.9

v0.8.6

6 months ago

What's Changed

Add consumer complaints generation dataset by @connor-mccorm in https://github.com/ludwig-ai/ludwig/pull/3685
Set the metadata only during first training run by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3684
Add ability to upload Ludwig models to Predibase. by @martindavis in https://github.com/ludwig-ai/ludwig/pull/3687
Log additional per-GPU information in model metadata files and GPU utilization on tensorboard. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3712
QoL: Only log generation config being used once at inference time by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3715
[MAINTENANCE] Adding typehint annotations in backend and data components and fixing mypy errors. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3709
QoL: Limit top-level trainer logging messages such as saving model or resuming model training to main coordinator process by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3718
Add sample_size as a global preprocessing parameter by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3650
QOL: Update recommended vscode settings. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3717
Add new fine-tuning notebooks to README by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3722
Dynamically set max_new_tokens based on output feature length, GMSL and model window size by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3713
Fix issue while logging cuda device utilization to tensorboard by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3727

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.8.5...v0.8.6

v0.8.5

6 months ago

What's Changed

Add function to free GPU memory by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3643
❗ Enable LLM fine-tuning tests when no quantization is specified by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3626
Add check to ensure selected backend works with quantization for LLMs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3646
[CI] Use a torch-nightly-compatible version of torchaudio by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3644
Set do_sample default to True by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3641
FIX: Failure in audio feature related test by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3651
Remove unnecessary peft config updating by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3642
FIX: docker build error for ludwig-gpu by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3658
Exclude getdaft on Windows by @carlogrisetti in https://github.com/ludwig-ai/ludwig/pull/3629
Add daft back for windows since the wheels are now officially published by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3663
fix: The final batch of an epoch is skipped when batch size is 1 by @jeffkinnison in https://github.com/ludwig-ai/ludwig/pull/3653
Place metric functions for BLEU and Rogue on correct devices when using multiple GPUs by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3671
Remove duplicate metrics by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3670
Increment epochs based on last_batch() instead of at the end of the train loop. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3668
[FEATURE] Support Merging LoRA Weights Into Base Model (Issue-3603) by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3649
[FEATURE] Include Mistral-7B model in list of supported base models by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3674
[MAINTENANCE] Partially reconcile type hints, fix some warnings, and fix comments in parts of the codebase. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3673
Improve error message for when an LLM base model can't be loaded. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3675
Fix eos_token and pad_token issue by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3667
FIX: error with nightly CI tests for test_resize_image by @jimthompson5802 in https://github.com/ludwig-ai/ludwig/pull/3678
[BUGFIX] Remove spurious test directory at the end of the test_llm.py::test_local_path_loading test run by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3680
Add per-device logging to tensorboard by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3677
Fix dynamic generation config load during model.predict by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3666
[CI] Ensure that mlflow callback cleans up background-saving threads on trainer teardown. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3683
fix: temporarily remove config validation check for backend by @geoffreyangus in https://github.com/ludwig-ai/ludwig/pull/3688
fix: Failing test for backend with quantization by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3689
[BUGFIX] Ensure that full base models and not only adapter weights get saved when merge_and_unload is set by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3679
Add Ludwig Star History to README by @arnavgarg1 in https://github.com/ludwig-ai/ludwig/pull/3696
Use sphinx for all docstrings in api.py by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3693
Fix binary variables being visualized as 0 and 1 by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3691
[MAINTENANCE] Fix the linting warnings in two backend component classes. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3698
[BUGFIX] Pin deepspeed<0.11, skip Horovod tests by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3700
Unpin deepspeed following fix in v0.11.1 by @tgaddair in https://github.com/ludwig-ai/ludwig/pull/3706
Move on_epoch_end and epoch increment to after run_evaluation loop. by @justinxzhao in https://github.com/ludwig-ai/ludwig/pull/3690
Remove model_load_path from experiment by @Infernaught in https://github.com/ludwig-ai/ludwig/pull/3707
[FEATURE] Allow typehints without the quotes. by @alexsherstinsky in https://github.com/ludwig-ai/ludwig/pull/3699

New Contributors

@alexsherstinsky made their first contribution in https://github.com/ludwig-ai/ludwig/pull/3649

Full Changelog: https://github.com/ludwig-ai/ludwig/compare/v0.8.4...v0.8.5