PyTorch Lightning + Hydra. A very user-friendly template for ML experimentation. ⚡🔥⚡
setup.py by @tesfaldet in https://github.com/ashleve/lightning-hydra-template/pull/575
instantiators.py by @tesfaldet in https://github.com/ashleve/lightning-hydra-template/pull/577
pyrootutils to rootutils by @ashleve in https://github.com/ashleve/lightning-hydra-template/pull/592
train.log is saved in project root dir by @tesfaldet in https://github.com/ashleve/lightning-hydra-template/pull/588
tests/test_train.py by @caplett in https://github.com/ashleve/lightning-hydra-template/pull/595
README.md by @amorehead in https://github.com/ashleve/lightning-hydra-template/pull/601
torch.compile on nn.module instead of on LightningModule by @tesfaldet in https://github.com/ashleve/lightning-hydra-template/pull/587
Full Changelog: https://github.com/ashleve/lightning-hydra-template/compare/v2.0.2...v2.0.3
ddp in ddp config by @ashleve in https://github.com/ashleve/lightning-hydra-template/pull/571
sync_dist=True when logging best so far validation accuracy by @ashleve in https://github.com/ashleve/lightning-hydra-template/pull/572
Full Changelog: https://github.com/ashleve/lightning-hydra-template/compare/v2.0.1...v2.0.2
Full Changelog: https://github.com/ashleve/lightning-hydra-template/compare/v2.0.0...v2.0.1
Release for alignment with PyTorch 2.0 and Lightning 2.0.
Full Changelog: https://github.com/ashleve/lightning-hydra-template/compare/v1.5.3...v2.0.0
__init__.py to configs/ folder @ashleve (#539)README.md @ashleve (#524)__main__ methods @ashleve (#502)datamodules folder to data @ashleve (#501).project-root file @ashleve (#496)dev branch to PR tests workflow @ashleve (#497)README.md @ashleve (#499)codecov.yml to prevent failing CI pipeline on coverage decrease (#484)make clean-logs command (#430)make sync command (#423)step() to model_step() for compatibility with recent lightning release (#472)task_wrapper decorator (#488)training_epoch_end() (#486)README.md (#465)README.md (#419, #425, #442)@ashleve @yipliu @amorehead @atong01 @YuCao16 @Yongtae723 @cauliyang
hydra 1.2 - no more changing the working directory by defaulttest.py and test.yaml to eval.py and eval.yaml (so as to avoid confusion with project tests)train.py and eval.py inside src/
pyrootutils package for standardizing the project root setup in train.py and eval.py
pipelines to tasks
task_name to main config, which determines hydra output folder path@task_wrapper decorator for applying utilities before and after the task is executedTuple[metric_dict, object_dict]
SimpleDenseNet config to model config with recursive instantiation_partial_: true
_convert_=partial from trainer instantiation (no longer needed since recent lightning release)ckpt_path to main config, trainer.fit() and trainer.test(), for compatibility with recent lightning releaseval_acc_best metric at the start of the training to prevent storing results from validation sanity checkstags to main configutils.extras()
name (since tags and task_name are enough)config to cfg since it's the standard naming convention in hydrautils.py, rich_utils.py, pylogger.py
utils.instantiate_callbacks() and utils.instantiate_loggers() to reduce the boilerplate in tasksutils.get_metric_value() for safely retrieving optimized metric.utils.finish() to utils.close_loggers()
configs/extras/default.yaml
trainer.gpus argument with trainer.accelerator and trainer.devices
hydra.mode=MULTIRUN to mnist_optuna.yaml config (so using -m is no longer needed when attaching this config)mnist_opuna.yaml for compatibility with new search space syntax in hydra 1.2
debug/overfit.yaml)setup.cfg with pyproject.toml since it's a more versatile standard (PEP 518)pyupgrade (automatically upgrading python syntax to newer version), bandit (security linter), codespell (spelling linter), mdformat (markdown formatting)depandabot
setup.py
Makefile
README.md
@nils-werner @johnnynunez @elisim @yu-xiang-wang @yipliu @Gxinhu @binlee52
The template has been significantly refactored.
List of changes:
run.py has been replaced by train.py and test.py
mode group config has been removed since it was confusing, now every run is treated as an experiment, and debugging is moved to a separate config groupdebug config grouplog_dir config groupwandb-callbacks to make template logger-agnosticnbstripout to pre-commit hooks, for automatic clearing of jupyter notebooks outputs before commitrequirements.txt and pre-commit-config.yaml to newest versionsmnist_optuna.yaml and add more explanatory commentsutils.extras()
requirements.txt for mac compatibilitymnist_model.yaml -> mnist.yaml, mnist_datamodule.yaml -> datamodule.yaml
mnist_model.py -> mnist_module.py and MNISTLitModel -> MNISTLitModule
modules/ to components/
accelerator="ddp" to strategy="ddp" since it was depracated by lightningweight summary and progress_bar_refresh_rate
.pre-commit-config.yaml just in case someone deletes the setup.cfg
testpath in setup.cfg so pytest knows all test files are placed only in tests/ folderbash to scripts
vendor dir as a "best practice" for storing third party codeconfigs/local/, which can be used for storing machine/user specific configurations, e.g. configuration of slurm clusterRichModelSummary to default callbacksREADME.md
README.md improvementsSpecial thanks to: @nils-werner @charlesincharge @Steve-Tod for their PRs.