Flexible components pairing 🤗 Transformers with :zap: Pytorch Lightning
CNNDailyMailSummarizationDataModule (#310)self.device as default (#309)Full Changelog: https://github.com/Lightning-AI/lightning-transformers/compare/0.2.4...0.2.5
@rohitgr7
Full Changelog: https://github.com/Lightning-AI/lightning-transformers/compare/0.2.3...0.2.4
Use Lightning Utilities (#292) by @Borda
Full Changelog: https://github.com/Lightning-AI/lightning-transformers/compare/0.2.2...0.2.3
Update tests for latest PL release (#284) by @rohitgr7
Full Changelog: https://github.com/Lightning-AI/lightning-transformers/compare/0.2.1...0.2.2
⚡ Lightning Transformers 0.2.1
This is an incremental release with some documentation changes, DeepSpeed training support and a refactor to expose transformer model creation.
Full Changelog: https://github.com/Lightning-AI/lightning-transformers/compare/0.2.0...0.2.1
Below is a summary of the features/fixes we’ve added since the previous release!
Thanks to @tanmoyio we now have ViT support within Lightning Transformers!
Here is a simple example showing how you can fine-tune a ViT model using Lightning Transformers:
import pytorch_lightning as pl
from transformers import AutoFeatureExtractor
from lightning_transformers.task.vision.image_classification import (
ImageClassificationDataModule,
ImageClassificationTransformer,
)
feature_extractor = AutoFeatureExtractor.from_pretrained(pretrained_model_name_or_path="nateraw/vit-base-beans")
dm = ImageClassificationDataModule(
batch_size=8,
dataset_name="beans",
num_workers=8,
feature_extractor=feature_extractor,
)
model = ImageClassificationTransformer(
pretrained_model_name_or_path="nateraw/vit-base-beans", num_labels=dm.num_classes
)
trainer = pl.Trainer(accelerator="auto", devices="auto", max_epochs=5)
trainer.fit(model, dm)
More information in our documentation.
Many users have requested the ability to save HF Hub-compatible models. Look no further, we offer manual support + saving an additional HF compatible checkpoint automatically during training.
from lightning_transformers.task.nlp.text_classification import TextClassificationTransformer
model = TextClassificationTransformer(pretrained_model_name_or_path="prajjwal1/bert-tiny")
# saves a HF checkpoint to this path.
model.save_hf_checkpoint("checkpoint")
To save via training, just pass the HFSaveCheckpoint plugin within your training code:
import pytorch_lightning as pl
from lightning_transformers.plugins.checkpoint import HFSaveCheckpoint
...
model = TextClassificationTransformer(pretrained_model_name_or_path="prajjwal1/bert-tiny")
trainer = pl.Trainer(plugins=HFSaveCheckpoint(model=model))
trainer.fit(model, dm)
As transformer models get larger, they require more compute to run. In Lightning Transformers, we've utilized HF Accelerate to allow users to run billion parameter model inference.
This will in turn allow people who do not have the GPU memory or compute to run these models, by leveraging CPU memory & compute and disk space!
import torch
from accelerate import init_empty_weights
from transformers import AutoTokenizer
from lightning_transformers.task.nlp.language_modeling import LanguageModelingTransformer
# initializes empty model for us to the load the checkpoint.
with init_empty_weights():
model = LanguageModelingTransformer(
pretrained_model_name_or_path="EleutherAI/gpt-j-6B",
tokenizer=AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B")
)
# automatically selects the best devices (cpu/gpu) to load model layers based on available memory
# even offloads to disk if necessary.
model.load_checkpoint_and_dispatch("sharded-gpt-j-6B", device_map="auto", no_split_module_classes=["GPTJBlock"])
output = model.generate("Hello, my name is", device=torch.device("cuda"))
print(model.tokenizer.decode(output[0].tolist()))
We now have native support for SparseML! SparseML provides GPU-class performance on CPUs through sparsification, pruning, and quantization.
To enable SparseML, all you do is pass the callback to the Lightning Trainer with paths to your recipe!
import pytorch_lightning as pl
from lightning_transformers.callbacks import TransformerSparseMLCallback
pl.Trainer(
callbacks=TransformerSparseMLCallback(
output_dir="/content/MODELS",
recipe_path="/content/recipe.yaml"
)
)
See our medium blog post for more details.
Within this release we've simplified the API, removing complicated internal boilerplate and configuration that should exist outside this library. Keeping this library minimal means easier extensibility and easier contributions for everyone 🔥
Thanks to all the contributors that helped out!
The first release for Lightning Transformers!
Lightning Transformers offers a flexible interface for training and fine-tuning SOTA Transformer models using the PyTorch Lightning Trainer.
Lightning Transformers tasks allow you to train models using HuggingFace Transformer models and datasets, use Hydra to hotswap models, optimizers or schedulers and leverage all the advances features that Lightning has to offer, including custom Callbacks, Loggers, Accelerators and high performance scaling with minimal changes.
In this release, we introduce these Transformer Tasks to train and predict:
Each task supports various datasets, see our documentation for more information!