DeepSpeed Versions Save

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

v0.12.5

4 months ago

What's Changed

Fix DS Stable Diffusion for latest diffusers version by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4770
Resolve any '..' in the file paths using os.path.abspath() by @rraminen in https://github.com/microsoft/DeepSpeed/pull/4709
Update dockerfile with updated versions by @loadams in https://github.com/microsoft/DeepSpeed/pull/4780
Run workflows when they are edited by @loadams in https://github.com/microsoft/DeepSpeed/pull/4779
BF16_Optimizer: add support for bf16 grad acc by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4713
fix autoTP issue for mpt (trust_remote_code=True) by @sywangyi in https://github.com/microsoft/DeepSpeed/pull/4787
Fix Hybrid Engine metrics printing by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4789
[BUG] partition_balanced return wrong result. by @zjjMaiMai in https://github.com/microsoft/DeepSpeed/pull/4312
improve the way to determine whether a variable is None by @RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4782
[NPU] Add HcclBackend for 1-bit adam, 1-bit lamb, 0/1 adam by @RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4733
Fix for stage3 when setting different communication data type by @BacharL in https://github.com/microsoft/DeepSpeed/pull/4540
Add support of Falcon models (7b, 40b, 180b) to DeepSpeed-FastGen by @arashb in https://github.com/microsoft/DeepSpeed/pull/4790
Switch paths-ignore to single quotes, update paths-ignore on nv-pre-compile-ops by @loadams in https://github.com/microsoft/DeepSpeed/pull/4805
fix for tests using torch<2.1 by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4818
Universal Checkpoint for Sequence Parallelism by @samadejacobs in https://github.com/microsoft/DeepSpeed/pull/4752
Accelerate CI fix by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4819
fix [BUG] 'DeepSpeedGPTInference' object has no attribute 'dtype' for… by @jxysoft in https://github.com/microsoft/DeepSpeed/pull/4814
Update broken link in docs by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4822
Update imports from Transformers by @loadams in https://github.com/microsoft/DeepSpeed/pull/4817
Minor updates to CI workflows by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4823
fix falcon model load from_config meta_data error by @baodii in https://github.com/microsoft/DeepSpeed/pull/4783
mv DeepSpeedEngine param_names dict init post _configure_distributed_model by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4803
Refactor launcher user arg parsing by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4824
Fix 4649 by @Alienfeel in https://github.com/microsoft/DeepSpeed/pull/4650

New Contributors

@zjjMaiMai made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4312
@jxysoft made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4814
@baodii made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4783
@Alienfeel made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4650

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.12.4...v0.12.5

v0.12.4

5 months ago

What's Changed

Update version.txt after 0.12.3 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4673
[MII] catch error wrt HF version and Mistral by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4634
[NPU] Add NPU support for unit test by @RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4569
[op-builder] use unique exceptions for cuda issues by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4653
Add stable diffusion unit test by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/2496
[CANN] Support cpu offload optimizer for Ascend NPU by @hipudding in https://github.com/microsoft/DeepSpeed/pull/4568
Inference Checkpoints in V2 by @cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4664
KV Cache Improved Flexibility by @cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4668
Fix for when prompt contains an odd num of apostrophes by @oelayan7 in https://github.com/microsoft/DeepSpeed/pull/4660
universal-ckp: support megatron-deepspeed llama model by @mosheisland in https://github.com/microsoft/DeepSpeed/pull/4666
Add new MII unit tests by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4693
[Bug fix] WarmupCosineLR issues by @sbwww in https://github.com/microsoft/DeepSpeed/pull/4688
infV2 fix for OPT size variants by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4694
Add get and set APIs for the ZeRO-3 partitioned parameters by @yiliu30 in https://github.com/microsoft/DeepSpeed/pull/4681
Remove unneeded dict reinit (fix for #4565) by @eisene in https://github.com/microsoft/DeepSpeed/pull/4702
Update flops profiler to recurse by @loadams in https://github.com/microsoft/DeepSpeed/pull/4374
Communication Optimization for Large-Scale Training by @RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/4695
[docs] Intel inference blog by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4734
use all_gather_into_tensor instead of all_gather by @taozhiwei in https://github.com/microsoft/DeepSpeed/pull/4705
Install deepspeed-kernels only on Linux by @aphedges in https://github.com/microsoft/DeepSpeed/pull/4739
Add nv-sd badge to README by @loadams in https://github.com/microsoft/DeepSpeed/pull/4747
Re-organize .gitignore file to be parsed properly by @aphedges in https://github.com/microsoft/DeepSpeed/pull/4740
fix mics run with offload++ by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4749
Fix logger formatting for partitioning flags by @OAfzal in https://github.com/microsoft/DeepSpeed/pull/4728
fix: to solve #4726 by @RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4727
Add safetensors support by @jihnenglin in https://github.com/microsoft/DeepSpeed/pull/4659

New Contributors

@RUAN-ZX made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4569
@oelayan7 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4660
@sbwww made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4688
@yiliu30 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4681
@eisene made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4702
@taozhiwei made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4705
@OAfzal made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4728
@jihnenglin made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4659

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.12.3...v0.12.4

v0.12.3

5 months ago

New Bug Fixes

Stable Diffusion now supported with latest Torch, diffusers, and Triton versions.

What's Changed

Update version.txt after 0.12.2 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4617
Fix figure in FlexGen blog by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4624
Fix figure of llama2 13B in DS-FlexGen blog by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4625
Fix config format by @xu-song in https://github.com/microsoft/DeepSpeed/pull/4594
Guanhua/partial offload rebase v2 (#590) by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4636
offload++ blog (#623) by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4637
Update README in offloadpp blog by @GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4641
[docs] update news items by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4640
DeepSpeed-FastGen Chinese Blog by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/4642
Fix issues with torch cpu builds by @loadams in https://github.com/microsoft/DeepSpeed/pull/4639
Isolate src code and testing for DeepSpeed-FastGen by @cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4610
Add Japanese blog for DeepSpeed-FastGen by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4651
Fix for MII unit tests by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4652
Enhance the robustness of module_state_dict by @LZHgrla in https://github.com/microsoft/DeepSpeed/pull/4587
Enable ZeRO3 allgather for multiple dtypes by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4647
add option to disable pipeline partitioning by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4322
Added HIP_PLATFORM_AMD=1 for non JIT build by @rraminen in https://github.com/microsoft/DeepSpeed/pull/4585
Fix rope_theta arg for diffusers_attention by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4656
tl.dot(a,b, trans_b=True) is not supported by triton2.0+ , updating this api by @bmedishe in https://github.com/microsoft/DeepSpeed/pull/4541
Update ds-chat workflow to work w/ deepspeed-chat install by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4598
Diffusers attention script update triton2.1 by @bmedishe in https://github.com/microsoft/DeepSpeed/pull/4573
Fix the openfold training. by @cctry in https://github.com/microsoft/DeepSpeed/pull/4657
Universal ckp fixes by @mosheisland in https://github.com/microsoft/DeepSpeed/pull/4588
Update .gitignore [Adding comments , Improved documentation] by @Nadav23AnT in https://github.com/microsoft/DeepSpeed/pull/4631
Update lr_schedules.py by @CoinCheung in https://github.com/microsoft/DeepSpeed/pull/4563
Fix UNET and VAE implementations for new diffusers version by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4663
fix num_kv_heads sharding in autoTP for the new in-repo Falcon-40B by @dc3671 in https://github.com/microsoft/DeepSpeed/pull/4654

New Contributors

@xu-song made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4594
@LZHgrla made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4587
@mosheisland made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4588
@Nadav23AnT made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4631
@CoinCheung made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4563

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.12.2...v0.12.3

v0.12.2

6 months ago

What's Changed

Quick bug fix direct to master to ensure mismatched cuda environments are shown to the user https://github.com/microsoft/DeepSpeed/commit/4f7dd7214b1d81dbbdff826015a67accc10390d2
Update version.txt after 0.12.1 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4615

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.12.1...v0.12.2

v0.12.1

6 months ago

What's Changed

Update version.txt after 0.12.0 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4611
Add number for latency comparison by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4612
Update minor CUDA version compatibility. by @cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4613

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.12.0...v0.12.1

v0.12.0

6 months ago

New features

DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

What's Changed

Update version.txt after 0.11.2 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4609
Pin transformers in nv-inference by @loadams in https://github.com/microsoft/DeepSpeed/pull/4606
DeepSpeed-FastGen by @cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4604
DeepSpeed-FastGen blog by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4607

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.11.2...v0.12.0

v0.11.2

6 months ago

What's Changed

Update version.txt after 0.11.1 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4484
Update DS_BUILD_* references. by @loadams in https://github.com/microsoft/DeepSpeed/pull/4485
Introduce pydantic_v1 compatibility module for pydantic>=2.0.0 support by @ringohoffman in https://github.com/microsoft/DeepSpeed/pull/4407
Enable control over timeout with environment variable by @BramVanroy in https://github.com/microsoft/DeepSpeed/pull/4405
Update ROCm verison by @loadams in https://github.com/microsoft/DeepSpeed/pull/4486
adding 8bit dequantization kernel for asym fine-grained block quantization in zero-inference by @stephen-youn in https://github.com/microsoft/DeepSpeed/pull/4450
Fix scale factor on flops profiler by @loadams in https://github.com/microsoft/DeepSpeed/pull/4500
add DeepSpeed4Science white paper by @conglongli in https://github.com/microsoft/DeepSpeed/pull/4502
[CCLBackend] update API by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/4378
Ulysses: add col-ai evaluation by @samadejacobs in https://github.com/microsoft/DeepSpeed/pull/4517
Ulysses: Update README.md by @samadejacobs in https://github.com/microsoft/DeepSpeed/pull/4518
add available memory check to accelerators by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4508
clear redundant parameters in zero3 bwd hook by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/4520
Add NPU FusedAdam support by @CurryRice233 in https://github.com/microsoft/DeepSpeed/pull/4343
fix error type issue in deepspeed/comm/ccl.py by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/4521
Fixed deepspeed.comm.monitored_barrier call by @Quentin-Anthony in https://github.com/microsoft/DeepSpeed/pull/4496
[Bug fix] Add rope_theta for llama config by @cupertank in https://github.com/microsoft/DeepSpeed/pull/4480
[ROCm] Add rocblas header by @rraminen in https://github.com/microsoft/DeepSpeed/pull/4538
[docs] ZeRO infinity slides and blog by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4542
Switch from HIP_PLATFORM_HCC to HIP_PLATFORM_AMD by @loadams in https://github.com/microsoft/DeepSpeed/pull/4539
Turn off I_MPI_PIN for impi launcher by @delock in https://github.com/microsoft/DeepSpeed/pull/4531
[docs] paper updates by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4543
ROCm 6.0 prep changes by @loadams in https://github.com/microsoft/DeepSpeed/pull/4537
Fix RTD builds by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4558
pipe engine _aggregate_total_loss: more efficient loss concatenation by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4327
Add missing rocblas include by @loadams in https://github.com/microsoft/DeepSpeed/pull/4557
Enable universal checkpoint for zero stage 1 by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/4516
[AutoTP] Make AutoTP work when num_heads not divisible by number of workers by @delock in https://github.com/microsoft/DeepSpeed/pull/4011
Fix the sequence-parallelism for the dense model architecture by @RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/4530
engine.py - save_checkpoint: only rank-0 should create the save dir by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4536
Remove PP Grad Tail Check by @Quentin-Anthony in https://github.com/microsoft/DeepSpeed/pull/2538
Added HIP_PLATFORM_AMD=1 by @rraminen in https://github.com/microsoft/DeepSpeed/pull/4570
fix multiple definition while building evoformer by @fecet in https://github.com/microsoft/DeepSpeed/pull/4556
Don't check overflow for bf16 data type by @hablb in https://github.com/microsoft/DeepSpeed/pull/4512
Public update by @yaozhewei in https://github.com/microsoft/DeepSpeed/pull/4583
[docs] paper updates by @jeffra in https://github.com/microsoft/DeepSpeed/pull/4584
Disable CPU inference on PRs by @loadams in https://github.com/microsoft/DeepSpeed/pull/4590

New Contributors

@ringohoffman made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4407
@BramVanroy made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4405
@cupertank made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4480

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.11.1...v0.11.2

v0.11.1

7 months ago

What's Changed

Fix bug in bfloat16 optimizer related to checkpointing by @okoge-kaz in https://github.com/microsoft/DeepSpeed/pull/4434
Move tensors to device if mp is not enabled by @deepcharm in https://github.com/microsoft/DeepSpeed/pull/4461
Fix torch import causing release build failure by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4468
add lm_head and embed_out tensor parallel by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/3962
Fix release workflow by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4483

New Contributors

@okoge-kaz made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4434
@deepcharm made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4461

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.11.0...v0.11.1

v0.11.0

7 months ago

New features

DeepSpeed-VisualChat: Improve Your Chat Experience with Multi-Round Multi-Image Inputs [English] [中文] [日本語]
Announcing the DeepSpeed4Science Initiative: Enabling large-scale scientific discovery through sophisticated AI system technologies [DeepSpeed4Science website] [Tutorials] [Blog] [中文] [日本語]

What's Changed

added a model check for use_triton in deepspeed by @stephen-youn in https://github.com/microsoft/DeepSpeed/pull/4266
Update release and bump patch versioning flow by @loadams in https://github.com/microsoft/DeepSpeed/pull/4286
README update by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/4303
Update README.md by @NinoRisteski in https://github.com/microsoft/DeepSpeed/pull/4316
Handle empty parameter groups by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/4277
Clean up modeling code by @loadams in https://github.com/microsoft/DeepSpeed/pull/4320
Fix Zero3 contiguous grads, reduce scatter false accuracy issue by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4321
Add release version checking by @loadams in https://github.com/microsoft/DeepSpeed/pull/4328
clear redundant timers by @starkhu in https://github.com/microsoft/DeepSpeed/pull/4308
DS-Chat BLOOM: Fix Attention mask by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4338
Fix a bug in the implementation of dequantization for inference by @sakogan in https://github.com/microsoft/DeepSpeed/pull/3433
Suppress noise by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/4310
Fix skipped inference tests by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4336
Fix autotune to support Triton 2.1 by @stephen-youn in https://github.com/microsoft/DeepSpeed/pull/4340
Pass base_dir to model files can be loaded for auto-tp/meta-tensor. by @awan-10 in https://github.com/microsoft/DeepSpeed/pull/4348
Support InternLM by @wangruohui in https://github.com/microsoft/DeepSpeed/pull/4137
DeepSpeed4Science by @conglongli in https://github.com/microsoft/DeepSpeed/pull/4357
fix deepspeed4science links by @conglongli in https://github.com/microsoft/DeepSpeed/pull/4358
Add the policy to run llama model from the official repo by @RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/4313
Check inference input_id tokens length by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4349
add deepspeed4science blog link by @conglongli in https://github.com/microsoft/DeepSpeed/pull/4364
Update conda env to have max pydantic version by @loadams in https://github.com/microsoft/DeepSpeed/pull/4362
Enable workflow dispatch on Torch 1.10 CI tests by @loadams in https://github.com/microsoft/DeepSpeed/pull/4361
deepspeed4science chinese blog by @conglongli in https://github.com/microsoft/DeepSpeed/pull/4366
deepspeed4science japanese blog by @conglongli in https://github.com/microsoft/DeepSpeed/pull/4369
Openfold fix by @cctry in https://github.com/microsoft/DeepSpeed/pull/4368
[BUG] add the missing method to MPS accelerator by @cli99 in https://github.com/microsoft/DeepSpeed/pull/4363
Fix multinode runner to properly append to PDSH_SSH_ARGS_APPEND by @loadams in https://github.com/microsoft/DeepSpeed/pull/4373
Fix min torch version by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/4375
Fix llama meta tensor loading in AutoTP and kernel injected inference by @zeyugao in https://github.com/microsoft/DeepSpeed/pull/3608
adds triton flash attention2 kernel by @stephen-youn in https://github.com/microsoft/DeepSpeed/pull/4337
Allow multiple inference engines in single script by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4384
Save/restore step in param groups with zero 1 or 2 by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4396
Fix incorrect assignment of self.quantized_nontrainable_weights by @VeryLazyBoy in https://github.com/microsoft/DeepSpeed/pull/4399
update deepspeed4science blog by @conglongli in https://github.com/microsoft/DeepSpeed/pull/4408
Add torch no grad condition by @ajindal1 in https://github.com/microsoft/DeepSpeed/pull/4391
Update nv-transformers workflow to use cu11.6 by @loadams in https://github.com/microsoft/DeepSpeed/pull/4412
Add condition when dimension is greater than 2 by @ajindal1 in https://github.com/microsoft/DeepSpeed/pull/4390
[CPU] Add CPU AutoTP UT. by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4263
fix cpu loading model partition OOM by @Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4353
Update cpu_inference checkout action by @loadams in https://github.com/microsoft/DeepSpeed/pull/4424
Zero infinity xpu support by @Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/4130
[CCLBackend] Using parallel memcpy for inference_all_reduce by @delock in https://github.com/microsoft/DeepSpeed/pull/4404
Change default set_to_none=true in zero_grad methods by @Jackmin801 in https://github.com/microsoft/DeepSpeed/pull/4438
Small docstring fix by @Jackmin801 in https://github.com/microsoft/DeepSpeed/pull/4431
fix: check-license by @Jackmin801 in https://github.com/microsoft/DeepSpeed/pull/4432
Fixup check release version script by @loadams in https://github.com/microsoft/DeepSpeed/pull/4413
Enable ad-hoc running of cpu_inference by @loadams in https://github.com/microsoft/DeepSpeed/pull/4444
Fix wrong documentation of ignore_unused_parameters by @UniverseFly in https://github.com/microsoft/DeepSpeed/pull/4418
DeepSpeed-VisualChat Blog by @xiaoxiawu-microsoft in https://github.com/microsoft/DeepSpeed/pull/4446
Fix a bug in DeepSpeedMLP by @sakogan in https://github.com/microsoft/DeepSpeed/pull/4389
documenting load_from_fp32_weights config parameter by @clumsy in https://github.com/microsoft/DeepSpeed/pull/4449
Add Japanese translation of DS-VisualChat blog by @tohtana in https://github.com/microsoft/DeepSpeed/pull/4454
fix blog format by @conglongli in https://github.com/microsoft/DeepSpeed/pull/4456
Update README-Japanese.md by @conglongli in https://github.com/microsoft/DeepSpeed/pull/4457
DeepSpeed-VisualChat Chinese blog by @conglongli in https://github.com/microsoft/DeepSpeed/pull/4458
CI fix for torch 2.1 release by @mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4452
fix lm head overriden issue, move it from checkpoint in-loop loading … by @sywangyi in https://github.com/microsoft/DeepSpeed/pull/4206
feat: add Lion by @enneamer in https://github.com/microsoft/DeepSpeed/pull/4331
pipe engine eval_batch: add option to disable loss broadcast by @nelyahu in https://github.com/microsoft/DeepSpeed/pull/4326
Add release flow by @loadams in https://github.com/microsoft/DeepSpeed/pull/4467

New Contributors

@nelyahu made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4321
@starkhu made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4308
@sakogan made their first contribution in https://github.com/microsoft/DeepSpeed/pull/3433
@cctry made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4368
@zeyugao made their first contribution in https://github.com/microsoft/DeepSpeed/pull/3608
@VeryLazyBoy made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4399
@ajindal1 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4391
@Liangliang-Ma made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4130
@Jackmin801 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4438
@UniverseFly made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4418
@enneamer made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4331

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.10.3...v0.11.0

v0.10.3

7 months ago

New Features

ZeRO-Inference: 20X faster inference through weight quantization and KV cache offloading

What's Changed

Add Mixed Precision ZeRO++ tutorial by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/4241
DeepSpeed-Chat Llama2/stability release by @awan-10 in https://github.com/microsoft/DeepSpeed/pull/4240
Update README.md by @awan-10 in https://github.com/microsoft/DeepSpeed/pull/4244
Pin Triton version to >=2.0.0 and <2.1.0 by @lekurile in https://github.com/microsoft/DeepSpeed/pull/4251
Allow modification of zero partitioned parameters by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/4192
Checks for user injection policy by @satpalsr in https://github.com/microsoft/DeepSpeed/pull/3052
Add check that opening issues on CI failure requires schedule by @loadams in https://github.com/microsoft/DeepSpeed/pull/4242
Code Refactoring by @tosemml in https://github.com/microsoft/DeepSpeed/pull/4262
tolerating missing optimizer states for MoE [2nd attempt] by @clumsy in https://github.com/microsoft/DeepSpeed/pull/4120
Fix nv-inference/un-pin transformers by @loadams in https://github.com/microsoft/DeepSpeed/pull/4269
check for zero (empty) param groups in llama + hf/accelerate. by @awan-10 in https://github.com/microsoft/DeepSpeed/pull/4270
use non_reentrant_checkpoint fix requires_grad of input must be true for activation checkpoint layer in pipeline train. by @inkcherry in https://github.com/microsoft/DeepSpeed/pull/4224
The PostBackwardFunction class should be more clearly named to distinguish it from the PreBackwardFunction class. by @Crispig in https://github.com/microsoft/DeepSpeed/pull/2548
fix iteration timing used in autotuning when gradient_accumulation_steps > 1 by @cli99 in https://github.com/microsoft/DeepSpeed/pull/2888
Update README.md by @NinoRisteski in https://github.com/microsoft/DeepSpeed/pull/4284
update deepspeed to run with the most recent triton 2.1.0 by @stephen-youn in https://github.com/microsoft/DeepSpeed/pull/4278
Keep hpz secondary tensor in forward pass by @HeyangQin in https://github.com/microsoft/DeepSpeed/pull/4288
Support iterators with incompletely defined len functions by @codedecde in https://github.com/microsoft/DeepSpeed/pull/2445
AMD Kernel Compatibility Fixes by @cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/3180
ZeRO-Inference refresh by @tjruwase in https://github.com/microsoft/DeepSpeed/pull/4197
fix user args parsing of string with spaces on runner by @YudiZh in https://github.com/microsoft/DeepSpeed/pull/4265
Update index.md by @NinoRisteski in https://github.com/microsoft/DeepSpeed/pull/4297

New Contributors

@tosemml made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4262
@Crispig made their first contribution in https://github.com/microsoft/DeepSpeed/pull/2548
@NinoRisteski made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4284
@codedecde made their first contribution in https://github.com/microsoft/DeepSpeed/pull/2445
@YudiZh made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4265

Full Changelog: https://github.com/microsoft/DeepSpeed/compare/v0.10.2...v0.10.3