Torchx Versions Save

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

v0.1.2

2 years ago

Full Changelog: https://github.com/pytorch/torchx/compare/v0.1.1...v0.1.2

Milestone: https://github.com/pytorch/torchx/milestones/3

  • PyTorch 1.11 Support
  • Python 3.10 Support
  • torchx.workspace
    • TorchX now supports a concept of workspaces. This enables seamless launching of jobs using changes present in your local workspace. For Docker based schedulers, we automatically build a new docker container on job launch making it easier than ever to run experiments. #333
  • torchx.schedulers
    • Ray #329
    • AWS Batch #381
      • Newly added AWS Batch scheduler makes it easy to launch jobs in AWS with minimal infrastructure setup.
    • Slurm
      • Slurm jobs will by default launch in the current working directory to match local_cwd and workspace behavior. #372
      • Replicas now have their own log files and can be accessed programmatically. #373
      • Support for comment, mail-user and constraint fields. #391
      • Workspace support (prototype) - Slurm jobs can now be launched in isolated experiment directories. #416
    • Kubernetes
      • Support for running jobs under service accounts. #408
      • Support for specifying instance types. #433
    • All Docker-based Schedulers (Kubernetes, Batch, Docker)
      • Added bind mount and volume supports #420, #426
      • Bug fix: Better shm support for large dataloader #429
      • Support for .dockerignore and custom Dockerfiles #401
    • Local Scheduler
      • Automatically set CUDA_VISIBLE_DEVICES #383
      • Improved log ordering #366
  • torchx.components
  • torchx.cli
  • torchx.runner
    • Now supports workspace interfaces. #360
    • Returned lines now preserve whitespace to provide support for progress bars #425
    • Events are now logged to torch.monitor when available. #379
  • torchx.notebook (prototype)
    • Added new workspace interface for developing models and launching jobs via a Jupyter Notebook. #356
  • Docs
    • Improvements to clarify TorchX usage w/ workspaces and general cleanups.
    • #374, #402, #404, #407, #434