A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.
added context interface to WebDataset
specs working
changed handling of wids default decoder
fix
Added support for memory mapped random access for indexed tar files. This speeds up random access by more than an order of magnitude and allows cached data in the tar files to be shared across workers.
updated wids
wids improvements
fix