Seit Save Abandoned

Project README

SeiT: Storage-efficient Vision Training

Official Pytorch implementation of SeiT | Paper

Song Park   Sanghyuk Chun   Byeongho Heo   Wonjae Kim   Sangdoo Yun


Code will be available soon!


We need billion-scale images to achieve more generalizable and ground-breaking vision models, as well as massive dataset storage to ship the images (e.g., the LAION-4B dataset needs 240TB storage space). However, it has become challenging to deal with unlimited dataset storage with limited storage infrastructure. A number of storage-efficient training methods have been proposed to tackle the problem, but they are rarely scalable or suffer from severe damage to performance. In this paper, we propose a storage-efficient training strategy for vision classifiers for large-scale datasets (e.g., ImageNet) that only uses 1024 tokens per instance without using the raw level pixels; our token storage only needs <1% of the original JPEG-compressed raw pixels. We also propose token augmentations and a Stem-adaptor module to make our approach able to use the same architecture as pixel-based approaches with only minimal modifications on the stem layer and the carefully tuned optimization settings. Our experimental results on ImageNet-1k show that our method significantly outperforms other storage-efficient training methods with a large gap. We further show the effectiveness of our method in other practical scenarios, storage-efficient pre-training, and continual learning.


    title={SeiT: Storage-Efficient Vision Training with Tokens Using 1% of Pixel Storage},
    author={Park, Song and Chun, Sanghyuk and Heo, Byeongho and Kim, Wonjae and Yun, Sangdoo},
    journal={arXiv preprint arXiv:2303.11114},
Open Source Agenda is not affiliated with "Seit" Project. README Source: naver-ai/seit
Open Issues
Last Commit
6 months ago

Open Source Agenda Badge

Open Source Agenda Rating