[ECCV 2022] Official repository for "MaxViT: Multi-Axis Vision Transformer". SOTA foundation models for classification, detection, segmentation, image quality, and generative modeling...
This repository hosts the official TensorFlow implementation of MAXViT models:
MaxViT: Multi-Axis Vision Transformer. ECCV 2022.
Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li
Google Research, University of Texas at Austin
Disclaimer: This is not an officially supported Google product.
News:
MaxViT is a family of hybrid (CNN + ViT) image classification models, that achieves better performances across the board for both parameter and FLOPs efficiency than both SoTA ConvNets and Transformers. They can also scale well on large dataset sizes like ImageNet-21K. Notably, due to the linear-complexity of the grid attention used, MaxViT is able to ''see'' globally throughout the entire network, even in earlier, high-resolution stages.
MaxViT meta-architecture:
Results on ImageNet-1k train and test:
Results on ImageNet-21k and JFT pre-trained models:
We have released a Google Colab Demo on the tutorials of how to run MaxViT on images. Try it here
We have provided a list of results and checkpoints as follows:
Name | Resolution | Top1 Acc. | #Params | FLOPs | Model |
---|---|---|---|---|---|
MaxViT-T | 224x224 | 83.62% | 31M | 5.6B | ckpt |
MaxViT-T | 384x384 | 85.24% | 31M | 17.7B | ckpt |
MaxViT-T | 512x512 | 85.72% | 31M | 33.7B | ckpt |
MaxViT-S | 224x224 | 84.45% | 69M | 11.7B | ckpt |
MaxViT-S | 384x384 | 85.74% | 69M | 36.1B | ckpt |
MaxViT-S | 512x512 | 86.19% | 69M | 67.6B | ckpt |
MaxViT-B | 224x224 | 84.95% | 119M | 24.2B | ckpt |
MaxViT-B | 384x384 | 86.34% | 119M | 74.2B | ckpt |
MaxViT-B | 512x512 | 86.66% | 119M | 138.5B | ckpt |
MaxViT-L | 224x224 | 85.17% | 212M | 43.9B | ckpt |
MaxViT-L | 384x384 | 86.40% | 212M | 133.1B | ckpt |
MaxViT-L | 512x512 | 86.70% | 212M | 245.4B | ckpt |
Here are a list of ImageNet-21K pretrained and ImageNet-1K finetuned models:
Name | Resolution | Top1 Acc. | #Params | FLOPs | 21k model | 1k model |
---|---|---|---|---|---|---|
MaxViT-B | 224x224 | - | 119M | 24.2B | ckpt | - |
MaxViT-B | 384x384 | - | 119M | 74.2B | - | ckpt |
MaxViT-B | 512x512 | - | 119M | 138.5B | - | ckpt |
MaxViT-L | 224x224 | - | 212M | 43.9B | ckpt | - |
MaxViT-L | 384x384 | - | 212M | 133.1B | - | ckpt |
MaxViT-L | 512x512 | - | 212M | 245.4B | - | ckpt |
MaxViT-XL | 224x224 | - | 475M | 97.8B | ckpt | - |
MaxViT-XL | 384x384 | - | 475M | 293.7B | - | ckpt |
MaxViT-XL | 512x512 | - | 475M | 535.2B | - | ckpt |
Should you find this repository useful, please consider citing:
@article{tu2022maxvit,
title={MaxViT: Multi-Axis Vision Transformer},
author={Tu, Zhengzhong and Talebi, Hossein and Zhang, Han and Yang, Feng and Milanfar, Peyman and Bovik, Alan and Li, Yinxiao},
journal={ECCV},
year={2022},
}
Acknowledgement: This repository is built on the EfficientNets and CoAtNet.