Awesome Knowledge Distillation
Images to inference with no labeling (use foundation models to train sup...
Matching Guided Distillation (ECCV 2020)
Our open source implementation of MiniLMv2 (https://aclanthology.org/202...