The codes for recent knowledge distillation algorithms and benchmark results via TF2.0 low-level API
Defined knowledge by the neural response of the hidden layer or the output layer of the network
Full Dataset | 50% Dataset | 25% Dataset | 10% Dataset | |
---|---|---|---|---|
Methods | Accuracy | Last Accuracy | Last Accuracy | Last Accuracy |
Teacher | 78.59 | - | - | - |
Student | 76.25 | - | - | - |
Soft_logits | 76.57 | - | - | - |
FitNet | 75.78 | - | - | - |
AT | 78.14 | - | - | - |
FSP | 76.08 | - | - | - |
DML | - | - | - | - |
KD_SVD | - | - | - | - |
FT | 77.30 | - | - | - |
AB | 76.52 | - | - | - |
RKD | 77.69 | - | - | - |
VID | - | - | - | - |
MHGD | - | - | - | - |
CO | 78.54 | - | - | - |