| Method | Vanilla | KD | KD+ SRKD | AT | AT+ SRKD | SP | SP+ SRKD | Teacher | |
| T: Resnet20 | Top-1 | 78.25 | 86.52 | 86.81 | 86.77 | 87 | 86.5 | 86.74 | 85.81 |
| S: Resnet8 | Top-5 | 98.69 | 99.25 | 99.4 | 99.31 | 99.42 | 99.14 | 99.38 | 99.22 |
| T: Resnet32 | Top-1 | 85.81 | 88.67 | 88.97 | 88.83 | 89.06 | 89.05 | 89.09 | 86.97 |
| S: Resnet20 | Top-5 | 99.22 | 99.41 | 99.49 | 99.48 | 99.51 | 99.52 | 99.53 | 99.37 |