Method

Network Architecture

WRN-40-2

WRN-40-2

ResNet56

ResNet110

ResNet110

VGG13

WRN-16-2

WRN-40-1

ResNet20

ResNet20

ResNet32

VGG8

Teacher

75.61

75.61

72.32

74.31

74.31

74.64

Vanilla

73.26

71.98

69.06

69.06

71.14

70.36

KD [7]

74.92

73.54

70.66

70.67

73.08

72.98

FitNet [8]

73.58

72.24

69.21

68.99

71.06

71.02

AT [9]

74.08

72.77

70.55

70.22

72.32

71.43

SP

73.93

72.43

69.67

70.04

72.69

72.68

CC [16]

73.56

72.21

69.63

69.48

71.48

70.71

RKD [17]

73.55

72.22

69.61

69.25

71.82

71.48

PKT [18]

74.54

73.45

70.34

70.25

72.61

72.88

FSP

72.91

NA

69.95

70.11

71.89

70.20

NST [10]

73.68

72.24

69.6

69.53

71.96

71.53

SRKD

75.83

73.94

71.84

72.08

72.37

73.36