MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

1.Abstract

類神經網路在近幾年已取得巨大的突破，從最開始的Perceptron一直演化到AlexNet、VGG，精確度一直不斷的往上提升，但伴隨著的是模型複雜化，過度複雜的模型使得運算量極大以至於不能在低階產品(如手機上)運行，MobileNet的發明就是為了解決這想法而產生的，文中和許多當前有名的框架互相比較(Inception V3、VGG)，速度真的提升許多，雖然精確度有稍許下降，但也都控制在可以接受的範圍內。

2.Main idea

文中最核心的思想就是把傳統的卷積過程簡化成計算量更少的過程，算式並沒有完全等效，但有趨近於等效的功用，主要是把過程分成Depthwise convolution 和 pointwise convolution，文中用公式來舉例說明

* 傳統卷積計算量:
$D_F$:代表輸入特徵圖片的邊長
$D_K$:代表卷積核的邊長
$M$:輸入通道數量
$N$:輸出通道數量
$K$:代表卷積核
$F$:代表Feature map

• Mobilenet 計算量:
  
  前項是Depthwise convolution的計算量，後項是pointwise convolution的計算量

• 圖解說明:
  
  

文中用圖解說明分解的過程原理，可以這麼的想像，假設有個圖片維度是11X11X3，卷積核是3X3，輸出16個特徵圖形，假設stride是2，padding是1，那麼可以得到輸出為6X6X16的特徵圖形。現在可以把卷積核換成3個3X3X1的卷積核(b)，可以得到6X6X3的中間輸出，再通過16個1X1X3的卷積核做卷積同樣可以得到6X6X16的輸出。

• 優化的量:
  
  將mobile net的計算量除以傳統的計算量就可以算出優化的量，也就是說對輸出通道數量越多以及卷積核越大的傳統卷積神經網路提升效果越明顯。
  
  

• 傳統卷積與depwise convolution 比較圖:
  
  
  
  

3.Structure

• My note:
  
  

• Paper structure:
  
  
  
  Mobilenet把大部分的計算量都集中在1X1的卷積層，與傳統卷積網路將大部分計算都放在全連接層大有不同。
  
  

4.Parameter

Mobilenet 還提出了兩個全域參數的概念，一個稱為Width Multiplier(WM)，另一個稱為Resolution Multiplier(RM)，以第六個網路MobileNet_v1_0.75_192來說，WM為0.75、RM為6/7(因為224X6/7=192)，圖中比較各個網路的計算量(multiplication-accumulation–MAC)、參數量以及準確度(關於Top1、Top5不懂的人可以看這個)，準確度依次由高而低往下排列，這兩個參數提供給想要讓模型跑得更快的人使用，下面詳細解釋這兩個參數的意義。

• Width Multiplier:
  
  αα代表將通道數縮減的比例，介於(0,1]之間，它會對每一層的通道(輸入通道數M以及輸出通道數N)乘以α做縮減，因此底下為乘上α後的計算量:
  
  

• Resolution Multiplier:
  
  ρρ代表將每一層feature map縮減的比例，ρ如果是{1,6/7,5/7,4/7}就代表圖片解析度變為{224,192,160,128}，底下代表乘上這兩個參數後的計算量:
  
  

• 下表可以看出不同的方法所減少的計算量與參數量
  
  

5.Result

文獻中有多個圖表，這裡只挑幾張重要的來做說明，由下表8可以看出文獻中想表達mobilenet和VGG16的比較，雖然準度有下降一點點但是計算量卻減少30倍之多。表9證明採用0.5_160的mobile net效果就已經比AlexNet還好了。表10證明mobilenet和Inception V3在準確度上已經趨於同樣的程度了，但計算量卻大大的減少9倍之多。表13說明mobilenet配上object detection同樣也有不錯的效果。

6.tensorflow 每一層參數

Following are all sorted varibales in tensorflow model:

0 MobilenetV1/Conv2d_0/BatchNorm/beta [32]

1 MobilenetV1/Conv2d_0/BatchNorm/gamma [32]

2 MobilenetV1/Conv2d_0/BatchNorm/moving_mean [32]

3 MobilenetV1/Conv2d_0/BatchNorm/moving_variance [32]

4 MobilenetV1/Conv2d_0/weights [3, 3, 3, 32]

5 MobilenetV1/Conv2d_10_depthwise/BatchNorm/beta [512]

6 MobilenetV1/Conv2d_10_depthwise/BatchNorm/gamma [512]

7 MobilenetV1/Conv2d_10_depthwise/BatchNorm/moving_mean [512]

8 MobilenetV1/Conv2d_10_depthwise/BatchNorm/moving_variance [512]

9 MobilenetV1/Conv2d_10_depthwise/depthwise_weights [3, 3, 512, 1]

10 MobilenetV1/Conv2d_10_pointwise/BatchNorm/beta [512]

11 MobilenetV1/Conv2d_10_pointwise/BatchNorm/gamma [512]

12 MobilenetV1/Conv2d_10_pointwise/BatchNorm/moving_mean [512]

13 MobilenetV1/Conv2d_10_pointwise/BatchNorm/moving_variance [512]

14 MobilenetV1/Conv2d_10_pointwise/weights [1, 1, 512, 512]

15 MobilenetV1/Conv2d_11_depthwise/BatchNorm/beta [512]

16 MobilenetV1/Conv2d_11_depthwise/BatchNorm/gamma [512]

17 MobilenetV1/Conv2d_11_depthwise/BatchNorm/moving_mean [512]

18 MobilenetV1/Conv2d_11_depthwise/BatchNorm/moving_variance [512]

19 MobilenetV1/Conv2d_11_depthwise/depthwise_weights [3, 3, 512, 1]

20 MobilenetV1/Conv2d_11_pointwise/BatchNorm/beta [512]

21 MobilenetV1/Conv2d_11_pointwise/BatchNorm/gamma [512]

22 MobilenetV1/Conv2d_11_pointwise/BatchNorm/moving_mean [512]

23 MobilenetV1/Conv2d_11_pointwise/BatchNorm/moving_variance [512]

24 MobilenetV1/Conv2d_11_pointwise/weights [1, 1, 512, 512]

25 MobilenetV1/Conv2d_12_depthwise/BatchNorm/beta [512]

26 MobilenetV1/Conv2d_12_depthwise/BatchNorm/gamma [512]

27 MobilenetV1/Conv2d_12_depthwise/BatchNorm/moving_mean [512]

28 MobilenetV1/Conv2d_12_depthwise/BatchNorm/moving_variance [512]

29 MobilenetV1/Conv2d_12_depthwise/depthwise_weights [3, 3, 512, 1]

30 MobilenetV1/Conv2d_12_pointwise/BatchNorm/beta [1024]

31 MobilenetV1/Conv2d_12_pointwise/BatchNorm/gamma [1024]

32 MobilenetV1/Conv2d_12_pointwise/BatchNorm/moving_mean [1024]

33 MobilenetV1/Conv2d_12_pointwise/BatchNorm/moving_variance [1024]

34 MobilenetV1/Conv2d_12_pointwise/weights [1, 1, 512, 1024]

35 MobilenetV1/Conv2d_13_depthwise/BatchNorm/beta [1024]

36 MobilenetV1/Conv2d_13_depthwise/BatchNorm/gamma [1024]

37 MobilenetV1/Conv2d_13_depthwise/BatchNorm/moving_mean [1024]

38 MobilenetV1/Conv2d_13_depthwise/BatchNorm/moving_variance [1024]

39 MobilenetV1/Conv2d_13_depthwise/depthwise_weights [3, 3, 1024, 1]

40 MobilenetV1/Conv2d_13_pointwise/BatchNorm/beta [1024]

41 MobilenetV1/Conv2d_13_pointwise/BatchNorm/gamma [1024]

42 MobilenetV1/Conv2d_13_pointwise/BatchNorm/moving_mean [1024]

43 MobilenetV1/Conv2d_13_pointwise/BatchNorm/moving_variance [1024]

44 MobilenetV1/Conv2d_13_pointwise/weights [1, 1, 1024, 1024]

45 MobilenetV1/Conv2d_1_depthwise/BatchNorm/beta [32]

46 MobilenetV1/Conv2d_1_depthwise/BatchNorm/gamma [32]

47 MobilenetV1/Conv2d_1_depthwise/BatchNorm/moving_mean [32]

48 MobilenetV1/Conv2d_1_depthwise/BatchNorm/moving_variance [32]

49 MobilenetV1/Conv2d_1_depthwise/depthwise_weights [3, 3, 32, 1]

50 MobilenetV1/Conv2d_1_pointwise/BatchNorm/beta [64]

51 MobilenetV1/Conv2d_1_pointwise/BatchNorm/gamma [64]

52 MobilenetV1/Conv2d_1_pointwise/BatchNorm/moving_mean [64]

53 MobilenetV1/Conv2d_1_pointwise/BatchNorm/moving_variance [64]

54 MobilenetV1/Conv2d_1_pointwise/weights [1, 1, 32, 64]

55 MobilenetV1/Conv2d_2_depthwise/BatchNorm/beta [64]

56 MobilenetV1/Conv2d_2_depthwise/BatchNorm/gamma [64]

57 MobilenetV1/Conv2d_2_depthwise/BatchNorm/moving_mean [64]

58 MobilenetV1/Conv2d_2_depthwise/BatchNorm/moving_variance [64]

59 MobilenetV1/Conv2d_2_depthwise/depthwise_weights [3, 3, 64, 1]

60 MobilenetV1/Conv2d_2_pointwise/BatchNorm/beta [128]

61 MobilenetV1/Conv2d_2_pointwise/BatchNorm/gamma [128]

62 MobilenetV1/Conv2d_2_pointwise/BatchNorm/moving_mean [128]

63 MobilenetV1/Conv2d_2_pointwise/BatchNorm/moving_variance [128]

64 MobilenetV1/Conv2d_2_pointwise/weights [1, 1, 64, 128]

65 MobilenetV1/Conv2d_3_depthwise/BatchNorm/beta [128]

66 MobilenetV1/Conv2d_3_depthwise/BatchNorm/gamma [128]

67 MobilenetV1/Conv2d_3_depthwise/BatchNorm/moving_mean [128]

68 MobilenetV1/Conv2d_3_depthwise/BatchNorm/moving_variance [128]

69 MobilenetV1/Conv2d_3_depthwise/depthwise_weights [3, 3, 128, 1]

70 MobilenetV1/Conv2d_3_pointwise/BatchNorm/beta [128]

71 MobilenetV1/Conv2d_3_pointwise/BatchNorm/gamma [128]

72 MobilenetV1/Conv2d_3_pointwise/BatchNorm/moving_mean [128]

73 MobilenetV1/Conv2d_3_pointwise/BatchNorm/moving_variance [128]

74 MobilenetV1/Conv2d_3_pointwise/weights [1, 1, 128, 128]

75 MobilenetV1/Conv2d_4_depthwise/BatchNorm/beta [128]

76 MobilenetV1/Conv2d_4_depthwise/BatchNorm/gamma [128]

77 MobilenetV1/Conv2d_4_depthwise/BatchNorm/moving_mean [128]

78 MobilenetV1/Conv2d_4_depthwise/BatchNorm/moving_variance [128]

79 MobilenetV1/Conv2d_4_depthwise/depthwise_weights [3, 3, 128, 1]

80 MobilenetV1/Conv2d_4_pointwise/BatchNorm/beta [256]

81 MobilenetV1/Conv2d_4_pointwise/BatchNorm/gamma [256]

82 MobilenetV1/Conv2d_4_pointwise/BatchNorm/moving_mean [256]

83 MobilenetV1/Conv2d_4_pointwise/BatchNorm/moving_variance [256]

84 MobilenetV1/Conv2d_4_pointwise/weights [1, 1, 128, 256]

85 MobilenetV1/Conv2d_5_depthwise/BatchNorm/beta [256]

86 MobilenetV1/Conv2d_5_depthwise/BatchNorm/gamma [256]

87 MobilenetV1/Conv2d_5_depthwise/BatchNorm/moving_mean [256]

88 MobilenetV1/Conv2d_5_depthwise/BatchNorm/moving_variance [256]

89 MobilenetV1/Conv2d_5_depthwise/depthwise_weights [3, 3, 256, 1]

90 MobilenetV1/Conv2d_5_pointwise/BatchNorm/beta [256]

91 MobilenetV1/Conv2d_5_pointwise/BatchNorm/gamma [256]

92 MobilenetV1/Conv2d_5_pointwise/BatchNorm/moving_mean [256]

93 MobilenetV1/Conv2d_5_pointwise/BatchNorm/moving_variance [256]

94 MobilenetV1/Conv2d_5_pointwise/weights [1, 1, 256, 256]

95 MobilenetV1/Conv2d_6_depthwise/BatchNorm/beta [256]

96 MobilenetV1/Conv2d_6_depthwise/BatchNorm/gamma [256]

97 MobilenetV1/Conv2d_6_depthwise/BatchNorm/moving_mean [256]

98 MobilenetV1/Conv2d_6_depthwise/BatchNorm/moving_variance [256]

99 MobilenetV1/Conv2d_6_depthwise/depthwise_weights [3, 3, 256, 1]

100 MobilenetV1/Conv2d_6_pointwise/BatchNorm/beta [512]

101 MobilenetV1/Conv2d_6_pointwise/BatchNorm/gamma [512]

102 MobilenetV1/Conv2d_6_pointwise/BatchNorm/moving_mean [512]

103 MobilenetV1/Conv2d_6_pointwise/BatchNorm/moving_variance [512]

104 MobilenetV1/Conv2d_6_pointwise/weights [1, 1, 256, 512]

105 MobilenetV1/Conv2d_7_depthwise/BatchNorm/beta [512]

106 MobilenetV1/Conv2d_7_depthwise/BatchNorm/gamma [512]

107 MobilenetV1/Conv2d_7_depthwise/BatchNorm/moving_mean [512]

108 MobilenetV1/Conv2d_7_depthwise/BatchNorm/moving_variance [512]

109 MobilenetV1/Conv2d_7_depthwise/depthwise_weights [3, 3, 512, 1]

110 MobilenetV1/Conv2d_7_pointwise/BatchNorm/beta [512]

111 MobilenetV1/Conv2d_7_pointwise/BatchNorm/gamma [512]

112 MobilenetV1/Conv2d_7_pointwise/BatchNorm/moving_mean [512]

113 MobilenetV1/Conv2d_7_pointwise/BatchNorm/moving_variance [512]

114 MobilenetV1/Conv2d_7_pointwise/weights [1, 1, 512, 512]

115 MobilenetV1/Conv2d_8_depthwise/BatchNorm/beta [512]

116 MobilenetV1/Conv2d_8_depthwise/BatchNorm/gamma [512]

117 MobilenetV1/Conv2d_8_depthwise/BatchNorm/moving_mean [512]

118 MobilenetV1/Conv2d_8_depthwise/BatchNorm/moving_variance [512]

119 MobilenetV1/Conv2d_8_depthwise/depthwise_weights [3, 3, 512, 1]

120 MobilenetV1/Conv2d_8_pointwise/BatchNorm/beta [512]

121 MobilenetV1/Conv2d_8_pointwise/BatchNorm/gamma [512]

122 MobilenetV1/Conv2d_8_pointwise/BatchNorm/moving_mean [512]

123 MobilenetV1/Conv2d_8_pointwise/BatchNorm/moving_variance [512]

124 MobilenetV1/Conv2d_8_pointwise/weights [1, 1, 512, 512]

125 MobilenetV1/Conv2d_9_depthwise/BatchNorm/beta [512]

126 MobilenetV1/Conv2d_9_depthwise/BatchNorm/gamma [512]

127 MobilenetV1/Conv2d_9_depthwise/BatchNorm/moving_mean [512]

128 MobilenetV1/Conv2d_9_depthwise/BatchNorm/moving_variance [512]

129 MobilenetV1/Conv2d_9_depthwise/depthwise_weights [3, 3, 512, 1]

130 MobilenetV1/Conv2d_9_pointwise/BatchNorm/beta [512]

131 MobilenetV1/Conv2d_9_pointwise/BatchNorm/gamma [512]

132 MobilenetV1/Conv2d_9_pointwise/BatchNorm/moving_mean [512]

133 MobilenetV1/Conv2d_9_pointwise/BatchNorm/moving_variance [512]

134 MobilenetV1/Conv2d_9_pointwise/weights [1, 1, 512, 512]

135 MobilenetV1/Logits/Conv2d_1c_1x1/biases [1001]

136 MobilenetV1/Logits/Conv2d_1c_1x1/weights [1, 1, 1024, 1001]