參考論文：ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
作者：Xiangyu Zhang ，Xinyu Zhou，Mengxiao Lin，Jian Sun
1、論文摘要
??我們引入了一個高效計算的CNN結構名字叫做shuffleNet，這個結構被設計用來解區域署算力非常有限的移動設備問題，這個新的結構使用了兩個新的操作,pointwise group convolution 和 channel shuffle能夠在極大減少計算量的同時保持一定的精度，我們在ImageNet classification和MS COCO目標檢測資料集上做實驗論證了ShuffleNet和其他的結構相比有著很好的性能，比如,相比于mobilenet，shufflenet在ImageNet 分類任務上有著更低的top-1錯誤率(錯誤率是7.8%)需要的計算量為40MFLOPs，在一個ARM-based移動設備,ShuffleNet相比于AlexNet實作了保持一定的精度的同時，實作了13A的速度，
2 Group Convolution（分組卷積）
??第一個創新點就是分組1*1卷積
??簡單來說分組卷積就是將特征圖分為不同的組，再對每組特征圖分別進行卷積，這里的分組一般都是分為 n 個等份，理論上其實不是等份也可以，不 過一般為了實作方便都是分為等份，分組卷積的好處主要是可以減少模型的計算量和訓練參 數，同時對模型準確率影響不大，甚至有可能會提高模型準確率，
在分組卷積中，每個卷積核只處理部分通道，比如上圖中，紅色卷積核只處理紅色的通道，綠色卷積核只處理綠色通道，黃色卷積核只處理黃色通道，此時每個卷積核有2個通道，每個卷積核生成一張特征圖，
??下面我們通過幾個圖來詳細了解一下，下圖為普通卷積
圖中的 Conv 表示卷積，
??這里特征圖的大小和卷積和的大小都不是重點內容，所以圖中沒有標出，我們只要能看 出 6 個特征圖卷積后得到 12 個特征圖就可以了，不過為了讓大家理解分組卷積的計算量和 權值數量這里我們舉例計算一下，假設特征圖大小是 28×28，卷積核大小為 5×5，Same Padding，卷積層權值數量為 5×5×6×12+12=1812，乘法計算量為 5×5×28×28×6× 12=1411200，
??下面我們看一下分組卷積，分組卷積一般都是把特征圖分為 n 個等份，然后再對 n 個等 份的特征圖分別卷積，這里的 n 可以人為設定，如圖所示，
圖中的 Conv 表示卷積，
??為了跟普通卷積對比，所以這里分組卷積的例子輸入也是 6 個特征圖，輸出也是 12 個特 征圖，這里我們可以看到把 6 個特征圖分為了 3 組，每組 2 個特征圖，每組分別進行卷積， 卷積后得到 4 個特征圖，最后再把 3 個組共 12 個特征圖組合起來，假設特征圖大小是 28× 28，卷積核大小為 5×5，Same Padding，這里卷積層權值數量為 5×5×2×4× 3+12=612，乘法計算量為 5×5×28×28×2×4×3=470400，權值數量和計算量都約為普通 卷積的 1/3，
分組卷積程序也可以描述如下：
1、假設輸入的形狀為 H × W × C ，用 k 個 h × w 的卷積核對其進行卷積操作；
2、把輸入分為g組，每組形狀為H × W × ( C / g )(假設可以整除)
3、把卷積核也分為g組，每組為k/g(假設可整除)個h × w 卷積核
4、按順序，每組的輸入和該組內的卷積核分別做標準卷積操作，輸出 g 組形狀為H ′ × W ′ × ( k / g ) ；
5、將這 g 組特征合并起來,得到最終形狀為H ′ × W ′ × k 的特征;
下圖為分兩組時的舉例：
或者看下圖
左邊標準卷積，每個卷積核處理12個通道
右邊分組卷積，假設輸入的12個通道分為3組，每個卷積核只處理4個通道
3、Channel Shuffle(通道重排)
圖1
圖1（a）說明了兩個堆疊組卷積層的情況，很明顯，某個組的輸出只與組內的輸入有關，此屬性會阻塞通道組之間的資訊流并削弱表示，
??如果我們允許組卷積從不同組中獲取輸入資料（如圖 1（b）所示），輸入和輸出通道將完全相關，
??對于上一層生成的特征圖，我們可以先將每組中的通道劃分為幾個子組，然后將不同的子組饋入下一層中的每個組，這可以通過通道混洗操作有效而優雅地實作（圖 1（c））：假設一個卷積層具有 g 個組，其輸出有 g × n 個通道；我們首先將輸出通道維度重塑為 (g, n)，轉置然后將其展平作為下一層的輸入，請注意，即使兩個卷積的組數不同，該操作仍然有效，此外，channel shuffle 也是可微分的，這意味著它可以嵌入到網路結構中進行端到端訓練，
??舉個例子來說，如下圖，分組卷積生成的三組特征圖，第一組1~4；第二組5~8；第三組9~12，先將特征圖重塑，為三行N列的矩形，然后進行轉置，變成N行三列，最后壓平，從二維tensor變成一維tensor，每一組的特征圖交叉組合在一起，實作各組之間的資訊交融，
4、ShuffleNet Unit
圖2
圖 2. ShuffleNet 單元
a) 具有深度卷積 (DWConv) [3, 12] 的殘差單元 [9]；
b) 具有逐點組卷積 (GConv) 和通道混洗的 ShuffleNet 單元；
c) ShuffleNet 單元，stride = 2.
這里需要注意(b)中第一個分組卷積降維，然后通道重排，在進行3*3的DepthWise卷積，最后使用1*1的分組卷積升維，目的還是讓殘差前后的shape一致，保證可以進行Add操作，
???圖是ShuffleNet的下采樣模塊，注意，這里左右分支不是使用Add操作，因為最后左右分支的shape是不一致的，右分支的通道數和左分支的通道數疊加 == 輸出特征圖的通道數out_channel(重點，和上面的殘差是不一樣的)，殘差邊上使用了池化視窗為3*3，stride=2的平均池化Add是逐元素求個，Concat是某個方向的疊加，這里是再通道方向疊加，
5、ShuffleNet網路結構設計
上圖為不同分組數的ShuffleNet網路結構，通常我們將g=3的那一個網路作為baseline Network
我們在表 1 中展示了整體 ShuffleNet 架構，所提出的網路主要由一組 ShuffleNet 單元組成，分為三個階段，每個階段的第一個構建塊應用 stride = 2，一個階段內的其他超引數保持不變，下一個階段的輸出通道加倍，與 [9] 類似，我們將每個 ShuffleNet 的瓶頸通道數設定為輸出通道的 1/4
??上表中stage2的第一個block上不用GConv,用普通的1*1卷積，因為此時輸入通道數只有24，太少了，且每個stage中的第一個block的stride=2（對應ShuffleNet Unit中的下采樣模塊,c圖），其他block的stride=1(對應ShuffleNet基本模塊，圖b)
6、代碼復現
這里只是簡單復現下，細節問題并沒有管，比如那個stage2的第一個block上不用GConv,用普通的1*1卷積就沒管，
1 import tensorflow as tf 2 from tensorflow.keras.layers import concatenate, Conv2D, Activation, BatchNormalization, DepthwiseConv2D 3 from tensorflow.keras.layers import add, AvgPool2D,MaxPool2D,GlobalAveragePooling2D,Dense 4 from tensorflow.keras.models import Model 5 from plot_model import plot_model
6.1 Channel Shuffle模塊
1 # 通道重排,跨組資訊互動 2 def channel_shuffle(inputs, num_groups): 3 # 先得到輸入特征圖的shape，b:batch size，h,w:一張圖的size，c:通道數 4 b, h, w, c = inputs.shape 5 6 # 確定shape = [b, h, w, num_groups, c//num_groups]，通道維度原來是一個長為c的一維tensor，變成num_groups行n列的矩陣 7 # 在通道維度上將特征圖reshape為num_groups行n列的矩陣 8 x_reshaped = tf.reshape(inputs, [-1, h, w, num_groups, c // num_groups]) 9 10 # 確定轉置的矩形的shape = [b, h, w, c//num_groups, num_groups] 11 # 矩陣轉置，最后兩個維度從num_groups行n列變成n行num_groups列 12 x_transposed = tf.transpose(x_reshaped, [0, 1, 2, 4, 3]) 13 14 # 重新排列，shotcut和x的通道像素交叉排列，通道維度重新變成一維tensor 15 output = tf.reshape(x_transposed, [-1, h, w, c]) 16 # 回傳通道維度交叉排序后的tensor 17 return output
6.2 分組卷積模塊
我這里tensorflow版本為2.0，我看官網API中高版本的Conv2D引數中已經有了groups屬性了，
1 def group_conv(inputs, filters, kernel, strides, num_groups): 2 conv_side_layers_tmp = tf.split(inputs, num_groups, axis=3) 3 conv_side_layers = [] 4 for layer in conv_side_layers_tmp: 5 conv_side_layers.append(tf.keras.layers.Conv2D(filters // num_groups, kernel, strides, padding='same')(layer)) 6 x = concatenate(conv_side_layers, axis=-1) 7 8 return x
6.3 普通卷積模塊
1 # 普通卷積：卷積+批標準化+ReLU激活 2 def conv(inputs, filters, kernel_size, stride=1): 3 x = Conv2D(filters, kernel_size, stride, padding='same', use_bias=False)(inputs) 4 x = BatchNormalization()(x) 5 x = Activation('relu')(x) 6 return x
6.4 DepthWise卷積
1 # DWConv:深度可分離卷積塊(論文中DWConv卷積核全是3*3,步長有1和2兩種) 2 def depthwise_conv_bn(inputs, kernel_size, stride=1): 3 x = DepthwiseConv2D(kernel_size=kernel_size, 4 strides=stride, 5 padding='same', 6 use_bias=False)(inputs) 7 x = BatchNormalization()(x) 8 return x
注意，這里沒有用ReLU
6.5 ShuffleNetV1基本模塊
1 # ShuffleNetV1基本模塊(Add) 2 def shuffleNetUnitA(inputs, num_groups): 3 in_channels = inputs.shape[-1] 4 out_channels = in_channels 5 bottleneck_channels = out_channels // 4 6 7 # 1*1分組卷積降維 8 x = group_conv(inputs, bottleneck_channels, kernel=1, strides=1, num_groups=num_groups) 9 x = BatchNormalization()(x) 10 x = Activation('relu')(x) 11 # Channel Shuffle 12 x = channel_shuffle(x, num_groups) 13 # 3*3 DWConv 14 x = depthwise_conv_bn(x, kernel_size=3, stride=1) 15 # 1*1分組卷積升維(要保證殘差連接前后的shape一致) 16 x = group_conv(x, out_channels, kernel=1, strides=1, num_groups=num_groups) 17 x = BatchNormalization()(x) 18 x = add([inputs, x]) 19 x = Activation('relu')(x) 20 return x
6.6 ShuffleNetV1下采樣模塊
1 # ShuffleNetV1下采樣模塊(下采樣模塊，concat) 2 def shuffleNetUnitB(inputs, out_channels, num_groups): 3 in_channels = inputs.shape[-1] 4 # 右分支的通道數和左分支的通道數疊加 == 輸出特征圖的通道數out_channel(重點，和上面的殘差是不一樣的) 5 out_channels -= in_channels 6 bottleneck_channels = out_channels // 4 7 # （1）右分支 8 # 1*1 GConv 9 x = group_conv(inputs, bottleneck_channels, kernel=1, strides=1, num_groups=num_groups) 10 x = BatchNormalization()(x) 11 x = Activation('relu')(x) 12 # Channel Shuffle 13 x = channel_shuffle(x, num_groups) 14 # 3*3 DWConv,stide=2 15 x = depthwise_conv_bn(x, kernel_size=3, stride=2) 16 # 1*1 GConv 17 x = group_conv(x, out_channels, kernel=1, strides=1, num_groups=num_groups) 18 x = BatchNormalization()(x) 19 20 # (2)左分支:3*3 AVG Pool,stride=2 21 y = AvgPool2D(pool_size=3, strides=2, padding='same')(inputs) 22 # 在通道維度上堆疊 23 x = concatenate([y, x], axis=-1) 24 x = Activation('relu')(x) 25 return x
6.5和6.6的通道數需要結合論文好好看，要不你看不懂為什么會這樣設計，原論文中都有解釋，
6.6 stage
??每個stage中的第一個block的stride=2(即下采樣模塊)，其他block的stride=1(即基本模塊)
1 def stage(inputs, out_channels, num_groups, n): 2 # 每個stage中的第一個block的stride=2(即下采樣模塊)，其他block的stride=1(即基本模塊) 3 # 都是按照論文搭建的，要去看論文原文，要不你絕對不理解為什么這樣搭建，嘿嘿， 4 x = shuffleNetUnitB(inputs, out_channels, num_groups) 5 6 for _ in range(n): 7 x = shuffleNetUnitA(x, num_groups) 8 return x
6.7 網路搭建
1 # first_stage_channels為第一個stage的輸出通道數 2 # num_groups為分組數量 3 def ShuffleNet(inputs, first_stage_channels, num_groups,num_classes): 4 # 構建網路輸入tensor 5 inputs = tf.keras.Input(shape=inputs) 6 # 論文中先用了一個普通卷積和池化 7 x = Conv2D(filters=24, 8 kernel_size=3, 9 strides=2, 10 padding='same')(inputs) 11 x = MaxPool2D(pool_size=3, strides=2, padding='same')(x) 12 # 三個stage,每個stage的第一個block的stride=2 13 # 同一個stage內的其他超引數不變，下一個stage的輸出通道數加倍(這個可以通過論文中的表格看出，原文也給了) 14 # n為分組卷積的分組數量，論文中用g表示 15 x = stage(x, first_stage_channels, num_groups, n=3) 16 x = stage(x, first_stage_channels * 2, num_groups, n=7) 17 x = stage(x, first_stage_channels * 4, num_groups, n=3) 18 19 x = GlobalAveragePooling2D()(x) 20 # 我看過其他大佬的文章說compile的時候再用softmax，那樣更穩定，有時間再試試吧 21 x = Dense(num_classes, activation='softmax')(x) 22 23 # 完整網路架構 24 model = Model(inputs=inputs, outputs=x) 25 return model
6.8 自定義資料集測驗
1 # 類別數 2 num_classes = 17 3 # 批次大小 4 batch_size = 32 5 # 周期數 6 epochs = 100 7 # 圖片大小 8 image_size = 224
查看模型摘要
1 model=ShuffleNet(inputs[224,224,3],first_stage_channels=240,num_groups=3,num_classes=17) 2 model.summary()
1 Model: "functional_1" 2 __________________________________________________________________________________________________ 3 Layer (type) Output Shape Param # Connected to 4 ================================================================================================== 5 input_1 (InputLayer) [(None, 224, 224, 3) 0 6 __________________________________________________________________________________________________ 7 conv2d (Conv2D) (None, 112, 112, 24) 672 input_1[0][0] 8 __________________________________________________________________________________________________ 9 max_pooling2d (MaxPooling2D) (None, 56, 56, 24) 0 conv2d[0][0] 10 __________________________________________________________________________________________________ 11 tf_op_layer_split (TensorFlowOp [(None, 56, 56, 8), 0 max_pooling2d[0][0] 12 __________________________________________________________________________________________________ 13 conv2d_1 (Conv2D) (None, 56, 56, 18) 162 tf_op_layer_split[0][0] 14 __________________________________________________________________________________________________ 15 conv2d_2 (Conv2D) (None, 56, 56, 18) 162 tf_op_layer_split[0][1] 16 __________________________________________________________________________________________________ 17 conv2d_3 (Conv2D) (None, 56, 56, 18) 162 tf_op_layer_split[0][2] 18 __________________________________________________________________________________________________ 19 concatenate (Concatenate) (None, 56, 56, 54) 0 conv2d_1[0][0] 20 conv2d_2[0][0] 21 conv2d_3[0][0] 22 __________________________________________________________________________________________________ 23 batch_normalization (BatchNorma (None, 56, 56, 54) 216 concatenate[0][0] 24 __________________________________________________________________________________________________ 25 activation (Activation) (None, 56, 56, 54) 0 batch_normalization[0][0] 26 __________________________________________________________________________________________________ 27 tf_op_layer_Reshape (TensorFlow [(None, 56, 56, 3, 1 0 activation[0][0] 28 __________________________________________________________________________________________________ 29 tf_op_layer_Transpose (TensorFl [(None, 56, 56, 18, 0 tf_op_layer_Reshape[0][0] 30 __________________________________________________________________________________________________ 31 tf_op_layer_Reshape_1 (TensorFl [(None, 56, 56, 54)] 0 tf_op_layer_Transpose[0][0] 32 __________________________________________________________________________________________________ 33 depthwise_conv2d (DepthwiseConv (None, 28, 28, 54) 486 tf_op_layer_Reshape_1[0][0] 34 __________________________________________________________________________________________________ 35 batch_normalization_1 (BatchNor (None, 28, 28, 54) 216 depthwise_conv2d[0][0] 36 __________________________________________________________________________________________________ 37 tf_op_layer_split_1 (TensorFlow [(None, 28, 28, 18), 0 batch_normalization_1[0][0] 38 __________________________________________________________________________________________________ 39 conv2d_4 (Conv2D) (None, 28, 28, 72) 1368 tf_op_layer_split_1[0][0] 40 __________________________________________________________________________________________________ 41 conv2d_5 (Conv2D) (None, 28, 28, 72) 1368 tf_op_layer_split_1[0][1] 42 __________________________________________________________________________________________________ 43 conv2d_6 (Conv2D) (None, 28, 28, 72) 1368 tf_op_layer_split_1[0][2] 44 __________________________________________________________________________________________________ 45 concatenate_1 (Concatenate) (None, 28, 28, 216) 0 conv2d_4[0][0] 46 conv2d_5[0][0] 47 conv2d_6[0][0] 48 __________________________________________________________________________________________________ 49 average_pooling2d (AveragePooli (None, 28, 28, 24) 0 max_pooling2d[0][0] 50 __________________________________________________________________________________________________ 51 batch_normalization_2 (BatchNor (None, 28, 28, 216) 864 concatenate_1[0][0] 52 __________________________________________________________________________________________________ 53 concatenate_2 (Concatenate) (None, 28, 28, 240) 0 average_pooling2d[0][0] 54 batch_normalization_2[0][0] 55 __________________________________________________________________________________________________ 56 activation_1 (Activation) (None, 28, 28, 240) 0 concatenate_2[0][0] 57 __________________________________________________________________________________________________ 58 tf_op_layer_split_2 (TensorFlow [(None, 28, 28, 80), 0 activation_1[0][0] 59 __________________________________________________________________________________________________ 60 conv2d_7 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_2[0][0] 61 __________________________________________________________________________________________________ 62 conv2d_8 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_2[0][1] 63 __________________________________________________________________________________________________ 64 conv2d_9 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_2[0][2] 65 __________________________________________________________________________________________________ 66 concatenate_3 (Concatenate) (None, 28, 28, 60) 0 conv2d_7[0][0] 67 conv2d_8[0][0] 68 conv2d_9[0][0] 69 __________________________________________________________________________________________________ 70 batch_normalization_3 (BatchNor (None, 28, 28, 60) 240 concatenate_3[0][0] 71 __________________________________________________________________________________________________ 72 activation_2 (Activation) (None, 28, 28, 60) 0 batch_normalization_3[0][0] 73 __________________________________________________________________________________________________ 74 tf_op_layer_Reshape_2 (TensorFl [(None, 28, 28, 3, 2 0 activation_2[0][0] 75 __________________________________________________________________________________________________ 76 tf_op_layer_Transpose_1 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_2[0][0] 77 __________________________________________________________________________________________________ 78 tf_op_layer_Reshape_3 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_1[0][0] 79 __________________________________________________________________________________________________ 80 depthwise_conv2d_1 (DepthwiseCo (None, 28, 28, 60) 540 tf_op_layer_Reshape_3[0][0] 81 __________________________________________________________________________________________________ 82 batch_normalization_4 (BatchNor (None, 28, 28, 60) 240 depthwise_conv2d_1[0][0] 83 __________________________________________________________________________________________________ 84 tf_op_layer_split_3 (TensorFlow [(None, 28, 28, 20), 0 batch_normalization_4[0][0] 85 __________________________________________________________________________________________________ 86 conv2d_10 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_3[0][0] 87 __________________________________________________________________________________________________ 88 conv2d_11 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_3[0][1] 89 __________________________________________________________________________________________________ 90 conv2d_12 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_3[0][2] 91 __________________________________________________________________________________________________ 92 concatenate_4 (Concatenate) (None, 28, 28, 240) 0 conv2d_10[0][0] 93 conv2d_11[0][0] 94 conv2d_12[0][0] 95 __________________________________________________________________________________________________ 96 batch_normalization_5 (BatchNor (None, 28, 28, 240) 960 concatenate_4[0][0] 97 __________________________________________________________________________________________________ 98 add (Add) (None, 28, 28, 240) 0 activation_1[0][0] 99 batch_normalization_5[0][0] 100 __________________________________________________________________________________________________ 101 activation_3 (Activation) (None, 28, 28, 240) 0 add[0][0] 102 __________________________________________________________________________________________________ 103 tf_op_layer_split_4 (TensorFlow [(None, 28, 28, 80), 0 activation_3[0][0] 104 __________________________________________________________________________________________________ 105 conv2d_13 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_4[0][0] 106 __________________________________________________________________________________________________ 107 conv2d_14 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_4[0][1] 108 __________________________________________________________________________________________________ 109 conv2d_15 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_4[0][2] 110 __________________________________________________________________________________________________ 111 concatenate_5 (Concatenate) (None, 28, 28, 60) 0 conv2d_13[0][0] 112 conv2d_14[0][0] 113 conv2d_15[0][0] 114 __________________________________________________________________________________________________ 115 batch_normalization_6 (BatchNor (None, 28, 28, 60) 240 concatenate_5[0][0] 116 __________________________________________________________________________________________________ 117 activation_4 (Activation) (None, 28, 28, 60) 0 batch_normalization_6[0][0] 118 __________________________________________________________________________________________________ 119 tf_op_layer_Reshape_4 (TensorFl [(None, 28, 28, 3, 2 0 activation_4[0][0] 120 __________________________________________________________________________________________________ 121 tf_op_layer_Transpose_2 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_4[0][0] 122 __________________________________________________________________________________________________ 123 tf_op_layer_Reshape_5 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_2[0][0] 124 __________________________________________________________________________________________________ 125 depthwise_conv2d_2 (DepthwiseCo (None, 28, 28, 60) 540 tf_op_layer_Reshape_5[0][0] 126 __________________________________________________________________________________________________ 127 batch_normalization_7 (BatchNor (None, 28, 28, 60) 240 depthwise_conv2d_2[0][0] 128 __________________________________________________________________________________________________ 129 tf_op_layer_split_5 (TensorFlow [(None, 28, 28, 20), 0 batch_normalization_7[0][0] 130 __________________________________________________________________________________________________ 131 conv2d_16 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_5[0][0] 132 __________________________________________________________________________________________________ 133 conv2d_17 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_5[0][1] 134 __________________________________________________________________________________________________ 135 conv2d_18 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_5[0][2] 136 __________________________________________________________________________________________________ 137 concatenate_6 (Concatenate) (None, 28, 28, 240) 0 conv2d_16[0][0] 138 conv2d_17[0][0] 139 conv2d_18[0][0] 140 __________________________________________________________________________________________________ 141 batch_normalization_8 (BatchNor (None, 28, 28, 240) 960 concatenate_6[0][0] 142 __________________________________________________________________________________________________ 143 add_1 (Add) (None, 28, 28, 240) 0 activation_3[0][0] 144 batch_normalization_8[0][0] 145 __________________________________________________________________________________________________ 146 activation_5 (Activation) (None, 28, 28, 240) 0 add_1[0][0] 147 __________________________________________________________________________________________________ 148 tf_op_layer_split_6 (TensorFlow [(None, 28, 28, 80), 0 activation_5[0][0] 149 __________________________________________________________________________________________________ 150 conv2d_19 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_6[0][0] 151 __________________________________________________________________________________________________ 152 conv2d_20 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_6[0][1] 153 __________________________________________________________________________________________________ 154 conv2d_21 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_6[0][2] 155 __________________________________________________________________________________________________ 156 concatenate_7 (Concatenate) (None, 28, 28, 60) 0 conv2d_19[0][0] 157 conv2d_20[0][0] 158 conv2d_21[0][0] 159 __________________________________________________________________________________________________ 160 batch_normalization_9 (BatchNor (None, 28, 28, 60) 240 concatenate_7[0][0] 161 __________________________________________________________________________________________________ 162 activation_6 (Activation) (None, 28, 28, 60) 0 batch_normalization_9[0][0] 163 __________________________________________________________________________________________________ 164 tf_op_layer_Reshape_6 (TensorFl [(None, 28, 28, 3, 2 0 activation_6[0][0] 165 __________________________________________________________________________________________________ 166 tf_op_layer_Transpose_3 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_6[0][0] 167 __________________________________________________________________________________________________ 168 tf_op_layer_Reshape_7 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_3[0][0] 169 __________________________________________________________________________________________________ 170 depthwise_conv2d_3 (DepthwiseCo (None, 28, 28, 60) 540 tf_op_layer_Reshape_7[0][0] 171 __________________________________________________________________________________________________ 172 batch_normalization_10 (BatchNo (None, 28, 28, 60) 240 depthwise_conv2d_3[0][0] 173 __________________________________________________________________________________________________ 174 tf_op_layer_split_7 (TensorFlow [(None, 28, 28, 20), 0 batch_normalization_10[0][0] 175 __________________________________________________________________________________________________ 176 conv2d_22 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_7[0][0] 177 __________________________________________________________________________________________________ 178 conv2d_23 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_7[0][1] 179 __________________________________________________________________________________________________ 180 conv2d_24 (Conv2D) (None, 28, 28, 80) 1680 tf_op_layer_split_7[0][2] 181 __________________________________________________________________________________________________ 182 concatenate_8 (Concatenate) (None, 28, 28, 240) 0 conv2d_22[0][0] 183 conv2d_23[0][0] 184 conv2d_24[0][0] 185 __________________________________________________________________________________________________ 186 batch_normalization_11 (BatchNo (None, 28, 28, 240) 960 concatenate_8[0][0] 187 __________________________________________________________________________________________________ 188 add_2 (Add) (None, 28, 28, 240) 0 activation_5[0][0] 189 batch_normalization_11[0][0] 190 __________________________________________________________________________________________________ 191 activation_7 (Activation) (None, 28, 28, 240) 0 add_2[0][0] 192 __________________________________________________________________________________________________ 193 tf_op_layer_split_8 (TensorFlow [(None, 28, 28, 80), 0 activation_7[0][0] 194 __________________________________________________________________________________________________ 195 conv2d_25 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_8[0][0] 196 __________________________________________________________________________________________________ 197 conv2d_26 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_8[0][1] 198 __________________________________________________________________________________________________ 199 conv2d_27 (Conv2D) (None, 28, 28, 20) 1620 tf_op_layer_split_8[0][2] 200 __________________________________________________________________________________________________ 201 concatenate_9 (Concatenate) (None, 28, 28, 60) 0 conv2d_25[0][0] 202 conv2d_26[0][0] 203 conv2d_27[0][0] 204 __________________________________________________________________________________________________ 205 batch_normalization_12 (BatchNo (None, 28, 28, 60) 240 concatenate_9[0][0] 206 __________________________________________________________________________________________________ 207 activation_8 (Activation) (None, 28, 28, 60) 0 batch_normalization_12[0][0] 208 __________________________________________________________________________________________________ 209 tf_op_layer_Reshape_8 (TensorFl [(None, 28, 28, 3, 2 0 activation_8[0][0] 210 __________________________________________________________________________________________________ 211 tf_op_layer_Transpose_4 (Tensor [(None, 28, 28, 20, 0 tf_op_layer_Reshape_8[0][0] 212 __________________________________________________________________________________________________ 213 tf_op_layer_Reshape_9 (TensorFl [(None, 28, 28, 60)] 0 tf_op_layer_Transpose_4[0][0] 214 __________________________________________________________________________________________________ 215 depthwise_conv2d_4 (DepthwiseCo (None, 14, 14, 60) 540 tf_op_layer_Reshape_9[0][0] 216 __________________________________________________________________________________________________ 217 batch_normalization_13 (BatchNo (None, 14, 14, 60) 240 depthwise_conv2d_4[0][0] 218 __________________________________________________________________________________________________ 219 tf_op_layer_split_9 (TensorFlow [(None, 14, 14, 20), 0 batch_normalization_13[0][0] 220 __________________________________________________________________________________________________ 221 conv2d_28 (Conv2D) (None, 14, 14, 80) 1680 tf_op_layer_split_9[0][0] 222 __________________________________________________________________________________________________ 223 conv2d_29 (Conv2D) (None, 14, 14, 80) 1680 tf_op_layer_split_9[0][1] 224 __________________________________________________________________________________________________ 225 conv2d_30 (Conv2D) (None, 14, 14, 80) 1680 tf_op_layer_split_9[0][2] 226 __________________________________________________________________________________________________ 227 concatenate_10 (Concatenate) (None, 14, 14, 240) 0 conv2d_28[0][0] 228 conv2d_29[0][0] 229 conv2d_30[0][0] 230 __________________________________________________________________________________________________ 231 average_pooling2d_1 (AveragePoo (None, 14, 14, 240) 0 activation_7[0][0] 232 __________________________________________________________________________________________________ 233 batch_normalization_14 (BatchNo (None, 14, 14, 240) 960 concatenate_10[0][0] 234 __________________________________________________________________________________________________ 235 concatenate_11 (Concatenate) (None, 14, 14, 480) 0 average_pooling2d_1[0][0] 236 batch_normalization_14[0][0] 237 __________________________________________________________________________________________________ 238 activation_9 (Activation) (None, 14, 14, 480) 0 concatenate_11[0][0] 239 __________________________________________________________________________________________________ 240 tf_op_layer_split_10 (TensorFlo [(None, 14, 14, 160) 0 activation_9[0][0] 241 __________________________________________________________________________________________________ 242 conv2d_31 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_10[0][0] 243 __________________________________________________________________________________________________ 244 conv2d_32 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_10[0][1] 245 __________________________________________________________________________________________________ 246 conv2d_33 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_10[0][2] 247 __________________________________________________________________________________________________ 248 concatenate_12 (Concatenate) (None, 14, 14, 120) 0 conv2d_31[0][0] 249 conv2d_32[0][0] 250 conv2d_33[0][0] 251 __________________________________________________________________________________________________ 252 batch_normalization_15 (BatchNo (None, 14, 14, 120) 480 concatenate_12[0][0] 253 __________________________________________________________________________________________________ 254 activation_10 (Activation) (None, 14, 14, 120) 0 batch_normalization_15[0][0] 255 __________________________________________________________________________________________________ 256 tf_op_layer_Reshape_10 (TensorF [(None, 14, 14, 3, 4 0 activation_10[0][0] 257 __________________________________________________________________________________________________ 258 tf_op_layer_Transpose_5 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_10[0][0] 259 __________________________________________________________________________________________________ 260 tf_op_layer_Reshape_11 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_5[0][0] 261 __________________________________________________________________________________________________ 262 depthwise_conv2d_5 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_11[0][0] 263 __________________________________________________________________________________________________ 264 batch_normalization_16 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_5[0][0] 265 __________________________________________________________________________________________________ 266 tf_op_layer_split_11 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_16[0][0] 267 __________________________________________________________________________________________________ 268 conv2d_34 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_11[0][0] 269 __________________________________________________________________________________________________ 270 conv2d_35 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_11[0][1] 271 __________________________________________________________________________________________________ 272 conv2d_36 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_11[0][2] 273 __________________________________________________________________________________________________ 274 concatenate_13 (Concatenate) (None, 14, 14, 480) 0 conv2d_34[0][0] 275 conv2d_35[0][0] 276 conv2d_36[0][0] 277 __________________________________________________________________________________________________ 278 batch_normalization_17 (BatchNo (None, 14, 14, 480) 1920 concatenate_13[0][0] 279 __________________________________________________________________________________________________ 280 add_3 (Add) (None, 14, 14, 480) 0 activation_9[0][0] 281 batch_normalization_17[0][0] 282 __________________________________________________________________________________________________ 283 activation_11 (Activation) (None, 14, 14, 480) 0 add_3[0][0] 284 __________________________________________________________________________________________________ 285 tf_op_layer_split_12 (TensorFlo [(None, 14, 14, 160) 0 activation_11[0][0] 286 __________________________________________________________________________________________________ 287 conv2d_37 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_12[0][0] 288 __________________________________________________________________________________________________ 289 conv2d_38 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_12[0][1] 290 __________________________________________________________________________________________________ 291 conv2d_39 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_12[0][2] 292 __________________________________________________________________________________________________ 293 concatenate_14 (Concatenate) (None, 14, 14, 120) 0 conv2d_37[0][0] 294 conv2d_38[0][0] 295 conv2d_39[0][0] 296 __________________________________________________________________________________________________ 297 batch_normalization_18 (BatchNo (None, 14, 14, 120) 480 concatenate_14[0][0] 298 __________________________________________________________________________________________________ 299 activation_12 (Activation) (None, 14, 14, 120) 0 batch_normalization_18[0][0] 300 __________________________________________________________________________________________________ 301 tf_op_layer_Reshape_12 (TensorF [(None, 14, 14, 3, 4 0 activation_12[0][0] 302 __________________________________________________________________________________________________ 303 tf_op_layer_Transpose_6 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_12[0][0] 304 __________________________________________________________________________________________________ 305 tf_op_layer_Reshape_13 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_6[0][0] 306 __________________________________________________________________________________________________ 307 depthwise_conv2d_6 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_13[0][0] 308 __________________________________________________________________________________________________ 309 batch_normalization_19 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_6[0][0] 310 __________________________________________________________________________________________________ 311 tf_op_layer_split_13 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_19[0][0] 312 __________________________________________________________________________________________________ 313 conv2d_40 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_13[0][0] 314 __________________________________________________________________________________________________ 315 conv2d_41 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_13[0][1] 316 __________________________________________________________________________________________________ 317 conv2d_42 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_13[0][2] 318 __________________________________________________________________________________________________ 319 concatenate_15 (Concatenate) (None, 14, 14, 480) 0 conv2d_40[0][0] 320 conv2d_41[0][0] 321 conv2d_42[0][0] 322 __________________________________________________________________________________________________ 323 batch_normalization_20 (BatchNo (None, 14, 14, 480) 1920 concatenate_15[0][0] 324 __________________________________________________________________________________________________ 325 add_4 (Add) (None, 14, 14, 480) 0 activation_11[0][0] 326 batch_normalization_20[0][0] 327 __________________________________________________________________________________________________ 328 activation_13 (Activation) (None, 14, 14, 480) 0 add_4[0][0] 329 __________________________________________________________________________________________________ 330 tf_op_layer_split_14 (TensorFlo [(None, 14, 14, 160) 0 activation_13[0][0] 331 __________________________________________________________________________________________________ 332 conv2d_43 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_14[0][0] 333 __________________________________________________________________________________________________ 334 conv2d_44 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_14[0][1] 335 __________________________________________________________________________________________________ 336 conv2d_45 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_14[0][2] 337 __________________________________________________________________________________________________ 338 concatenate_16 (Concatenate) (None, 14, 14, 120) 0 conv2d_43[0][0] 339 conv2d_44[0][0] 340 conv2d_45[0][0] 341 __________________________________________________________________________________________________ 342 batch_normalization_21 (BatchNo (None, 14, 14, 120) 480 concatenate_16[0][0] 343 __________________________________________________________________________________________________ 344 activation_14 (Activation) (None, 14, 14, 120) 0 batch_normalization_21[0][0] 345 __________________________________________________________________________________________________ 346 tf_op_layer_Reshape_14 (TensorF [(None, 14, 14, 3, 4 0 activation_14[0][0] 347 __________________________________________________________________________________________________ 348 tf_op_layer_Transpose_7 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_14[0][0] 349 __________________________________________________________________________________________________ 350 tf_op_layer_Reshape_15 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_7[0][0] 351 __________________________________________________________________________________________________ 352 depthwise_conv2d_7 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_15[0][0] 353 __________________________________________________________________________________________________ 354 batch_normalization_22 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_7[0][0] 355 __________________________________________________________________________________________________ 356 tf_op_layer_split_15 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_22[0][0] 357 __________________________________________________________________________________________________ 358 conv2d_46 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_15[0][0] 359 __________________________________________________________________________________________________ 360 conv2d_47 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_15[0][1] 361 __________________________________________________________________________________________________ 362 conv2d_48 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_15[0][2] 363 __________________________________________________________________________________________________ 364 concatenate_17 (Concatenate) (None, 14, 14, 480) 0 conv2d_46[0][0] 365 conv2d_47[0][0] 366 conv2d_48[0][0] 367 __________________________________________________________________________________________________ 368 batch_normalization_23 (BatchNo (None, 14, 14, 480) 1920 concatenate_17[0][0] 369 __________________________________________________________________________________________________ 370 add_5 (Add) (None, 14, 14, 480) 0 activation_13[0][0] 371 batch_normalization_23[0][0] 372 __________________________________________________________________________________________________ 373 activation_15 (Activation) (None, 14, 14, 480) 0 add_5[0][0] 374 __________________________________________________________________________________________________ 375 tf_op_layer_split_16 (TensorFlo [(None, 14, 14, 160) 0 activation_15[0][0] 376 __________________________________________________________________________________________________ 377 conv2d_49 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_16[0][0] 378 __________________________________________________________________________________________________ 379 conv2d_50 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_16[0][1] 380 __________________________________________________________________________________________________ 381 conv2d_51 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_16[0][2] 382 __________________________________________________________________________________________________ 383 concatenate_18 (Concatenate) (None, 14, 14, 120) 0 conv2d_49[0][0] 384 conv2d_50[0][0] 385 conv2d_51[0][0] 386 __________________________________________________________________________________________________ 387 batch_normalization_24 (BatchNo (None, 14, 14, 120) 480 concatenate_18[0][0] 388 __________________________________________________________________________________________________ 389 activation_16 (Activation) (None, 14, 14, 120) 0 batch_normalization_24[0][0] 390 __________________________________________________________________________________________________ 391 tf_op_layer_Reshape_16 (TensorF [(None, 14, 14, 3, 4 0 activation_16[0][0] 392 __________________________________________________________________________________________________ 393 tf_op_layer_Transpose_8 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_16[0][0] 394 __________________________________________________________________________________________________ 395 tf_op_layer_Reshape_17 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_8[0][0] 396 __________________________________________________________________________________________________ 397 depthwise_conv2d_8 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_17[0][0] 398 __________________________________________________________________________________________________ 399 batch_normalization_25 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_8[0][0] 400 __________________________________________________________________________________________________ 401 tf_op_layer_split_17 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_25[0][0] 402 __________________________________________________________________________________________________ 403 conv2d_52 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_17[0][0] 404 __________________________________________________________________________________________________ 405 conv2d_53 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_17[0][1] 406 __________________________________________________________________________________________________ 407 conv2d_54 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_17[0][2] 408 __________________________________________________________________________________________________ 409 concatenate_19 (Concatenate) (None, 14, 14, 480) 0 conv2d_52[0][0] 410 conv2d_53[0][0] 411 conv2d_54[0][0] 412 __________________________________________________________________________________________________ 413 batch_normalization_26 (BatchNo (None, 14, 14, 480) 1920 concatenate_19[0][0] 414 __________________________________________________________________________________________________ 415 add_6 (Add) (None, 14, 14, 480) 0 activation_15[0][0] 416 batch_normalization_26[0][0] 417 __________________________________________________________________________________________________ 418 activation_17 (Activation) (None, 14, 14, 480) 0 add_6[0][0] 419 __________________________________________________________________________________________________ 420 tf_op_layer_split_18 (TensorFlo [(None, 14, 14, 160) 0 activation_17[0][0] 421 __________________________________________________________________________________________________ 422 conv2d_55 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_18[0][0] 423 __________________________________________________________________________________________________ 424 conv2d_56 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_18[0][1] 425 __________________________________________________________________________________________________ 426 conv2d_57 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_18[0][2] 427 __________________________________________________________________________________________________ 428 concatenate_20 (Concatenate) (None, 14, 14, 120) 0 conv2d_55[0][0] 429 conv2d_56[0][0] 430 conv2d_57[0][0] 431 __________________________________________________________________________________________________ 432 batch_normalization_27 (BatchNo (None, 14, 14, 120) 480 concatenate_20[0][0] 433 __________________________________________________________________________________________________ 434 activation_18 (Activation) (None, 14, 14, 120) 0 batch_normalization_27[0][0] 435 __________________________________________________________________________________________________ 436 tf_op_layer_Reshape_18 (TensorF [(None, 14, 14, 3, 4 0 activation_18[0][0] 437 __________________________________________________________________________________________________ 438 tf_op_layer_Transpose_9 (Tensor [(None, 14, 14, 40, 0 tf_op_layer_Reshape_18[0][0] 439 __________________________________________________________________________________________________ 440 tf_op_layer_Reshape_19 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_9[0][0] 441 __________________________________________________________________________________________________ 442 depthwise_conv2d_9 (DepthwiseCo (None, 14, 14, 120) 1080 tf_op_layer_Reshape_19[0][0] 443 __________________________________________________________________________________________________ 444 batch_normalization_28 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_9[0][0] 445 __________________________________________________________________________________________________ 446 tf_op_layer_split_19 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_28[0][0] 447 __________________________________________________________________________________________________ 448 conv2d_58 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_19[0][0] 449 __________________________________________________________________________________________________ 450 conv2d_59 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_19[0][1] 451 __________________________________________________________________________________________________ 452 conv2d_60 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_19[0][2] 453 __________________________________________________________________________________________________ 454 concatenate_21 (Concatenate) (None, 14, 14, 480) 0 conv2d_58[0][0] 455 conv2d_59[0][0] 456 conv2d_60[0][0] 457 __________________________________________________________________________________________________ 458 batch_normalization_29 (BatchNo (None, 14, 14, 480) 1920 concatenate_21[0][0] 459 __________________________________________________________________________________________________ 460 add_7 (Add) (None, 14, 14, 480) 0 activation_17[0][0] 461 batch_normalization_29[0][0] 462 __________________________________________________________________________________________________ 463 activation_19 (Activation) (None, 14, 14, 480) 0 add_7[0][0] 464 __________________________________________________________________________________________________ 465 tf_op_layer_split_20 (TensorFlo [(None, 14, 14, 160) 0 activation_19[0][0] 466 __________________________________________________________________________________________________ 467 conv2d_61 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_20[0][0] 468 __________________________________________________________________________________________________ 469 conv2d_62 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_20[0][1] 470 __________________________________________________________________________________________________ 471 conv2d_63 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_20[0][2] 472 __________________________________________________________________________________________________ 473 concatenate_22 (Concatenate) (None, 14, 14, 120) 0 conv2d_61[0][0] 474 conv2d_62[0][0] 475 conv2d_63[0][0] 476 __________________________________________________________________________________________________ 477 batch_normalization_30 (BatchNo (None, 14, 14, 120) 480 concatenate_22[0][0] 478 __________________________________________________________________________________________________ 479 activation_20 (Activation) (None, 14, 14, 120) 0 batch_normalization_30[0][0] 480 __________________________________________________________________________________________________ 481 tf_op_layer_Reshape_20 (TensorF [(None, 14, 14, 3, 4 0 activation_20[0][0] 482 __________________________________________________________________________________________________ 483 tf_op_layer_Transpose_10 (Tenso [(None, 14, 14, 40, 0 tf_op_layer_Reshape_20[0][0] 484 __________________________________________________________________________________________________ 485 tf_op_layer_Reshape_21 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_10[0][0] 486 __________________________________________________________________________________________________ 487 depthwise_conv2d_10 (DepthwiseC (None, 14, 14, 120) 1080 tf_op_layer_Reshape_21[0][0] 488 __________________________________________________________________________________________________ 489 batch_normalization_31 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_10[0][0] 490 __________________________________________________________________________________________________ 491 tf_op_layer_split_21 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_31[0][0] 492 __________________________________________________________________________________________________ 493 conv2d_64 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_21[0][0] 494 __________________________________________________________________________________________________ 495 conv2d_65 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_21[0][1] 496 __________________________________________________________________________________________________ 497 conv2d_66 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_21[0][2] 498 __________________________________________________________________________________________________ 499 concatenate_23 (Concatenate) (None, 14, 14, 480) 0 conv2d_64[0][0] 500 conv2d_65[0][0] 501 conv2d_66[0][0] 502 __________________________________________________________________________________________________ 503 batch_normalization_32 (BatchNo (None, 14, 14, 480) 1920 concatenate_23[0][0] 504 __________________________________________________________________________________________________ 505 add_8 (Add) (None, 14, 14, 480) 0 activation_19[0][0] 506 batch_normalization_32[0][0] 507 __________________________________________________________________________________________________ 508 activation_21 (Activation) (None, 14, 14, 480) 0 add_8[0][0] 509 __________________________________________________________________________________________________ 510 tf_op_layer_split_22 (TensorFlo [(None, 14, 14, 160) 0 activation_21[0][0] 511 __________________________________________________________________________________________________ 512 conv2d_67 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_22[0][0] 513 __________________________________________________________________________________________________ 514 conv2d_68 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_22[0][1] 515 __________________________________________________________________________________________________ 516 conv2d_69 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_22[0][2] 517 __________________________________________________________________________________________________ 518 concatenate_24 (Concatenate) (None, 14, 14, 120) 0 conv2d_67[0][0] 519 conv2d_68[0][0] 520 conv2d_69[0][0] 521 __________________________________________________________________________________________________ 522 batch_normalization_33 (BatchNo (None, 14, 14, 120) 480 concatenate_24[0][0] 523 __________________________________________________________________________________________________ 524 activation_22 (Activation) (None, 14, 14, 120) 0 batch_normalization_33[0][0] 525 __________________________________________________________________________________________________ 526 tf_op_layer_Reshape_22 (TensorF [(None, 14, 14, 3, 4 0 activation_22[0][0] 527 __________________________________________________________________________________________________ 528 tf_op_layer_Transpose_11 (Tenso [(None, 14, 14, 40, 0 tf_op_layer_Reshape_22[0][0] 529 __________________________________________________________________________________________________ 530 tf_op_layer_Reshape_23 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_11[0][0] 531 __________________________________________________________________________________________________ 532 depthwise_conv2d_11 (DepthwiseC (None, 14, 14, 120) 1080 tf_op_layer_Reshape_23[0][0] 533 __________________________________________________________________________________________________ 534 batch_normalization_34 (BatchNo (None, 14, 14, 120) 480 depthwise_conv2d_11[0][0] 535 __________________________________________________________________________________________________ 536 tf_op_layer_split_23 (TensorFlo [(None, 14, 14, 40), 0 batch_normalization_34[0][0] 537 __________________________________________________________________________________________________ 538 conv2d_70 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_23[0][0] 539 __________________________________________________________________________________________________ 540 conv2d_71 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_23[0][1] 541 __________________________________________________________________________________________________ 542 conv2d_72 (Conv2D) (None, 14, 14, 160) 6560 tf_op_layer_split_23[0][2] 543 __________________________________________________________________________________________________ 544 concatenate_25 (Concatenate) (None, 14, 14, 480) 0 conv2d_70[0][0] 545 conv2d_71[0][0] 546 conv2d_72[0][0] 547 __________________________________________________________________________________________________ 548 batch_normalization_35 (BatchNo (None, 14, 14, 480) 1920 concatenate_25[0][0] 549 __________________________________________________________________________________________________ 550 add_9 (Add) (None, 14, 14, 480) 0 activation_21[0][0] 551 batch_normalization_35[0][0] 552 __________________________________________________________________________________________________ 553 activation_23 (Activation) (None, 14, 14, 480) 0 add_9[0][0] 554 __________________________________________________________________________________________________ 555 tf_op_layer_split_24 (TensorFlo [(None, 14, 14, 160) 0 activation_23[0][0] 556 __________________________________________________________________________________________________ 557 conv2d_73 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_24[0][0] 558 __________________________________________________________________________________________________ 559 conv2d_74 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_24[0][1] 560 __________________________________________________________________________________________________ 561 conv2d_75 (Conv2D) (None, 14, 14, 40) 6440 tf_op_layer_split_24[0][2] 562 __________________________________________________________________________________________________ 563 concatenate_26 (Concatenate) (None, 14, 14, 120) 0 conv2d_73[0][0] 564 conv2d_74[0][0] 565 conv2d_75[0][0] 566 __________________________________________________________________________________________________ 567 batch_normalization_36 (BatchNo (None, 14, 14, 120) 480 concatenate_26[0][0] 568 __________________________________________________________________________________________________ 569 activation_24 (Activation) (None, 14, 14, 120) 0 batch_normalization_36[0][0] 570 __________________________________________________________________________________________________ 571 tf_op_layer_Reshape_24 (TensorF [(None, 14, 14, 3, 4 0 activation_24[0][0] 572 __________________________________________________________________________________________________ 573 tf_op_layer_Transpose_12 (Tenso [(None, 14, 14, 40, 0 tf_op_layer_Reshape_24[0][0] 574 __________________________________________________________________________________________________ 575 tf_op_layer_Reshape_25 (TensorF [(None, 14, 14, 120) 0 tf_op_layer_Transpose_12[0][0] 576 __________________________________________________________________________________________________ 577 depthwise_conv2d_12 (DepthwiseC (None, 7, 7, 120) 1080 tf_op_layer_Reshape_25[0][0] 578 __________________________________________________________________________________________________ 579 batch_normalization_37 (BatchNo (None, 7, 7, 120) 480 depthwise_conv2d_12[0][0] 580 __________________________________________________________________________________________________ 581 tf_op_layer_split_25 (TensorFlo [(None, 7, 7, 40), ( 0 batch_normalization_37[0][0] 582 __________________________________________________________________________________________________ 583 conv2d_76 (Conv2D) (None, 7, 7, 160) 6560 tf_op_layer_split_25[0][0] 584 __________________________________________________________________________________________________ 585 conv2d_77 (Conv2D) (None, 7, 7, 160) 6560 tf_op_layer_split_25[0][1] 586 __________________________________________________________________________________________________ 587 conv2d_78 (Conv2D) (None, 7, 7, 160) 6560 tf_op_layer_split_25[0][2] 588 __________________________________________________________________________________________________ 589 concatenate_27 (Concatenate) (None, 7, 7, 480) 0 conv2d_76[0][0] 590 conv2d_77[0][0] 591 conv2d_78[0][0] 592 __________________________________________________________________________________________________ 593 average_pooling2d_2 (AveragePoo (None, 7, 7, 480) 0 activation_23[0][0] 594 __________________________________________________________________________________________________ 595 batch_normalization_38 (BatchNo (None, 7, 7, 480) 1920 concatenate_27[0][0] 596 __________________________________________________________________________________________________ 597 concatenate_28 (Concatenate) (None, 7, 7, 960) 0 average_pooling2d_2[0][0] 598 batch_normalization_38[0][0] 599 __________________________________________________________________________________________________ 600 activation_25 (Activation) (None, 7, 7, 960) 0 concatenate_28[0][0] 601 __________________________________________________________________________________________________ 602 tf_op_layer_split_26 (TensorFlo [(None, 7, 7, 320), 0 activation_25[0][0] 603 __________________________________________________________________________________________________ 604 conv2d_79 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_26[0][0] 605 __________________________________________________________________________________________________ 606 conv2d_80 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_26[0][1] 607 __________________________________________________________________________________________________ 608 conv2d_81 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_26[0][2] 609 __________________________________________________________________________________________________ 610 concatenate_29 (Concatenate) (None, 7, 7, 240) 0 conv2d_79[0][0] 611 conv2d_80[0][0] 612 conv2d_81[0][0] 613 __________________________________________________________________________________________________ 614 batch_normalization_39 (BatchNo (None, 7, 7, 240) 960 concatenate_29[0][0] 615 __________________________________________________________________________________________________ 616 activation_26 (Activation) (None, 7, 7, 240) 0 batch_normalization_39[0][0] 617 __________________________________________________________________________________________________ 618 tf_op_layer_Reshape_26 (TensorF [(None, 7, 7, 3, 80) 0 activation_26[0][0] 619 __________________________________________________________________________________________________ 620 tf_op_layer_Transpose_13 (Tenso [(None, 7, 7, 80, 3) 0 tf_op_layer_Reshape_26[0][0] 621 __________________________________________________________________________________________________ 622 tf_op_layer_Reshape_27 (TensorF [(None, 7, 7, 240)] 0 tf_op_layer_Transpose_13[0][0] 623 __________________________________________________________________________________________________ 624 depthwise_conv2d_13 (DepthwiseC (None, 7, 7, 240) 2160 tf_op_layer_Reshape_27[0][0] 625 __________________________________________________________________________________________________ 626 batch_normalization_40 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_13[0][0] 627 __________________________________________________________________________________________________ 628 tf_op_layer_split_27 (TensorFlo [(None, 7, 7, 80), ( 0 batch_normalization_40[0][0] 629 __________________________________________________________________________________________________ 630 conv2d_82 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_27[0][0] 631 __________________________________________________________________________________________________ 632 conv2d_83 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_27[0][1] 633 __________________________________________________________________________________________________ 634 conv2d_84 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_27[0][2] 635 __________________________________________________________________________________________________ 636 concatenate_30 (Concatenate) (None, 7, 7, 960) 0 conv2d_82[0][0] 637 conv2d_83[0][0] 638 conv2d_84[0][0] 639 __________________________________________________________________________________________________ 640 batch_normalization_41 (BatchNo (None, 7, 7, 960) 3840 concatenate_30[0][0] 641 __________________________________________________________________________________________________ 642 add_10 (Add) (None, 7, 7, 960) 0 activation_25[0][0] 643 batch_normalization_41[0][0] 644 __________________________________________________________________________________________________ 645 activation_27 (Activation) (None, 7, 7, 960) 0 add_10[0][0] 646 __________________________________________________________________________________________________ 647 tf_op_layer_split_28 (TensorFlo [(None, 7, 7, 320), 0 activation_27[0][0] 648 __________________________________________________________________________________________________ 649 conv2d_85 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_28[0][0] 650 __________________________________________________________________________________________________ 651 conv2d_86 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_28[0][1] 652 __________________________________________________________________________________________________ 653 conv2d_87 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_28[0][2] 654 __________________________________________________________________________________________________ 655 concatenate_31 (Concatenate) (None, 7, 7, 240) 0 conv2d_85[0][0] 656 conv2d_86[0][0] 657 conv2d_87[0][0] 658 __________________________________________________________________________________________________ 659 batch_normalization_42 (BatchNo (None, 7, 7, 240) 960 concatenate_31[0][0] 660 __________________________________________________________________________________________________ 661 activation_28 (Activation) (None, 7, 7, 240) 0 batch_normalization_42[0][0] 662 __________________________________________________________________________________________________ 663 tf_op_layer_Reshape_28 (TensorF [(None, 7, 7, 3, 80) 0 activation_28[0][0] 664 __________________________________________________________________________________________________ 665 tf_op_layer_Transpose_14 (Tenso [(None, 7, 7, 80, 3) 0 tf_op_layer_Reshape_28[0][0] 666 __________________________________________________________________________________________________ 667 tf_op_layer_Reshape_29 (TensorF [(None, 7, 7, 240)] 0 tf_op_layer_Transpose_14[0][0] 668 __________________________________________________________________________________________________ 669 depthwise_conv2d_14 (DepthwiseC (None, 7, 7, 240) 2160 tf_op_layer_Reshape_29[0][0] 670 __________________________________________________________________________________________________ 671 batch_normalization_43 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_14[0][0] 672 __________________________________________________________________________________________________ 673 tf_op_layer_split_29 (TensorFlo [(None, 7, 7, 80), ( 0 batch_normalization_43[0][0] 674 __________________________________________________________________________________________________ 675 conv2d_88 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_29[0][0] 676 __________________________________________________________________________________________________ 677 conv2d_89 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_29[0][1] 678 __________________________________________________________________________________________________ 679 conv2d_90 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_29[0][2] 680 __________________________________________________________________________________________________ 681 concatenate_32 (Concatenate) (None, 7, 7, 960) 0 conv2d_88[0][0] 682 conv2d_89[0][0] 683 conv2d_90[0][0] 684 __________________________________________________________________________________________________ 685 batch_normalization_44 (BatchNo (None, 7, 7, 960) 3840 concatenate_32[0][0] 686 __________________________________________________________________________________________________ 687 add_11 (Add) (None, 7, 7, 960) 0 activation_27[0][0] 688 batch_normalization_44[0][0] 689 __________________________________________________________________________________________________ 690 activation_29 (Activation) (None, 7, 7, 960) 0 add_11[0][0] 691 __________________________________________________________________________________________________ 692 tf_op_layer_split_30 (TensorFlo [(None, 7, 7, 320), 0 activation_29[0][0] 693 __________________________________________________________________________________________________ 694 conv2d_91 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_30[0][0] 695 __________________________________________________________________________________________________ 696 conv2d_92 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_30[0][1] 697 __________________________________________________________________________________________________ 698 conv2d_93 (Conv2D) (None, 7, 7, 80) 25680 tf_op_layer_split_30[0][2] 699 __________________________________________________________________________________________________ 700 concatenate_33 (Concatenate) (None, 7, 7, 240) 0 conv2d_91[0][0] 701 conv2d_92[0][0] 702 conv2d_93[0][0] 703 __________________________________________________________________________________________________ 704 batch_normalization_45 (BatchNo (None, 7, 7, 240) 960 concatenate_33[0][0] 705 __________________________________________________________________________________________________ 706 activation_30 (Activation) (None, 7, 7, 240) 0 batch_normalization_45[0][0] 707 __________________________________________________________________________________________________ 708 tf_op_layer_Reshape_30 (TensorF [(None, 7, 7, 3, 80) 0 activation_30[0][0] 709 __________________________________________________________________________________________________ 710 tf_op_layer_Transpose_15 (Tenso [(None, 7, 7, 80, 3) 0 tf_op_layer_Reshape_30[0][0] 711 __________________________________________________________________________________________________ 712 tf_op_layer_Reshape_31 (TensorF [(None, 7, 7, 240)] 0 tf_op_layer_Transpose_15[0][0] 713 __________________________________________________________________________________________________ 714 depthwise_conv2d_15 (DepthwiseC (None, 7, 7, 240) 2160 tf_op_layer_Reshape_31[0][0] 715 __________________________________________________________________________________________________ 716 batch_normalization_46 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_15[0][0] 717 __________________________________________________________________________________________________ 718 tf_op_layer_split_31 (TensorFlo [(None, 7, 7, 80), ( 0 batch_normalization_46[0][0] 719 __________________________________________________________________________________________________ 720 conv2d_94 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_31[0][0] 721 __________________________________________________________________________________________________ 722 conv2d_95 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_31[0][1] 723 __________________________________________________________________________________________________ 724 conv2d_96 (Conv2D) (None, 7, 7, 320) 25920 tf_op_layer_split_31[0][2] 725 __________________________________________________________________________________________________ 726 concatenate_34 (Concatenate) (None, 7, 7, 960) 0 conv2d_94[0][0] 727 conv2d_95[0][0] 728 conv2d_96[0][0] 729 __________________________________________________________________________________________________ 730 batch_normalization_47 (BatchNo (None, 7, 7, 960) 3840 concatenate_34[0][0] 731 __________________________________________________________________________________________________ 732 add_12 (Add) (None, 7, 7, 960) 0 activation_29[0][0] 733 batch_normalization_47[0][0] 734 __________________________________________________________________________________________________ 735 activation_31 (Activation) (None, 7, 7, 960) 0 add_12[0][0] 736 __________________________________________________________________________________________________ 737 global_average_pooling2d (Globa (None, 960) 0 activation_31[0][0] 738 __________________________________________________________________________________________________ 739 dense (Dense) (None, 17) 16337 global_average_pooling2d[0][0] 740 ================================================================================================== 741 Total params: 902,741 742 Trainable params: 879,053 743 Non-trainable params: 23,688View Code
資料增強
1 # 訓練集資料進行資料增強 2 train_datagen = ImageDataGenerator( 3 rotation_range=20, # 隨機旋轉度數 4 width_shift_range=0.1, # 隨機水平平移 5 height_shift_range=0.1, # 隨機豎直平移 6 rescale=1 / 255, # 資料歸一化 7 shear_range=10, # 隨機錯切變換 8 zoom_range=0.1, # 隨機放大 9 horizontal_flip=True, # 水平翻轉 10 brightness_range=(0.7, 1.3), # 亮度變化 11 fill_mode='nearest', # 填充方式 12 ) 13 # 測驗集資料只需要歸一化就可以 14 test_datagen = ImageDataGenerator( 15 rescale=1 / 255, # 資料歸一化 16 )
資料生成器
1 # 訓練集資料生成器，可以在訓練時自動產生資料進行訓練 2 # 從'data/train'獲得訓練集資料 3 # 獲得資料后會把圖片resize為image_size×image_size的大小 4 # generator每次會產生batch_size個資料 5 train_generator = train_datagen.flow_from_directory( 6 '../data/train', 7 target_size=(image_size, image_size), 8 batch_size=batch_size, 9 ) 10 11 # 測驗集資料生成器 12 test_generator = test_datagen.flow_from_directory( 13 '../data/test', 14 target_size=(image_size, image_size), 15 batch_size=batch_size, 16 ) 17 # 字典的鍵為17個檔案夾的名字，值為對應的分類編號 18 print(train_generator.class_indices)
回呼設定
1 # 學習率調節函式，逐漸減小學習率 2 def adjust_learning_rate(epoch): 3 # 前40周期 4 if epoch<=40: 5 lr = 1e-4 6 # 前40到80周期 7 elif epoch>40 and epoch<=80: 8 lr = 1e-5 9 # 80到100周期 10 else: 11 lr = 1e-6 12 return lr 13 14 # 定義優化器 15 adam = Adam(lr=1e-4) 16 17 # 讀取模型 18 checkpoint_save_path = "./checkpoint/ShuffleNetV1.ckpt" 19 if os.path.exists(checkpoint_save_path + '.index'): 20 print('-------------load the model-----------------') 21 model.load_weights(checkpoint_save_path) 22 # 保存模型 23 cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_save_path, 24 save_weights_only=True, 25 save_best_only=True) 26 27 # 定義學習率衰減策略 28 callbacks = [] 29 callbacks.append(LearningRateScheduler(adjust_learning_rate)) 30 callbacks.append(cp_callback)
1 # 定義優化器，loss function，訓練程序中計算準確率 2 model.compile(optimizer=adam,loss='categorical_crossentropy',metrics=['accuracy']) 3 4 # Tensorflow2.1版本(包括2.1)之后可以直接使用fit訓練模型 5 history = model.fit(x=train_generator,epochs=epochs,validation_data=https://www.cnblogs.com/wsfbb/archive/2022/10/12/test_generator,callbacks=callbacks)
1 Epoch 1/100 2 34/34 [==============================] - 18s 526ms/step - loss: 2.9157 - accuracy: 0.1544 - val_loss: 2.8353 - val_accuracy: 0.0588 3 Epoch 2/100 4 34/34 [==============================] - 16s 478ms/step - loss: 2.2678 - accuracy: 0.2822 - val_loss: 2.8387 - val_accuracy: 0.0588 5 Epoch 3/100 6 34/34 [==============================] - 16s 483ms/step - loss: 1.9987 - accuracy: 0.3778 - val_loss: 2.8482 - val_accuracy: 0.0588 7 Epoch 4/100 8 34/34 [==============================] - 16s 481ms/step - loss: 1.8269 - accuracy: 0.4200 - val_loss: 2.8607 - val_accuracy: 0.0625 9 Epoch 5/100 10 34/34 [==============================] - 17s 487ms/step - loss: 1.7022 - accuracy: 0.4651 - val_loss: 2.8800 - val_accuracy: 0.0588 11 Epoch 6/100 12 34/34 [==============================] - 16s 479ms/step - loss: 1.5436 - accuracy: 0.5138 - val_loss: 2.9101 - val_accuracy: 0.0588 13 Epoch 7/100 14 34/34 [==============================] - 16s 484ms/step - loss: 1.4806 - accuracy: 0.5221 - val_loss: 2.9455 - val_accuracy: 0.0588 15 Epoch 8/100 16 34/34 [==============================] - 17s 486ms/step - loss: 1.3658 - accuracy: 0.5699 - val_loss: 3.0113 - val_accuracy: 0.0588 17 Epoch 9/100 18 34/34 [==============================] - 16s 478ms/step - loss: 1.3244 - accuracy: 0.5772 - val_loss: 3.0792 - val_accuracy: 0.0588 19 Epoch 10/100 20 34/34 [==============================] - 16s 480ms/step - loss: 1.2401 - accuracy: 0.6029 - val_loss: 3.2077 - val_accuracy: 0.0588 21 Epoch 11/100 22 34/34 [==============================] - 16s 480ms/step - loss: 1.1940 - accuracy: 0.6048 - val_loss: 3.3061 - val_accuracy: 0.0588 23 Epoch 12/100 24 34/34 [==============================] - 16s 480ms/step - loss: 1.1689 - accuracy: 0.6213 - val_loss: 3.4508 - val_accuracy: 0.0772 25 Epoch 13/100 26 34/34 [==============================] - 16s 481ms/step - loss: 1.1282 - accuracy: 0.6241 - val_loss: 3.5842 - val_accuracy: 0.0919 27 Epoch 14/100 28 34/34 [==============================] - 16s 479ms/step - loss: 1.0759 - accuracy: 0.6572 - val_loss: 3.5697 - val_accuracy: 0.1140 29 Epoch 15/100 30 34/34 [==============================] - 17s 488ms/step - loss: 1.0264 - accuracy: 0.6618 - val_loss: 3.6231 - val_accuracy: 0.1176 31 Epoch 16/100 32 34/34 [==============================] - 16s 475ms/step - loss: 1.0210 - accuracy: 0.6719 - val_loss: 3.3065 - val_accuracy: 0.1471 33 Epoch 17/100 34 34/34 [==============================] - 16s 477ms/step - loss: 0.9543 - accuracy: 0.6939 - val_loss: 2.9929 - val_accuracy: 0.2169 35 Epoch 18/100 36 34/34 [==============================] - 17s 493ms/step - loss: 0.9405 - accuracy: 0.6939 - val_loss: 2.3133 - val_accuracy: 0.2941 37 Epoch 19/100 38 34/34 [==============================] - 17s 488ms/step - loss: 0.9069 - accuracy: 0.7031 - val_loss: 1.8180 - val_accuracy: 0.4081 39 Epoch 20/100 40 34/34 [==============================] - 17s 491ms/step - loss: 0.8797 - accuracy: 0.7050 - val_loss: 1.7820 - val_accuracy: 0.4926 41 Epoch 21/100 42 34/34 [==============================] - 17s 493ms/step - loss: 0.8911 - accuracy: 0.7215 - val_loss: 1.4993 - val_accuracy: 0.5625 43 Epoch 22/100 44 34/34 [==============================] - 17s 496ms/step - loss: 0.8480 - accuracy: 0.7215 - val_loss: 1.4438 - val_accuracy: 0.5699 45 Epoch 23/100 46 34/34 [==============================] - 17s 496ms/step - loss: 0.7861 - accuracy: 0.7472 - val_loss: 1.2822 - val_accuracy: 0.6103 47 Epoch 24/100 48 34/34 [==============================] - 16s 477ms/step - loss: 0.8111 - accuracy: 0.7307 - val_loss: 1.7404 - val_accuracy: 0.5404 49 Epoch 25/100 50 34/34 [==============================] - 16s 477ms/step - loss: 0.7855 - accuracy: 0.7316 - val_loss: 1.4184 - val_accuracy: 0.6287 51 Epoch 26/100 52 34/34 [==============================] - 17s 491ms/step - loss: 0.7679 - accuracy: 0.7555 - val_loss: 1.2750 - val_accuracy: 0.6213 53 Epoch 27/100 54 34/34 [==============================] - 16s 481ms/step - loss: 0.7451 - accuracy: 0.7454 - val_loss: 1.4109 - val_accuracy: 0.5882 55 Epoch 28/100 56 34/34 [==============================] - 16s 482ms/step - loss: 0.7201 - accuracy: 0.7408 - val_loss: 1.3238 - val_accuracy: 0.6507 57 Epoch 29/100 58 34/34 [==============================] - 16s 475ms/step - loss: 0.7238 - accuracy: 0.7629 - val_loss: 1.2997 - val_accuracy: 0.6360 59 Epoch 30/100 60 34/34 [==============================] - 16s 482ms/step - loss: 0.6587 - accuracy: 0.7849 - val_loss: 1.2941 - val_accuracy: 0.6471 61 Epoch 31/100 62 34/34 [==============================] - 17s 488ms/step - loss: 0.6508 - accuracy: 0.7858 - val_loss: 1.4063 - val_accuracy: 0.6397 63 Epoch 32/100 64 34/34 [==============================] - 17s 495ms/step - loss: 0.6294 - accuracy: 0.8006 - val_loss: 1.1997 - val_accuracy: 0.6691 65 Epoch 33/100 66 34/34 [==============================] - 16s 482ms/step - loss: 0.6579 - accuracy: 0.7665 - val_loss: 1.2146 - val_accuracy: 0.6434 67 Epoch 34/100 68 34/34 [==============================] - 16s 481ms/step - loss: 0.6207 - accuracy: 0.7941 - val_loss: 1.3010 - val_accuracy: 0.6360 69 Epoch 35/100 70 34/34 [==============================] - 16s 480ms/step - loss: 0.5873 - accuracy: 0.8033 - val_loss: 1.2448 - val_accuracy: 0.6618 71 Epoch 36/100 72 34/34 [==============================] - 16s 477ms/step - loss: 0.5755 - accuracy: 0.7941 - val_loss: 1.2584 - val_accuracy: 0.6434 73 Epoch 37/100 74 34/34 [==============================] - 17s 491ms/step - loss: 0.5668 - accuracy: 0.8189 - val_loss: 1.1914 - val_accuracy: 0.6728 75 Epoch 38/100 76 34/34 [==============================] - 16s 485ms/step - loss: 0.5527 - accuracy: 0.8107 - val_loss: 1.2917 - val_accuracy: 0.6434 77 Epoch 39/100 78 34/34 [==============================] - 16s 475ms/step - loss: 0.5715 - accuracy: 0.8088 - val_loss: 1.2613 - val_accuracy: 0.6581 79 Epoch 40/100 80 34/34 [==============================] - 16s 477ms/step - loss: 0.5286 - accuracy: 0.8272 - val_loss: 1.2632 - val_accuracy: 0.6581 81 Epoch 41/100 82 34/34 [==============================] - 17s 495ms/step - loss: 0.5098 - accuracy: 0.8327 - val_loss: 1.1116 - val_accuracy: 0.6985 83 Epoch 42/100 84 34/34 [==============================] - 17s 493ms/step - loss: 0.4543 - accuracy: 0.8428 - val_loss: 1.0116 - val_accuracy: 0.7206 85 Epoch 43/100 86 34/34 [==============================] - 17s 492ms/step - loss: 0.4145 - accuracy: 0.8695 - val_loss: 0.9978 - val_accuracy: 0.7206 87 Epoch 44/100 88 34/34 [==============================] - 17s 489ms/step - loss: 0.4421 - accuracy: 0.8612 - val_loss: 0.9956 - val_accuracy: 0.7169 89 Epoch 45/100 90 34/34 [==============================] - 17s 486ms/step - loss: 0.4015 - accuracy: 0.8750 - val_loss: 0.9964 - val_accuracy: 0.7206 91 Epoch 46/100 92 34/34 [==============================] - 16s 476ms/step - loss: 0.3937 - accuracy: 0.8805 - val_loss: 1.0131 - val_accuracy: 0.7279 93 Epoch 47/100 94 34/34 [==============================] - 16s 477ms/step - loss: 0.4492 - accuracy: 0.8539 - val_loss: 1.0172 - val_accuracy: 0.7243 95 Epoch 48/100 96 34/34 [==============================] - 16s 483ms/step - loss: 0.4238 - accuracy: 0.8649 - val_loss: 1.0143 - val_accuracy: 0.7279 97 Epoch 49/100 98 34/34 [==============================] - 16s 480ms/step - loss: 0.4192 - accuracy: 0.8658 - val_loss: 1.0148 - val_accuracy: 0.7206 99 Epoch 50/100 100 34/34 [==============================] - 16s 479ms/step - loss: 0.4071 - accuracy: 0.8695 - val_loss: 1.0148 - val_accuracy: 0.7316 101 Epoch 51/100 102 34/34 [==============================] - 16s 483ms/step - loss: 0.4115 - accuracy: 0.8557 - val_loss: 1.0180 - val_accuracy: 0.7463 103 Epoch 52/100 104 34/34 [==============================] - 16s 480ms/step - loss: 0.4106 - accuracy: 0.8704 - val_loss: 1.0214 - val_accuracy: 0.7353 105 Epoch 53/100 106 34/34 [==============================] - 16s 477ms/step - loss: 0.4026 - accuracy: 0.8713 - val_loss: 1.0171 - val_accuracy: 0.7390 107 Epoch 54/100 108 34/34 [==============================] - 16s 476ms/step - loss: 0.4073 - accuracy: 0.8750 - val_loss: 1.0067 - val_accuracy: 0.7426 109 Epoch 55/100 110 34/34 [==============================] - 16s 485ms/step - loss: 0.3608 - accuracy: 0.8842 - val_loss: 1.0039 - val_accuracy: 0.7500 111 Epoch 56/100 112 34/34 [==============================] - 16s 478ms/step - loss: 0.4101 - accuracy: 0.8658 - val_loss: 1.0056 - val_accuracy: 0.7426 113 Epoch 57/100 114 34/34 [==============================] - 16s 478ms/step - loss: 0.3797 - accuracy: 0.8787 - val_loss: 0.9983 - val_accuracy: 0.7426 115 Epoch 58/100 116 34/34 [==============================] - 16s 484ms/step - loss: 0.4077 - accuracy: 0.8695 - val_loss: 1.0055 - val_accuracy: 0.7390 117 Epoch 59/100 118 34/34 [==============================] - 16s 474ms/step - loss: 0.4265 - accuracy: 0.8649 - val_loss: 1.0111 - val_accuracy: 0.7500 119 Epoch 60/100 120 34/34 [==============================] - 17s 492ms/step - loss: 0.3475 - accuracy: 0.8961 - val_loss: 0.9925 - val_accuracy: 0.7500 121 Epoch 61/100 122 34/34 [==============================] - 17s 497ms/step - loss: 0.3883 - accuracy: 0.8796 - val_loss: 0.9693 - val_accuracy: 0.7390 123 Epoch 62/100 124 34/34 [==============================] - 17s 494ms/step - loss: 0.3615 - accuracy: 0.8934 - val_loss: 0.9608 - val_accuracy: 0.7463 125 Epoch 63/100 126 34/34 [==============================] - 16s 480ms/step - loss: 0.4017 - accuracy: 0.8695 - val_loss: 0.9706 - val_accuracy: 0.7610 127 Epoch 64/100 128 34/34 [==============================] - 17s 495ms/step - loss: 0.3727 - accuracy: 0.8851 - val_loss: 0.9587 - val_accuracy: 0.7574 129 Epoch 65/100 130 34/34 [==============================] - 16s 480ms/step - loss: 0.4053 - accuracy: 0.8750 - val_loss: 0.9707 - val_accuracy: 0.7390 131 Epoch 66/100 132 34/34 [==============================] - 16s 481ms/step - loss: 0.3722 - accuracy: 0.8860 - val_loss: 0.9836 - val_accuracy: 0.7426 133 Epoch 67/100 134 34/34 [==============================] - 16s 477ms/step - loss: 0.3841 - accuracy: 0.8824 - val_loss: 0.9597 - val_accuracy: 0.7426 135 Epoch 68/100 136 34/34 [==============================] - 16s 480ms/step - loss: 0.4085 - accuracy: 0.8621 - val_loss: 0.9813 - val_accuracy: 0.7390 137 Epoch 69/100 138 34/34 [==============================] - 16s 479ms/step - loss: 0.3578 - accuracy: 0.8943 - val_loss: 0.9673 - val_accuracy: 0.7463 139 Epoch 70/100 140 34/34 [==============================] - 16s 478ms/step - loss: 0.3628 - accuracy: 0.8915 - val_loss: 0.9789 - val_accuracy: 0.7353 141 Epoch 71/100 142 34/34 [==============================] - 17s 486ms/step - loss: 0.3904 - accuracy: 0.8686 - val_loss: 0.9738 - val_accuracy: 0.7537 143 Epoch 72/100 144 34/34 [==============================] - 16s 484ms/step - loss: 0.3585 - accuracy: 0.8943 - val_loss: 0.9827 - val_accuracy: 0.7574 145 Epoch 73/100 146 34/34 [==============================] - 16s 479ms/step - loss: 0.3593 - accuracy: 0.8879 - val_loss: 0.9722 - val_accuracy: 0.7574 147 Epoch 74/100 148 34/34 [==============================] - 16s 481ms/step - loss: 0.3636 - accuracy: 0.8869 - val_loss: 0.9719 - val_accuracy: 0.7500 149 Epoch 75/100 150 34/34 [==============================] - 16s 479ms/step - loss: 0.3776 - accuracy: 0.8787 - val_loss: 0.9690 - val_accuracy: 0.7500 151 Epoch 76/100 152 34/34 [==============================] - 17s 490ms/step - loss: 0.3815 - accuracy: 0.8768 - val_loss: 0.9569 - val_accuracy: 0.7574 153 Epoch 77/100 154 34/34 [==============================] - 16s 480ms/step - loss: 0.3608 - accuracy: 0.8833 - val_loss: 0.9737 - val_accuracy: 0.7537 155 Epoch 78/100 156 34/34 [==============================] - 17s 496ms/step - loss: 0.3801 - accuracy: 0.8722 - val_loss: 0.9393 - val_accuracy: 0.7610 157 Epoch 79/100 158 34/34 [==============================] - 16s 478ms/step - loss: 0.3385 - accuracy: 0.8943 - val_loss: 0.9577 - val_accuracy: 0.7500 159 Epoch 80/100 160 34/34 [==============================] - 16s 477ms/step - loss: 0.3565 - accuracy: 0.8869 - val_loss: 0.9693 - val_accuracy: 0.7537 161 Epoch 81/100 162 34/34 [==============================] - 16s 482ms/step - loss: 0.3614 - accuracy: 0.8961 - val_loss: 0.9911 - val_accuracy: 0.7574 163 Epoch 82/100 164 34/34 [==============================] - 16s 478ms/step - loss: 0.3581 - accuracy: 0.8814 - val_loss: 0.9848 - val_accuracy: 0.7574 165 Epoch 83/100 166 34/34 [==============================] - 16s 479ms/step - loss: 0.3880 - accuracy: 0.8833 - val_loss: 0.9779 - val_accuracy: 0.7574 167 Epoch 84/100 168 34/34 [==============================] - 16s 478ms/step - loss: 0.3549 - accuracy: 0.8851 - val_loss: 0.9744 - val_accuracy: 0.7537 169 Epoch 85/100 170 34/34 [==============================] - 17s 487ms/step - loss: 0.3552 - accuracy: 0.8934 - val_loss: 0.9736 - val_accuracy: 0.7537 171 Epoch 86/100 172 34/34 [==============================] - 16s 483ms/step - loss: 0.3681 - accuracy: 0.8842 - val_loss: 0.9715 - val_accuracy: 0.7537 173 Epoch 87/100 174 34/34 [==============================] - 16s 479ms/step - loss: 0.3416 - accuracy: 0.8934 - val_loss: 0.9662 - val_accuracy: 0.7537 175 Epoch 88/100 176 34/34 [==============================] - 17s 486ms/step - loss: 0.3562 - accuracy: 0.8934 - val_loss: 0.9662 - val_accuracy: 0.7537 177 Epoch 89/100 178 34/34 [==============================] - 17s 485ms/step - loss: 0.3682 - accuracy: 0.8851 - val_loss: 0.9636 - val_accuracy: 0.7500 179 Epoch 90/100 180 34/34 [==============================] - 16s 480ms/step - loss: 0.3739 - accuracy: 0.8787 - val_loss: 0.9614 - val_accuracy: 0.7500 181 Epoch 91/100 182 34/34 [==============================] - 16s 483ms/step - loss: 0.3442 - accuracy: 0.8943 - val_loss: 0.9631 - val_accuracy: 0.7500 183 Epoch 92/100 184 34/34 [==============================] - 16s 478ms/step - loss: 0.3598 - accuracy: 0.8915 - val_loss: 0.9624 - val_accuracy: 0.7500 185 Epoch 93/100 186 34/34 [==============================] - 16s 474ms/step - loss: 0.3845 - accuracy: 0.8778 - val_loss: 0.9641 - val_accuracy: 0.7500 187 Epoch 94/100 188 34/34 [==============================] - 16s 478ms/step - loss: 0.3526 - accuracy: 0.8915 - val_loss: 0.9655 - val_accuracy: 0.7574 189 Epoch 95/100 190 34/34 [==============================] - 16s 477ms/step - loss: 0.3581 - accuracy: 0.8869 - val_loss: 0.9652 - val_accuracy: 0.7574 191 Epoch 96/100 192 34/34 [==============================] - 16s 477ms/step - loss: 0.3392 - accuracy: 0.8897 - val_loss: 0.9608 - val_accuracy: 0.7537 193 Epoch 97/100 194 34/34 [==============================] - 16s 478ms/step - loss: 0.3525 - accuracy: 0.8943 - val_loss: 0.9628 - val_accuracy: 0.7537 195 Epoch 98/100 196 34/34 [==============================] - 16s 480ms/step - loss: 0.3532 - accuracy: 0.8943 - val_loss: 0.9632 - val_accuracy: 0.7537 197 Epoch 99/100 198 34/34 [==============================] - 16s 479ms/step - loss: 0.3759 - accuracy: 0.8851 - val_loss: 0.9571 - val_accuracy: 0.7574 199 Epoch 100/100 200 34/34 [==============================] - 16s 476ms/step - loss: 0.3292 - accuracy: 0.8971 - val_loss: 0.9545 - val_accuracy: 0.7537View Code
沒有預訓練的權重，訓練出來的結果肯定不是特別好，但是還算能接受吧，
可視化
1 # 畫出訓練集準確率曲線圖 2 plt.plot(np.arange(epochs),history.history['accuracy'],c='b',label='train_accuracy') 3 # 畫出驗證集準確率曲線圖 4 plt.plot(np.arange(epochs),history.history['val_accuracy'],c='y',label='val_accuracy') 5 # 圖例 6 plt.legend() 7 # x坐標描述 8 plt.xlabel('epochs') 9 # y坐標描述 10 plt.ylabel('accuracy') 11 # 顯示影像 12 plt.show()
1 # 畫出訓練集loss曲線圖 2 plt.plot(np.arange(epochs),history.history['loss'],c='b',label='train_loss') 3 # 畫出驗證集loss曲線圖 4 plt.plot(np.arange(epochs),history.history['val_loss'],c='y',label='val_loss') 5 # 圖例 6 plt.legend() 7 # x坐標描述 8 plt.xlabel('epochs') 9 # y坐標描述 10 plt.ylabel('loss') 11 # 顯示影像 12 plt.show()
7、模型結構大圖
這里給出1000分類的模型結構圖，圖太大，翻起來不好翻，需要pdf的留下郵箱我發給你，
宣告：本文參照CSDN博主「別團等shy哥發育」的文章
原文鏈接：https://blog.csdn.net/qq_43753724/article/details/126415101
轉載請註明出處，本文鏈接：https://www.uj5u.com/qita/514107.html
標籤：其他