我正在研究一個文本分類問題，我面臨著訓練和驗證準確度增長極慢的問題，而且還有振蕩，請幫助我解決我們的問題。
以下是我的資料劃分和模型的資訊
x_train (47723)
x_test (3372) x_valid (1631) 批次大小 = 64 epochs = 50
模型
import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
maxlen=2500
max_features= 40000
embedding_dim=100
num_filters=128
filter_sizes = [1,3,5]
inp = keras.layers.Input(shape=(2500,), dtype="int32")
embedding_layer = keras.layers.Embedding(max_features,
embedding_dim,
weights=[embedding_matrix],
input_length=maxlen,
trainable=False)(inp)
print(embedding_layer.shape)
reshape = Reshape((maxlen,embedding_dim,1))(embedding_layer)
print(reshape.shape)
conv_0 = Conv2D(num_filters, kernel_size=(filter_sizes[0], embedding_dim), padding='valid', kernel_initializer='normal', activation='relu')(reshape)
conv_1 = Conv2D(num_filters, kernel_size=(filter_sizes[1], embedding_dim), padding='valid', kernel_initializer='normal', activation='relu')(reshape)
conv_2 = Conv2D(num_filters, kernel_size=(filter_sizes[2], embedding_dim), padding='valid', kernel_initializer='normal', activation='relu')(reshape)
maxpool_0 = MaxPool2D(pool_size=(maxlen - filter_sizes[0] 1, 1), strides=(1,1), padding='valid')(conv_0)
maxpool_1 = MaxPool2D(pool_size=(maxlen - filter_sizes[1] 1, 1), strides=(1,1), padding='valid')(conv_1)
maxpool_2 = MaxPool2D(pool_size=(maxlen - filter_sizes[2] 1, 1), strides=(1,1), padding='valid')(conv_2)
concatenated_tensor = Concatenate(axis=1)([maxpool_0, maxpool_1, maxpool_2])
newdim = tuple([x for x in concatenated_tensor.shape.as_list() if x != 1 and x is not None])
reshape_layer = Reshape(newdim) (concatenated_tensor)
attention = Attention()(reshape_layer)
dropout = Dropout(0.2)(attention)
output = Dense(units=8922, activation='sigmoid')(dropout)
# this creates a model that includes
model = keras.models.Model(inputs=inp, outputs=output)
model.summary()
模型訓練
Train...
Epoch 1/50
746/746 [==============================] - 302s 392ms/step - loss: 0.0115 - f1_m: 0.0470 - val_loss: 0.0093 - val_f1_m: 0.0754
Epoch 2/50
746/746 [==============================] - 292s 391ms/step - loss: 0.0080 - f1_m: 0.1368 - val_loss: 0.0086 - val_f1_m: 0.1276
Epoch 3/50
746/746 [==============================] - 292s 391ms/step - loss: 0.0075 - f1_m: 0.1763 - val_loss: 0.0082 - val_f1_m: 0.1459
Epoch 4/50
746/746 [==============================] - 292s 391ms/step - loss: 0.0072 - f1_m: 0.2030 - val_loss: 0.0079 - val_f1_m: 0.1566
Epoch 5/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0070 - f1_m: 0.2232 - val_loss: 0.0077 - val_f1_m: 0.1654
Epoch 6/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0069 - f1_m: 0.2370 - val_loss: 0.0076 - val_f1_m: 0.1794
Epoch 7/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0067 - f1_m: 0.2485 - val_loss: 0.0074 - val_f1_m: 0.2144
Epoch 8/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0066 - f1_m: 0.2603 - val_loss: 0.0073 - val_f1_m: 0.2255
Epoch 9/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0065 - f1_m: 0.2689 - val_loss: 0.0073 - val_f1_m: 0.2131
Epoch 10/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0064 - f1_m: 0.2759 - val_loss: 0.0072 - val_f1_m: 0.2311
Epoch 11/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0064 - f1_m: 0.2841 - val_loss: 0.0072 - val_f1_m: 0.2394
Epoch 12/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0063 - f1_m: 0.2892 - val_loss: 0.0072 - val_f1_m: 0.2298
Epoch 13/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0062 - f1_m: 0.2938 - val_loss: 0.0071 - val_f1_m: 0.2480
Epoch 14/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0062 - f1_m: 0.2991 - val_loss: 0.0071 - val_f1_m: 0.2436
Epoch 15/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0061 - f1_m: 0.3018 - val_loss: 0.0070 - val_f1_m: 0.2595
Epoch 16/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0061 - f1_m: 0.3062 - val_loss: 0.0070 - val_f1_m: 0.2679
Epoch 17/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0060 - f1_m: 0.3095 - val_loss: 0.0070 - val_f1_m: 0.2566
Epoch 18/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0060 - f1_m: 0.3138 - val_loss: 0.0070 - val_f1_m: 0.2655
Epoch 19/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0060 - f1_m: 0.3166 - val_loss: 0.0070 - val_f1_m: 0.2498
Epoch 20/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0059 - f1_m: 0.3192 - val_loss: 0.0070 - val_f1_m: 0.2789
Epoch 21/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0059 - f1_m: 0.3210 - val_loss: 0.0071 - val_f1_m: 0.2638
Epoch 22/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0059 - f1_m: 0.3236 - val_loss: 0.0071 - val_f1_m: 0.2537
Epoch 23/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0058 - f1_m: 0.3255 - val_loss: 0.0069 - val_f1_m: 0.2832
Epoch 24/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0058 - f1_m: 0.3268 - val_loss: 0.0070 - val_f1_m: 0.2770
Epoch 25/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0058 - f1_m: 0.3291 - val_loss: 0.0070 - val_f1_m: 0.2723
Epoch 26/50
746/746 [==============================] - 291s 391ms/step - loss: 0.0058 - f1_m: 0.3300 - val_loss: 0.0069 - val_f1_m: 0.2864
Epoch 27/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0058 - f1_m: 0.3314 - val_loss: 0.0069 - val_f1_m: 0.2779
Epoch 28/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0058 - f1_m: 0.3324 - val_loss: 0.0070 - val_f1_m: 0.2778
Epoch 29/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0057 - f1_m: 0.3344 - val_loss: 0.0070 - val_f1_m: 0.2824
Epoch 30/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0057 - f1_m: 0.3353 - val_loss: 0.0070 - val_f1_m: 0.2656
Epoch 31/50
746/746 [==============================] - 292s 391ms/step - loss: 0.0057 - f1_m: 0.3367 - val_loss: 0.0070 - val_f1_m: 0.2889
Epoch 32/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0057 - f1_m: 0.3376 - val_loss: 0.0070 - val_f1_m: 0.2822
Epoch 33/50
746/746 [==============================] - 292s 391ms/step - loss: 0.0057 - f1_m: 0.3386 - val_loss: 0.0070 - val_f1_m: 0.2904
Epoch 34/50
746/746 [==============================] - 292s 392ms/step - loss: 0.0057 - f1_m: 0.3397 - val_loss: 0.0070 - val_f1_m: 0.2849
Epoch 35/50
746/746 [==============================] - 292s 391ms/step - loss: 0.0057 - f1_m: 0.3407 - val_loss: 0.0070 - val_f1_m: 0.2767
Epoch 36/50
746/746 [==============================] - 292s 391ms/step - loss: 0.0057 - f1_m: 0.3415 - val_loss: 0.0069 - val_f1_m: 0.2957
Epoch 37/50
746/746 [==============================] - 292s 391ms/step - loss: 0.0057 - f1_m: 0.3426 - val_loss: 0.0070 - val_f1_m: 0.2946
Epoch 38/50
746/746 [==============================] - 291s 390ms/step - loss: 0.0056 - f1_m: 0.3441 - val_loss: 0.0070 - val_f1_m: 0.2793
Epoch 39/50
746/746 [==============================] - 292s 391ms/step - loss: 0.0056 - f1_m: 0.3443 - val_loss: 0.0070 - val_f1_m: 0.2928
Epoch 40/50
746/746 [==============================] - 292s 391ms/step - loss: 0.0056 - f1_m: 0.3449 - val_loss: 0.0070 - val_f1_m: 0.2852
Epoch 41/50
746/746 [==============================] - 292s 391ms/step - loss: 0.0056 - f1_m: 0.3447 - val_loss: 0.0070 - val_f1_m: 0.2883
Epoch 42/50
746/746 [==============================] - 292s 392ms/step - loss: 0.0056 - f1_m: 0.3468 - val_loss: 0.0069 - val_f1_m: 0.3026
Epoch 43/50
333/746 [============>.................] - ETA: 2:40 - loss: 0.0056 - f1_m: 0.3519
uj5u.com熱心網友回復：
您是否嘗試過提高學習率？這可能是它學習如此緩慢的原因。
uj5u.com熱心網友回復：
有什么理由在你的架構中使用卷積層嗎？我認為您的資料格式是文本，卷積層用于影像資料。
我認為您應該嘗試僅使用密集層，看看準確性是否有任何變化。
轉載請註明出處，本文鏈接：https://www.uj5u.com/qiye/381611.html
下一篇：推導模型可學習引數