我是 ML 的新手，所以如果這是任何人都可以想出的愚蠢問題，我很抱歉。我在這里使用 TensorFlow 和 Keras。
所以這是我的代碼：
import tensorflow as tf
import numpy as np
from tensorflow import keras
model = keras.Sequential([
keras.layers.Dense(units=1, input_shape=[1])
])
model.compile(optimizer="sgd", loss="mean_squared_error")
xs = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0], dtype=float)
ys = np.array([0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5, 9.0, 9.5, 10.0], dtype=float)
model.fit(xs, ys, epochs=500)
print(model.predict([25.0]))
我得到這個作為輸出 [我沒有顯示整個 500 行，只有 20 個時期：
Epoch 1/500
1/1 [==============================] - 0s 210ms/step - loss: 450.9794
Epoch 2/500
1/1 [==============================] - 0s 4ms/step - loss: 1603.0852
Epoch 3/500
1/1 [==============================] - 0s 10ms/step - loss: 5698.4731
Epoch 4/500
1/1 [==============================] - 0s 7ms/step - loss: 20256.3398
Epoch 5/500
1/1 [==============================] - 0s 10ms/step - loss: 72005.1719
Epoch 6/500
1/1 [==============================] - 0s 4ms/step - loss: 255956.5938
Epoch 7/500
1/1 [==============================] - 0s 3ms/step - loss: 909848.5000
Epoch 8/500
1/1 [==============================] - 0s 5ms/step - loss: 3234236.0000
Epoch 9/500
1/1 [==============================] - 0s 3ms/step - loss: 11496730.0000
Epoch 10/500
1/1 [==============================] - 0s 3ms/step - loss: 40867392.0000
Epoch 11/500
1/1 [==============================] - 0s 3ms/step - loss: 145271264.0000
Epoch 12/500
1/1 [==============================] - 0s 3ms/step - loss: 516395584.0000
Epoch 13/500
1/1 [==============================] - 0s 4ms/step - loss: 1835629312.0000
Epoch 14/500
1/1 [==============================] - 0s 3ms/step - loss: 6525110272.0000
Epoch 15/500
1/1 [==============================] - 0s 3ms/step - loss: 23194802176.0000
Epoch 16/500
1/1 [==============================] - 0s 3ms/step - loss: 82450513920.0000
Epoch 17/500
1/1 [==============================] - 0s 3ms/step - loss: 293086593024.0000
Epoch 18/500
1/1 [==============================] - 0s 5ms/step - loss: 1041834835968.0000
Epoch 19/500
1/1 [==============================] - 0s 3ms/step - loss: 3703408164864.0000
Epoch 20/500
1/1 [==============================] - 0s 3ms/step - loss: 13164500484096.0000
如您所見，它呈指數級增長。很快（在第 64 個時期），這些數字變為inf。然后，從無窮大開始，它做某事并變成NaN（非數字）。我認為隨著時間的推移，模??型會更好地找出模式，這是怎么回事？
有一件事我注意到，如果我減少的長度xs和ys從20到10，損失減少，成為7.9193e-05。在我將兩個 numpy 陣列的長度增加到18它之后，它開始不受控制地增加，否則就可以了。我給出了 20 個值，因為我認為如果我給出更多資料，模型會更好，這就是我給出 20 個值的原因。
uj5u.com熱心網友回復：
你的阿爾法/學習率似乎太大了。
嘗試使用較低的學習率，如下所示：
import tensorflow as tf
import numpy as np
from tensorflow import keras
model = keras.Sequential([
keras.layers.Dense(units=1, input_shape=[1])
])
# manually set the optimizer, default learning_rate=0.01
opt = keras.optimizers.SGD(learning_rate=0.0001)
model.compile(optimizer=opt, loss="mean_squared_error")
xs = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0], dtype=float)
ys = np.array([0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5, 9.0, 9.5, 10.0], dtype=float)
model.fit(xs, ys, epochs=500)
print(model.predict([25.0]))
...實際上有效。
ADAM 效果更好的原因，可能是因為它自適應地估計了學習率 - 我認為 ADAM 中的 A 代表 Adaptive ;))
Epoch 1/500
1/1 [==============================] - 0s 129ms/step - loss: 1.2133
Epoch 2/500
1/1 [==============================] - 0s 990us/step - loss: 1.1442
Epoch 3/500
1/1 [==============================] - 0s 0s/step - loss: 1.0792
Epoch 4/500
1/1 [==============================] - 0s 1ms/step - loss: 1.0178
Epoch 5/500
1/1 [==============================] - 0s 1ms/step - loss: 0.9599
Epoch 6/500
1/1 [==============================] - 0s 1ms/step - loss: 0.9053
Epoch 7/500
1/1 [==============================] - 0s 0s/step - loss: 0.8538
Epoch 8/500
1/1 [==============================] - 0s 1ms/step - loss: 0.8053
Epoch 9/500
1/1 [==============================] - 0s 999us/step - loss: 0.7595
Epoch 10/500
1/1 [==============================] - 0s 1ms/step - loss: 0.7163
...
Epoch 499/500
1/1 [==============================] - 0s 1ms/step - loss: 9.9431e-06
Epoch 500/500
1/1 [==============================] - 0s 999us/step - loss: 9.9420e-06
編輯：
來自https://arxiv.org/pdf/1412.6980.pdf
該方法根據梯度的一階和二階矩的估計計算不同引數的個體自適應學習率；Adam 這個名字來源于自適應矩估計
uj5u.com熱心網友回復：
優化器 SGD 似乎在您的資料集上表現不佳。如果你用“adam”替換優化器，你應該得到你期望的結果。
model.compile(optimizer="adam", loss="mean_squared_error")
預測應該是你所期望的
print(model.predict([25.0]))
# [[12.487587]]
我不是 100% 關于為什么 SGD 優化器作業如此糟糕。
編輯：
@MortenJensen（下文）很好地解釋了為什么 adam 優化器做得更好。總結：sgd 做得不好的原因是它需要較小的學習率。然而，Adam 具有自適應學習率。
轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/341010.html