我正在训练一个两层神经网络并在迭代期间输出成本函数，并注意到成本函数随着迭代次数的增加而急剧增加。最初我怀疑是因为梯度下降的学习率太高，所以我从0.05改为0.005。但是，这根本没有帮助。任何建议都非常感谢！

以下是我的部分代码：

# construct model
y_pred = multilayer_perceptron(x, weights, biases)

# define cost function(mean squred error) and optimizer(gradient descent)
cost = tf.reduce_mean(tf.square(y - y_pred))
optimizer = tf.train.MomentumOptimizer(learning_rate = learning_rate, 
            momentum = momentum).minimize(cost)

EPOCHS = 100
# initialize parameters
init_op = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init_op)
    for epoch_no in range(EPOCHS):
        _, c = sess.run([optimizer, cost], feed_dict={x:X_train , y:Y_train})
       print('Epoch number: {}, cost: {}'.format(epoch_no, c))

# For running test dataset
results, test_cost = sess.run([y_pred, cost], feed_dict={x:X_train, y:Y_train})
print('test cost: {:.3f}'.format(test_cost))
print(y_pred)

前 3 个 Epoch 的输出：

Epoch number: 0, cost: 509.89886474609375
Epoch number: 1, cost: 287486752.0
Epoch number: 2, cost: 2.262859251393233e+18