在我见过的大多数 Tensorflow 代码中，Adam Optimizer 以恒定的学习率1e-4（即 0.0001）使用。代码通常如下所示：

...build the model...
# Add the optimizer
train_op = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
# Add the ops to initialize variables.  These will include 
# the optimizer slots added by AdamOptimizer().
init_op = tf.initialize_all_variables()

# launch the graph in a session
sess = tf.Session()
# Actually intialize the variables
sess.run(init_op)
# now train your model
for ...:
  sess.run(train_op)

我想知道，在使用亚当优化器时使用指数衰减是否有用，即使用以下代码：

...build the model...
# Add the optimizer
step = tf.Variable(0, trainable=False)
rate = tf.train.exponential_decay(0.15, step, 1, 0.9999)
optimizer = tf.train.AdamOptimizer(rate).minimize(cross_entropy, global_step=step)
# Add the ops to initialize variables.  These will include 
# the optimizer slots added by AdamOptimizer().
init_op = tf.initialize_all_variables()

# launch the graph in a session
sess = tf.Session()
# Actually intialize the variables
sess.run(init_op)
# now train your model
for ...:
  sess.run(train_op)

通常，人们使用某种学习率衰减，对于亚当来说这似乎并不常见。这有什么理论上的原因吗？将 Adam 优化器与衰减结合起来有用吗？