LSTM 模型中的参数数量

数据挖掘 深度学习 rnn
2021-10-13 20:18:47

单个堆叠 LSTM 有多少个参数?参数的数量对所需的训练示例数量施加了一个下限,并且还影响了训练时间。因此,知道参数的数量对于使用 LSTM 训练模型很有用。

4个回答

LSTM 有一组 2 个矩阵:(3) 个门中的每一个的 U 和 W。图中的 (.) 表示这些矩阵与输入的乘积x 和输出 h.

  • U 有尺寸 n×m
  • W 有尺寸 n×n
  • 三个门中的每一个都有一组不同的矩阵(比如 üFrGe用于遗忘门等)
  • 还有另一组这些矩阵用于更新单元状态 S
  • 在提到的矩阵之上,您需要计算偏差(不在图片中)

因此总#参数=4(n+n2+n)

LSTM 抽象块

按照先前的答案,LSTM 的参数数量,采用大小的输入向量 并给出大小的输出向量 n 是:

4(nm+n2)

However in case your LSTM includes bias vectors, (this is the default in keras for example), the number becomes:

4(nm+n2+n)

According to this:

LSTM cell structure

LSTM cell structure

LSTM equations

LSTM equations

Ingoring non-linearities

Ingoring non-linearities

If the input x_t is of size n×1, and there are d memory cells, then the size of each of W∗ and U∗ is d×n, and d×d resp. The size of W will then be 4d×(n+d). Note that each one of the dd memory cells has its own weights W∗ and U∗, and that the only time memory cell values are shared with other LSTM units is during the product with U∗.

Thanks to Arun Mallya for great presentation.

to completely receive you'r answer and to have a good insight visit : https://towardsdatascience.com/counting-no-of-parameters-in-deep-learning-models-by-hand-8f1716241889

g, no. of FFNNs in a unit (RNN has 1, GRU has 3, LSTM has 4)

h, size of hidden units

i, dimension/size of input

Since every FFNN(feed forward neural network) has h(h+i) + h parameters, we have

num_params = g × [h(h+i) + h]

Example 2.1: LSTM with 2 hidden units and input dimension 3.

enter image description here

g = 4 (LSTM has 4 FFNNs)

h = 2

i = 3

num_params

= g × [h(h+i) + h]

= 4 × [2(2+3) + 2]

= 48

    input = Input((None, 3))
    lstm = LSTM(2)(input)
    model = Model(input, lstm)

thanks to RAIMI KARIM