Keras 中的自定义权重初始化

数据挖掘 喀拉斯
2021-10-12 07:33:35

根据我上一个问题的评论,我正在尝试为 RNN 构建我自己的自定义权重初始化程序。根据此处给出的代码(小心 - 根据fchollet ,Keras 的更新版本使用“initializers”而不是“initializations” ),我进行了一次尝试。

import numpy as np
import pandas, math, sys, keras
from keras import optimizers
from keras import initializers
from keras.models import Sequential
from keras.layers import Dense, SimpleRNN
from keras.regularizers import l2
import numpy as np

def rnn_model(hid_dim=10, ker_reg=0.01, rec_reg=0.01, optimizer="sgd", w_mean = 0., w_var = 0.05):
        my_init = lambda shape: initializers.TruncatedNormal(mean=w_mean, stddev=w_var)
        model = Sequential()
        model.add(SimpleRNN(units=hid_dim, activation='relu', kernel_initializer=my_init, recurrent_initializer=my_init, input_shape = (X.shape[1], X.shape[2]), kernel_regularizer=l2(ker_reg), recurrent_regularizer = l2(rec_reg), return_sequences = False))
        model.add(Dense(Y.shape[1], activation='softmax'))
        model.compile(loss='categorical_crossentropy', optimizer=optimizer, metrics=['accuracy'])
        print 'fitting a model'
        return model

当我rnn_model稍后打电话时,我收到一个错误:

    model = rnn_model(hid_dim=hid_val, ker_reg=ker_reg_best, rec_reg=rec_reg_best, optimizer=optim, w_mean=ave_weights, w_var=var_weights)
  File "rnn.py", line 187, in rnn_model
    model.add(SimpleRNN(units=hid_dim, activation='relu', kernel_initializer=my_init, recurrent_initializer=my_init, input_shape = (X.shape[1], X.shape[2]), kernel_regularizer=l2(ker_reg), recurrent_regularizer = l2(rec_reg), return_sequences = False))
  File "/user/pkgs/anaconda2/lib/python2.7/site-packages/keras/models.py", line 430, in add
    layer(x)
  File "/user/pkgs/anaconda2/lib/python2.7/site-packages/keras/layers/recurrent.py", line 257, in __call__
    return super(Recurrent, self).__call__(inputs, **kwargs)
  File "/user/pkgs/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 551, in __call__
    self.build(input_shapes[0])
  File "/user/pkgs/anaconda2/lib/python2.7/site-packages/keras/layers/recurrent.py", line 478, in build
    constraint=self.kernel_constraint)
  File "/user/pkgs/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 384, in add_weight
    weight = K.variable(initializer(shape), dtype=K.floatx(), name=name)
  File "/user/pkgs/anaconda2/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 288, in variable
    v = tf.Variable(value, dtype=_convert_string_dtype(dtype), name=name)
  File "/user/pkgs/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 197, in __init__
    expected_shape=expected_shape)
  File "/user/pkgs/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 274, in _init_from_args
    initial_value(), name="initial_value", dtype=dtype)
TypeError: __call__() takes at least 2 arguments (1 given)

有谁知道如何使用初始化器的自定义参数来初始化 Keras 模型?

1个回答

检查此示例(Quoc V. Le、Navdeep Jaitly、Geoffrey E. Hinton 的“A Simple Way to Initialize Recurrent Networks of Rectified Linear Units”中逐像素顺序 MNIST 的 IRNN 实验的再现)。

相关代码:

model.add(SimpleRNN(hidden_units,
                    kernel_initializer=initializers.RandomNormal(stddev=0.001),
                    recurrent_initializer=initializers.Identity(gain=1.0),
                    activation='relu',
                    input_shape=x_train.shape[1:]))