我正在使用 Keras 创建一个深度学习模型,我想通过使用加权和或加权积来合并两个 CNN。
如何使用加权和和加权积合并两个 CNN?
我正在使用 Keras 创建一个深度学习模型,我想通过使用加权和或加权积来合并两个 CNN。
如何使用加权和和加权积合并两个 CNN?
我认为最优雅的方法是编写一个执行此操作的层。例如对于加权和的情况:
class WeightedSum(Layer):
def __init__(self, a, **kwargs):
self.a = a # "weight" of the weighted sum
super(WeightedSum, self).__init__(**kwargs)
def call(self, model_outputs):
return self.a * model_outputs[0] + (1 - self.a) * model_outputs[1]
def compute_output_shape(self, input_shape):
return input_shape[0]
假设您有两个模型model1,并且分别model2具有输出out1和out2。该层简单地执行操作:
您可以以相同的方式计算加权乘积,只需更改call方法即可。
from keras.layers import Layer, Input, Dense
from keras.models import Model
import keras.backend as K
import tensorflow as tf
# Define the custom layer
class WeightedSum(Layer):
def __init__(self, a, **kwargs):
self.a = a
super(WeightedSum, self).__init__(**kwargs)
def call(self, model_outputs):
return self.a * model_outputs[0] + (1 - self.a) * model_outputs[1]
def compute_output_shape(self, input_shape):
return input_shape[0]
# Create model1
inp1 = Input((5,))
d1 = Dense(100)(inp1)
out1 = Dense(10)(d1)
model1 = Model(inp1, out1)
# Create model2
inp2 = Input((7,))
d2 = Dense(70)(inp2)
out2 = Dense(10)(d2)
model2 = Model(inp2, out2)
# Weighed sum of the two models' outputs with a = 0.1
out = WeightedSum(0.1)([model1.output, model2.output])
# Create the merged model
model = Model(inputs=[inp1, inp2], outputs=[out])
让我们检查一下摘要:
>>> model.summary()
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_5 (InputLayer) (None, 5) 0
__________________________________________________________________________________________________
input_6 (InputLayer) (None, 7) 0
__________________________________________________________________________________________________
dense_9 (Dense) (None, 100) 600 input_5[0][0]
__________________________________________________________________________________________________
dense_11 (Dense) (None, 70) 560 input_6[0][0]
__________________________________________________________________________________________________
dense_10 (Dense) (None, 10) 1010 dense_9[0][0]
__________________________________________________________________________________________________
dense_12 (Dense) (None, 10) 710 dense_11[0][0]
__________________________________________________________________________________________________
weighted_sum_10 (WeightedSum) (None, 10) 0 dense_10[0][0]
dense_12[0][0]
==================================================================================================
Total params: 2,880
Trainable params: 2,880
Non-trainable params: 0
__________________________________________________________________________________________________
让我们看看它是否有效:
import numpy as np
a = np.random.random(size=(32, 5)) # input for model1 (batch size 32)
b = np.random.random(size=(32, 7)) # input for model2 (batch size 32)
pred = model.predict([a, b])
让我们看看它是否具有正确的形状:
>>> model.shape
(32, 10)
让我们看看它是否正确:
# Generate model outputs manually:
o1 = model1.predict(a) # model1's output for array a
o2 = model2.predict(b) # model2's output for array b
# Compute their weighted sum manually:
o = 0.1 * o1 + 0.9 * o2
现在,如果我们是正确的,o应该等于pred:
>>> np.array_equal(o, pred)
True
跟进我的评论,因为我认为这对来到这里的任何人都有用。“a”可以是 tf.keras 中的可训练权重
class WeightedSum(layers.Layer):
"""A custom keras layer to learn a weighted sum of tensors"""
def __init__(self, **kwargs):
super(WeightedSum, self).__init__(**kwargs)
def build(self, input_shape=1):
self.a = self.add_weight(
name='alpha',
shape=(),
initializer='ones',
dtype='float32',
trainable=True,
)
super(WeightedSum, self).build(input_shape)
def call(self, model_outputs):
return self.a * model_outputs[0] + (1 - self.a) * model_outputs[1]
def compute_output_shape(self, input_shape):
return input_shape[0]
将 alpha 约束为有界 [0,1] 也是可取的。
constraint=tf.keras.constraints.min_max_norm(max_value=1,min_value=0)