如何在 Keras 中使用加权和和加权乘积合并两个 CNN 深度学习模型?

数据挖掘 深度学习 喀拉斯 张量流 美国有线电视新闻网 卷积神经网络
2022-03-01 11:04:06

我正在使用 Keras 创建一个深度学习模型,我想通过使用加权和或加权积来合并两个 CNN。

如何使用加权和和加权积合并两个 CNN?

2个回答

我认为最优雅的方法是编写一个执行此操作的层。例如对于加权和的情况:

class WeightedSum(Layer):

    def __init__(self, a, **kwargs):
        self.a = a  # "weight" of the weighted sum
        super(WeightedSum, self).__init__(**kwargs)

    def call(self, model_outputs):
        return self.a * model_outputs[0] + (1 - self.a) * model_outputs[1]

    def compute_output_shape(self, input_shape):
        return input_shape[0]

假设您有两个模型model1,并且分别model2具有输出out1out2该层简单地执行操作:

out=aout1+(1a)out2

您可以以相同的方式计算加权乘积,只需更改call方法即可。


例子

from keras.layers import Layer, Input, Dense
from keras.models import Model
import keras.backend as K
import tensorflow as tf

# Define the custom layer
class WeightedSum(Layer):
    def __init__(self, a, **kwargs):
        self.a = a
        super(WeightedSum, self).__init__(**kwargs)
    def call(self, model_outputs):
        return self.a * model_outputs[0] + (1 - self.a) * model_outputs[1]
    def compute_output_shape(self, input_shape):
        return input_shape[0]

# Create model1
inp1 = Input((5,))
d1 = Dense(100)(inp1)
out1 = Dense(10)(d1)
model1 = Model(inp1, out1)

# Create model2
inp2 = Input((7,))
d2 = Dense(70)(inp2)
out2 = Dense(10)(d2)
model2 = Model(inp2, out2)

# Weighed sum of the two models' outputs with a = 0.1
out = WeightedSum(0.1)([model1.output, model2.output])

# Create the merged model
model = Model(inputs=[inp1, inp2], outputs=[out])

让我们检查一下摘要:

>>> model.summary()
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_5 (InputLayer)            (None, 5)            0
__________________________________________________________________________________________________
input_6 (InputLayer)            (None, 7)            0
__________________________________________________________________________________________________
dense_9 (Dense)                 (None, 100)          600         input_5[0][0]
__________________________________________________________________________________________________
dense_11 (Dense)                (None, 70)           560         input_6[0][0]
__________________________________________________________________________________________________
dense_10 (Dense)                (None, 10)           1010        dense_9[0][0]
__________________________________________________________________________________________________
dense_12 (Dense)                (None, 10)           710         dense_11[0][0]
__________________________________________________________________________________________________
weighted_sum_10 (WeightedSum)   (None, 10)           0           dense_10[0][0]
                                                                 dense_12[0][0]
==================================================================================================
Total params: 2,880
Trainable params: 2,880
Non-trainable params: 0
__________________________________________________________________________________________________

让我们看看它是否有效:

import numpy as np

a = np.random.random(size=(32, 5))  # input for model1 (batch size 32)
b = np.random.random(size=(32, 7))  # input for model2 (batch size 32)

pred = model.predict([a, b])

让我们看看它是否具有正确的形状:

>>> model.shape
(32, 10)

让我们看看它是否正确:

# Generate model outputs manually:
o1 = model1.predict(a)  # model1's output for array a
o2 = model2.predict(b)  # model2's output for array b

# Compute their weighted sum manually:
o = 0.1 * o1 + 0.9 * o2

现在,如果我们是正确的,o应该等于pred

>>> np.array_equal(o, pred)
True

跟进我的评论,因为我认为这对来到这里的任何人都有用。“a”可以是 tf.keras 中的可训练权重

class WeightedSum(layers.Layer):
    """A custom keras layer to learn a weighted sum of tensors"""

    def __init__(self, **kwargs):
        super(WeightedSum, self).__init__(**kwargs)

    def build(self, input_shape=1):
        self.a = self.add_weight(
            name='alpha',
            shape=(),
            initializer='ones',
            dtype='float32',
            trainable=True,
        )
        super(WeightedSum, self).build(input_shape)

    def call(self, model_outputs):
        return self.a * model_outputs[0] + (1 - self.a) * model_outputs[1]

    def compute_output_shape(self, input_shape):
        return input_shape[0]

将 alpha 约束为有界 [0,1] 也是可取的。

constraint=tf.keras.constraints.min_max_norm(max_value=1,min_value=0)