如果隐藏层增加,神经网络就会中断

数据挖掘 Python
2022-02-21 07:16:54

我试图让我的神经网络尽可能动态。例如,我希望能够增加 hidden_​​layers 数量。到目前为止,它只适用于一个隐藏层,但如果我将其增加到两个,它会立即中断。当反向传播到达Inputs <- 1st_hidden之间的层时会出现问题这是我的代码。我试图通过将其提供给我的 NN 来解决 XOR 问题:

inputs  = np.array([[1, 0], [0, 1], [1, 1], [0, 0]]) 
targets = np.array([[1], [1], [0], [0]])

谁能帮我?我似乎无法解决它。

nn = NeuralNetwork(input_nodes = 2, hidden_layers = 1, hidden_nodes = 3, output_nodes = 1)

我初始化我的神经网络:

class NeuralNetwork:
def __init__(self, input_nodes, hidden_layers, hidden_nodes, output_nodes):

    self.input_nodes   = input_nodes
    self.hidden_layers = hidden_layers
    self.hidden_nodes  = hidden_nodes
    self.output_nodes  = output_nodes
    self.layers_count  = self.hidden_layers + 1

    self.weights       = []
    self.bias          = []

    min_value = -2
    max_value =  2

    # Initialize weights and bias matrices and place them in a list
    # Matrix shape within lists
    # [(2,2), (2,2), (1,2)]

    for i in range(self.layers_count):
        if i == 0:
            self.weights.append(np.random.uniform(min_value, max_value, (self.hidden_nodes, self.input_nodes)))
            self.bias.append(np.random.uniform(min_value, max_value, (self.hidden_nodes,)))
        elif i == self.hidden_layers:
            self.weights.append(np.random.uniform(min_value, max_value, (self.output_nodes, self.hidden_nodes)))
            self.bias.append(np.random.uniform(min_value, max_value, (self.output_nodes,)))
        else:
            self.weights.append(np.random.uniform(min_value, max_value, (self.hidden_nodes, self.hidden_nodes)))
            self.bias.append(np.random.uniform(min_value, max_value, (self.hidden_nodes,)))

    self.learning_rate = 0.1

def train(self, input_array, targets):

    inputs = np.transpose(input_array)

    applied_activation = nn.feedforward(inputs)

    output_errors = np.subtract(targets, applied_activation[-1])

    # Backpropogation
    for i in reversed(range(self.layers_count)):
        delta    = np.ndarray((self.input_nodes,))
        gradient = np.ndarray((self.hidden_layers,))

        if i == self.hidden_layers:
            errors = output_errors
            delta, gradient = calculate_weights_deltas(applied_activation[i], output_errors, applied_activation[i-1])
        elif i == 0:     
            # IT BREAKS HERE
            errors          = calculate_errors(self.weights[i+1], output_errors)
            delta, gradient = calculate_weights_deltas(applied_activation[i], errors, inputs)
        else:
            errors          = calculate_errors(self.weights[i+1], output_errors)
            delta, gradient = calculate_weights_deltas(applied_activation[i], errors, applied_activation[i-1])

        delta, gradient = calculate_weights_deltas(applied_activation[i], errors, applied_activation[i-1])
        # Adjust weights by deltas
        self.weights[i] = np.add(self.weights[i], delta)
        # Adjust the bias by its deltas 
        self.bias[i] = np.add(self.bias[i], gradient)

我得到的错误如下:

            elif i == 0:       
                errors          = calculate_errors(self.weights[i+1], output_errors)

            ValueError: shapes (3,3) and (1,) not aligned: 3 (dim 1) != 1 (dim 0)
2个回答

我发现了我的错误。我早期的计算只涉及一个隐藏层,但是,在计算 hidden_​​error 时,我没有包括先前权重的导数。这意味着在以下神经网络的场景中:

Input -> H1 -> H2 -> Output

为了计算最后一步的反向传播(输入 <- H1),我没有计算 weights_hidden_​​output(H2 -> Output)和 weights_hidden_​​to_hidden(H1 -> H2)的导数。

我建议您为给定的 NN 手动运行计算图。然后尝试推导反向传播方程并检查每个梯度的维度。