逻辑回归的早期停止。西阿诺

数据挖掘 逻辑回归 西阿诺
2022-03-08 08:19:15

我试图理解官方文档中逻辑回归的代码,但我很难理解这段代码背后的逻辑:

# early-stopping parameters
patience = 5000  # look as this many examples regardless
patience_increase = 2     # wait this much longer when a new best is
                              # found
improvement_threshold = 0.995  # a relative improvement of this much is
                               # considered significant
validation_frequency = min(n_train_batches, patience/2)
                              # go through this many
                              # minibatches before checking the network
                              # on the validation set; in this case we
                              # check every epoch

best_params = None
best_validation_loss = numpy.inf
test_score = 0.
start_time = time.clock()

done_looping = False
epoch = 0
while (epoch < n_epochs) and (not done_looping):
    # Report "1" for first epoch, "n_epochs" for last epoch
    epoch = epoch + 1
    for minibatch_index in xrange(n_train_batches):

        d_loss_wrt_params = ... # compute gradient
        params -= learning_rate * d_loss_wrt_params # gradient descent

        # iteration number. We want it to start at 0.
        iter = (epoch - 1) * n_train_batches + minibatch_index
        # note that if we do `iter % validation_frequency` it will be
        # true for iter = 0 which we do not want. We want it true for
        # iter = validation_frequency - 1.
        if (iter + 1) % validation_frequency == 0:

            this_validation_loss = ... # compute zero-one loss on validation set

            if this_validation_loss < best_validation_loss:

                # improve patience if loss improvement is good enough
                if this_validation_loss < best_validation_loss * improvement_threshold:

                    patience = max(patience, iter * patience_increase)
                best_params = copy.deepcopy(params)
                best_validation_loss = this_validation_loss

        if patience <= iter:
            done_looping = True
            break

任何人都可以向我解释一下,这些变量是什么:耐心,耐心_增加,改进阈值,验证频率,迭代器,代表什么?

这个条件做什么?

if (iter + 1) % validation_frequency == 0:
1个回答

Patience是停止之前要进行的训练批次数。iter是您看到的训练批次数。每次迭代,你决定你的验证是否低于你以前的最好。仅当分数低于 时才存储改进improvement_threshold * validation_score

这似乎patience_increase是一个乘数。每次你有一个新的最低分数时,你将总数或训练批次增加到iter*patience_increase,但不低于 的当前值patience

validation_frequency只是您检查验证分数之间的批次数。