理解共同民主学习的伪代码

数据挖掘 机器学习 算法
2022-01-30 19:34:54

我正在阅读来自 Deepmind 研究员科学家 Sebastian Ruder blog-link的博客,并且在理解这个民主共同学习的伪代码时遇到了问题。有人可以帮助我了解发生了什么吗?具体来说,我只是想了解第 1-6 行。

博客中民主合作学习的描述:

Democratic Co-learning Rather than treating different feature sets as views, democratic co-learning (Zhou and Goldman, 2004) [18] employs models with different inductive biases. These can be different network architectures in the case of neural networks or completely different learning algorithms. Democratic co-learning first trains each model separately on the complete labelled data L. The models then make predictions on the unlabelled data U. If a majority of models confidently agree on the label of an example, the example is added to the labelled dataset. Confidence is measured in the original formulation by measuring if the sum of the mean confidence intervals w of the models, which agreed on the label is larger than the sum of the models that disagreed. This process is repeated until no more examples are added. The final prediction is made with a majority vote weighted with the confidence intervals of the models. The full algorithm can be seen below. M is the set of all models that predict the same label j for an example x.

共同学习的伪代码

我猜第 1-3 行是在标记数据上训练几个模型。我是标记数据集中的每个样本吗?mi 是使用标记数据集中所有样本的训练模型吗?我猜第 4-6 行正在遍历未标记数据集中的每个样本,但是 j 是什么?它是否遍历每个标签?

我特别困惑的这一行是 M <- {i | pi(x) = j}。我是否被分配了来自未标记样本的模型输出,然​​后被分配给 M,M 是什么?

1个回答

让我们试着弄明白这里发生了什么:

如您所说,在前 3 行中,我们构建n模型,正在m单个模型。

第二个循环开始迭代未标记的样本集U. 对于每个未标记的样本x,我们遍历可能的标签(例如{0,1}对于二元分类,或dog vs. cat等...)从集合C我们称之为j.

M因此是模型集m,即预测该样本x属于班级j

之后,如果模型数量|M|(在哪里0|M|n) 同意标签是j超过一半的可用型号(|M|>n/2)加权平均值 (w是模型的置信区间)同意这个预测的模型高于不同意的模型(第 7 行)→ 然后样本x包含在L带标签j(第 8 行)。