数据挖掘 - 特征和结果之间的大 .corr() 回归的交叉验证分数非常低 - 吾爱随笔录

特征和结果之间的大 .corr() 回归的交叉验证分数非常低

数据挖掘 Python 机器学习 scikit-学习交叉验证

2022-02-23 08:07:27

我试图用 sklearn 在一个特征和一个结果之间进行回归。这是我拥有的数据集：

       bruto  ukupno gradjevinski din
0    2494.98                857951.27
1    2912.60                694473.11
2    3397.50               1310529.72
3    2678.00                199688.14
4    4310.00               1377366.95
5    2086.28                569312.33
6    3061.80                660803.42
7    4095.00               1187732.61
8    3997.00               1304793.08
9    6503.88               1659629.13
10   6732.00               1264178.31
11    940.10                172497.94
12   1543.00                598772.40
13   5903.85                809681.19
14   2861.61                333983.85
15   3682.76               1430771.50
16   2802.00               1145812.21
17   3032.00                356840.54
18   2635.00                543912.80
19   3749.00               1004940.27
20   4300.50               1889560.55
21   9722.00               2137376.95
22   3823.33                891633.50
23   1648.21                335115.40
24  24575.00              19273129.14
25   3926.00               1223803.28
26   3228.00                874000.00
27   4062.00               1090000.00
28   1316.24                332718.54
29   2497.99                519398.70
30  12123.94               2504783.69
31   2057.50                957042.37
32   2495.00                857951.27
33   3770.73               1743978.85
34    864.00                251269.48
35    774.71                192487.26

我用 .corr() 发现了特征和结果之间的相关性：

                            bruto  ukupno gradjevinski din
bruto                    1.000000                 0.878914
ukupno gradjevinski din  0.878914                 1.000000

我的 corr 为 0.87，我认为这对于回归来说非常不错，但是当我制作回归模型并获得 cross-val 得分时，我得到的 cross-val 得分值为负且大于 1（有时为 -50.23）这对我来说很奇怪。我尝试了很多不同的模型和不同的折叠次数，但结果是一样的。这是回归的代码：

features = df[['bruto']]
results = df[['ukupno gradjevinski din']]

regressors = [["Linear Regression", LinearRegression(normalize=False)],
              ["Lasso Regression", Lasso(normalize=False)],
              ["Gaussian Process Regressor", GaussianProcessRegressor()],              
              ["SVR linear", SVR(kernel = 'linear', gamma='scale', max_iter = 1500)],
              ["SVR poly 2", SVR(kernel = 'poly', degree=2, gamma='scale', max_iter = 1500)],
              ["SVR poly 3", SVR(kernel = 'poly', degree=3, gamma='scale', max_iter = 1500)],
              ["SVR poly 4", SVR(kernel = 'poly', degree=4, gamma='scale', max_iter = 1500)],
              ["SVR poly 5", SVR(kernel = 'poly', degree=5, gamma='scale', max_iter = 1500)],
              ["SVR rbf C=0.01", SVR(kernel = 'rbf', C=0.01, gamma='scale', max_iter = 1500)],              
              ["SVR rbf C=0.1", SVR(kernel = 'rbf', C=0.1, gamma='scale', max_iter = 1500)],
              ["SVR rbf C=0.5", SVR(kernel = 'rbf', C=0.5, gamma='scale', max_iter = 1500)],
              ["SVR rbf C=1", SVR(kernel = 'rbf', C=1, gamma='scale', max_iter = 1500)],              
              ["SVR rbf C=10", SVR(kernel = 'rbf', C=10.0, gamma='scale', max_iter = 1500)],
              ["SVR rbf C=20", SVR(kernel = 'rbf', C=20.0, gamma='scale', max_iter = 1500)],
              ["SVR rbf C=50", SVR(kernel = 'rbf', C=50.0, gamma='scale', max_iter = 1500)],              
              ["SVR sigmoid", SVR(kernel = 'sigmoid', gamma='scale', max_iter = 1500)],
              ["GradientBoostingRegressor", GradientBoostingRegressor()],
              ["RandomForestRegressor", RandomForestRegressor(n_estimators = 150)],
              ["DecisionTreeRegressor", DecisionTreeRegressor(max_depth=10)],
              ["Bagging Regressor TREE", BaggingRegressor(base_estimator = DecisionTreeRegressor(max_depth=15))],
              ["Bagging Regressor FOREST", BaggingRegressor(base_estimator = RandomForestRegressor(n_estimators = 100))],
              ["Bagging Regressor linear", BaggingRegressor(base_estimator = LinearRegression(normalize=True))],
              ["Bagging Regressor lasso", BaggingRegressor(base_estimator = Lasso(normalize=True))],
              ["Bagging Regressor SVR rbf", BaggingRegressor(base_estimator = SVR(kernel = 'rbf', C=10.0, gamma='scale'))],
              ["Extra Trees Regressor", ExtraTreesRegressor(n_estimators = 150)],
              ["K-Neighbors Regressor 1", KNeighborsRegressor(n_neighbors=1)],
              ["K-Neighbors Regressor 2", KNeighborsRegressor(n_neighbors=2)],
              ["K-Neighbors Regressor 3", KNeighborsRegressor(n_neighbors=3)],
              ["AdaBoostRegressor", AdaBoostRegressor(base_estimator=None)],
              ["AdaBoostRegressor tree", AdaBoostRegressor(base_estimator=DecisionTreeRegressor(max_depth=15))],
              ["AdaBoostRegressor forest", AdaBoostRegressor(base_estimator=RandomForestRegressor(n_estimators = 100))],
              ["AdaBoostRegressor lin reg", AdaBoostRegressor(base_estimator=LinearRegression(normalize=True))],
              ["AdaBoostRegressor lasso", AdaBoostRegressor(base_estimator = Lasso(normalize=True))]]


for reg in regressors:

     try:

           scores = cross_val_score(reg[1], features, results, cv=5)
           scores = np.average(scores)
           print('cross val score', scores)
           print()

     except:
          continue

我尝试使用 Normalizer、StandardScaler 和 MinMaxScaler 来扩展我的功能，但结果是一样的。有什么帮助吗？

1个回答

我正要在另一个论坛上发布我的答案，但它已迁移到这里。

您应该记住一些关键的事情：

不是谁拥有最好的算法才能获胜。谁拥有最多的数据。（班科和布里尔，2001）

Bank 和 Brill 在 2001 年对 4 种不同的算法进行了比较，他们不断将训练集大小增加到数百万，并得出了上述结论。而且你的数据太少了！

每当您谈论线性模型时，请记住它们的敌人——异常值。如果您绘制数据，您可以清楚地看到这一点。

cross_val_score几乎所有线性模型（即回归器）默认返回 R^2。该度量的最佳值 = 1（即完全拟合），或 = 0（即水平线），或者它可以是负数（即比水平线差）。更多信息在这里。接下来在我进行的实验中，您将看到结果如何有效。
另一种模型是Multi-layer Perceptron Regressor; 层数 = 3，模型将映射任何复杂的函数。
如果您有足够的数据，交叉验证将是最好的选择。但是，在您的情况下，CV 分数差异很大。

请思考以下不言自明的实验的结果：

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score
from sklearn.neural_network import MLPRegressor
from scipy.stats import pearsonr
import numpy as np
import matplotlib.pyplot as plt

X = np.array([2494.98,2912.6,3397.5,2678,4310,2086.28,3061.8,4095,3997,
              6503.88,6732,940.1,1543,5903.85,2861.61,3682.76,2802,3032,
              2635,3749,4300.5,9722,3823.33,1648.21,24575,3926,3228,4062,1316.24,
              2497.99,12123.94,2057.5,2495,3770.73,864,774.71]).reshape(-1, 1)

y = np.array([857951.27,694473.11,1310529.72,199688.14,1377366.95,569312.33,660803.42,1187732.61,
          1304793.08,1659629.13,1264178.31,172497.94,598772.4,809681.19,333983.85,1430771.5,1145812.21,
          356840.54,543912.8,1004940.27,1889560.55,2137376.95,891633.5,335115.4,19273129.14,1223803.28,
          874000,1090000,332718.54,519398.7,2504783.69,957042.37,857951.27,1743978.85,251269.48,192487.26])

X_, y_ = zip(*sorted(zip(X, y)))
plt.plot(X_, y_, '-x')
plt.title("Plot of Dataset")
plt.show()

print("Linear Regression :: Before Removing An Outlier")
reg = LinearRegression()
print(np.average(cross_val_score(reg, X, y, cv=3)))

X, y = X_[:-1], y_[:-1]
plt.plot(X, y, '-x')
plt.title("Plot of Dataset After Removing Outlier")
plt.show()

print("Linear Regression :: After Removing An Outlier")
reg = LinearRegression()
print(np.average(cross_val_score(reg, np.array(X).reshape(-1, 1), y, cv=3)))

print("Multi-layer Perceptron Regressor :: The Effect of Mapping Complicated / Non-Linear Function")
mlp = MLPRegressor(hidden_layer_sizes=(16, 16, 16), random_state=2020, activation='identity', max_iter=1000)
print(np.average(cross_val_score(mlp, np.array(X).reshape(-1, 1), y, cv=3)))

结果

这是在仅删除一个极值之后（没有进一步探索，也没有做任何花哨的工作，例如使用任何异常值检测器）。如您所见，没有一条线适合所有点。

Linear Regression :: Before Removing An Outlier
Average CVs Score: -1.7085612243433703

Linear Regression :: After Removing An Outlier
Average CVs Score: -0.12386365189238795

Multi-layer Perceptron Regressor :: The Effect of Mapping Complicated / Non-Linear Function
Average CVs Score: 0.16131374234257037

其它你可能感兴趣的问题

上一篇可解释的AI，计算机是如何对文本进行分类的？下一篇哪种（天真的）NLP 方法用于关联聊天室中的人类消息？