我有 50000 个观察值，其中 70% 为正目标变量，30% 为负目标变量。我得到了大约 96-99% 的准确率，这当然看起来不真实，我担心我的模型过度拟合，我不明白为什么。我用第 5 和第 95 分位数替换了所有异常值。标准化了数据，但它显示了这种不真实的准确性。进行了一些在线搜索，人们建议检查训练和测试数据准确性之间的差异，我这样做了，而对于随机森林，它来了
Training Accuracy: 0.997975
Test Accuracy: 0.9715
对于逻辑回归，它显示
Training Accuracy: 0.967225
Test Accuracy: 0.9647
这是我用于运行模型的代码：
clf = LogisticRegression()
trained_model = clf.fit(X_train, y_train)
trained_model.fit(X_train, y_train)
predictions = trained_model.predict(X_test)
accuracy_score(y_train, trained_model.predict(X_train))
accuracy_score(y_test, predictions)
我还尝试了 kfold 交叉验证，它给出了类似的结果
skfold = StratifiedKFold(n_splits=10, random_state=100)
model_skfold = LogisticRegression()
results_skfold = model_selection.cross_val_score(model_skfold, X, Y, cv=skfold)
print("Accuracy: %.2f%%" % (results_skfold.mean()*100.0))
最后我应用正则化技术来检查结果，这就是我得到的结果
for c in C:
clf = LogisticRegression(penalty='l1', C=c, solver='liblinear')
clf.fit(X_train, y_train)
y_pred_log_reg = clf.predict(X_test)
acc_log_reg = round( clf.score(X_train, y_train) * 100, 2)
print (str(acc_log_reg) + ' percent')
print('C:', c)
print('Coefficient of each feature:', clf.coef_)
print('Training accuracy:', clf.score(X_train_std, y_train))
print('Test accuracy:', clf.score(X_test_std, y_test))
print('')
结果
96.72 percent
C: 10
Coefficient of each feature: [[-2.50e+00 -1.40e-03 2.65e+00 4.09e-02 -2.03e-03 2.75e-04 1.79e-02
-2.13e-03 -2.18e-03 2.90e-03 2.69e-03 -4.93e+00 -4.89e+00 -4.88e+00
-3.27e+00 -3.30e+00]]
Training accuracy: 0.5062
Test accuracy: 0.5027
96.72 percent
C: 1
Coefficient of each feature: [[-2.50e+00 -1.41e-03 2.66e+00 4.10e-02 -2.04e-03 2.39e-04 1.68e-02
-3.29e-03 -3.80e-03 2.52e-03 2.62e-03 -4.22e-02 -9.55e-03 0.00e+00
-1.73e+00 -1.77e+00]]
Training accuracy: 0.482525
Test accuracy: 0.4738
96.74 percent
C: 0.1
Coefficient of each feature: [[-2.46e+00 -1.38e-03 2.58e+00 4.03e-02 -1.99e-03 2.22e-04 1.44e-02
-4.49e-03 -5.13e-03 2.03e-03 2.20e-03 0.00e+00 0.00e+00 0.00e+00
0.00e+00 -6.54e-03]]
Training accuracy: 0.616675
Test accuracy: 0.6171
95.92 percent
C: 0.001
Coefficient of each feature: [[-1.43e+00 -6.82e-04 1.19e+00 2.73e-02 -1.10e-03 1.22e-04 0.00e+00
-2.74e-03 -2.55e-03 0.00e+00 0.00e+00 0.00e+00 0.00e+00 0.00e+00
0.00e+00 0.00e+00]]
Training accuracy: 0.655075
Test accuracy: 0.6565
我用于标准化和替换异常值的代码
std_scale = preprocessing.StandardScaler().fit(X_train)
X_train_std = std_scale.transform(X_train)
X_test_std = std_scale.transform(X_test)
X.clip(lower=X.quantile(0.05), upper=X.quantile(0.95), axis = 1, inplace = True)
如果需要任何其他信息，请告诉我，我们将不胜感激
