二元分类器模型中 ROC AUC 得分的异常值

数据挖掘 Python 神经网络 深度学习 喀拉斯 python-3.x
2022-02-20 15:21:57

我为二元分类器开发了以下模型。我必须使用roc_auc_score. 我得到了不寻常的价值roc_auc_score

当我使用
epoch = 1, roc_auc_score = 0.8
epoch = 2, roc_auc_score =0.53

在此之后,对于任何时期,它都保持在 0.5。为什么会这样?我的模型有问题吗?

数据预处理

#Load Dataset
test = pd.read_csv('test.csv')
train = pd.read_csv('train.csv')

#Combine Train and Test set for Data Cleaning
train['set'] = 'train'
test['set'] = 'test'
df = pd.concat([test, train])

#One Hot Encoding
df = pd.get_dummies(df, columns=['Gender','Driving_License','Previously_Insured','Vehicle_Age','Vehicle_Damage'])

#Moving Target Column to End
target = df['Response']
df.drop(labels=['Response'], axis=1, inplace = True)
df.insert(16, 'Response', target)

#Separating Train and Test Data
train = df[df['set']=='train']
test = df[df['set']=='test']
train = train.drop('set', 1)
test = test.drop('set', 1)

#Creating Input Features and Target Variables
X= train.iloc[:,1:15]
y= train.iloc[:,[15]]

#Standardizing the Input Features
scaler = StandardScaler()
X = scaler.fit_transform(X)

#Train Test Split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

代码

#Model
model = Sequential()

model.add(Dense(14, activation='relu', kernel_initializer='random_normal', input_dim=14))
  
#Output Layer
model.add(Dense(1, activation = 'sigmoid', kernel_initializer='random_normal'))

#Compiling the neural network
model.compile(optimizer ='adam',loss='binary_crossentropy', metrics =['accuracy'])

#Fitting the data to the training dataset  
model.fit(X_train,y_train, batch_size=32, epochs=1, verbose=0) 

#Make predictions and convert to binary value
pred_train = model.predict(X_train)
pred_test = model.predict(X_test)  

#ROC AUC Score
print('Train AUC = {:.5f}'.format(roc_auc_score(y_train,pred_train)))
print('Test AUC = {:.5f}'.format(roc_auc_score(y_test,pred_test)))

#Accuracy
print('Train Accuracy = {:.3f}'.format(accuracy_score(y_train,pred_train.round())))
print('Test Accuracy = {:.3f}'.format(accuracy_score(y_test,pred_test.round())))
0个回答
没有发现任何回复~