零基礎入門金融風控-貸款違約預測

一、賽題資料

賽題以預測用戶貸款是否違約為任務，資料集報名后可見并可下載，該資料來自某信貸平臺的貸款記錄，總資料量超過120w，包含47列變數資訊，其中15列為匿名變數，為了保證比賽的公平性，將會從中抽取80萬條作為訓練集，20萬條作為測驗集A，20萬條作為測驗集B，同時會對employmentTitle、purpose、postCode和title等資訊進行脫敏，

資料可在阿里云學習賽中獲得，

• 欄位表

二、評測標準

提交結果為每個測驗樣本是1的概率，也就是y為1的概率，評價方法為AUC評估模型效果（越大越好），

三、代碼演示

• 說明：下面運行結果貼出的是部分圖，
• 環境：本案例使用的庫
  • pandas 1.3.2
  • matplotlib 3.4.3
  • seaborn 0.11.2
  • numpy 1.21.2
  • scipy 1.4.1
  • scikit-learn 0.24.2
• 使用的是jupyter notebook

1. 資料分析及處理

1.1.0匯入相關庫

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
from scipy import stats
import matplotlib as mpl
#顯示所有列
pd.set_option('display.max_columns',None)
# 警告處理 
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline

1.1.1資料預處理

df_train = pd.read_csv('train.csv')
df_test = pd.read_csv('testA.csv')

df_train.shape, df_test.shape

df_train['train_test'] = 'train'
df_test['train_test'] = 'test'

合并訓練集和測驗集

df = df_train.append(df_test)
df.reset_index(inplace=True)
df.drop('index',inplace=True,axis=1)
display(df.head())

df.info()

缺失值處理

# 需要處理的列名
is_na_cols = [
    'employmentTitle', 'employmentLength', 'postCode', 'dti', 'pubRecBankruptcies',
    'revolUtil', 'title',] + [f'n{i}' for i in range(15)]

對缺失值 用眾數填充

# 對缺失值 用眾數填充
for i in range(len(is_na_cols)):
    most_num = df[is_na_cols[i]].value_counts().index[0]
    df[is_na_cols[i]] = df[is_na_cols[i]].fillna(most_num)
df.info()

分開訓練集和測驗集

df_train = df[df['train_test'] == 'train']
df_test = df[df['train_test'] == 'test']

del df_train['train_test']
del df_test['train_test']

df_train.shape, df_test.shape

洗掉測驗集的預測目標

del df_test['isDefault']

1.1.2數值型變數和非數值型變數的處理與分析

# 非數值型
non_numeric_cols = [
    'grade', 'subGrade', 'employmentLength', 'issueDate', 'earliesCreditLine'
]
# 數值型
numeric_cols = [
    x for x in df_test.columns if x not in non_numeric_cols + ['isDefault']
]
non_numeric_cols, numeric_cols

1.1.3數值型(numeric_cols)測驗集與訓練集分布

畫箱式圖可查看哪些列名是連續型和非連續型變數

# 畫箱式圖
column = numeric_cols # 串列頭
fig = plt.figure(figsize=(20, 40))  # 指定繪圖物件寬度和高度
for i in range(len(column)):
    plt.subplot(13, 4, i + 1)  # 13行3列子圖
    sns.boxplot(df[column[i]], orient="v", width=0.5)  # 箱式圖
    plt.ylabel(column[i], fontsize=8)
plt.show()

1.1.4取出數值連續性變數，查看資料分布

continuous_cols = [
    'id', 'loanAmnt', 'interestRate', 'installment', 'employmentTitle', 'homeOwnership',
    'annualIncome', 'purpose', 'postCode', 'regionCode', 'dti', 'delinquency_2years',
    'ficoRangeLow', 'ficoRangeHigh', 'openAcc', 'pubRec', 'revolBal', 'revolUtil','totalAcc',
    'title', 'n14'
] + [f'n{i}' for i in range(11)] 
non_continuous_cols = [
    x for x in numeric_cols if x not in continuous_cols
]

可視化正太分布，查看測驗集與訓練集的資料是否相同，相同可保留，差距會影響預測結果，就去除，

dist_cols = 6
dist_rows = len(df_test[continuous_cols].columns)
plt.figure(figsize=(4*dist_cols,4*dist_rows))

i=1
for col in df_test[continuous_cols].columns:
    ax=plt.subplot(dist_rows,dist_cols,i)
    ax = sns.kdeplot(df_train[continuous_cols][col], color="Red", shade=True)
    ax = sns.kdeplot(df_test[continuous_cols][col], color="Blue", shade=True)
    ax.set_xlabel(col)
    ax.set_ylabel("Frequency")
    ax = ax.legend(["train","test"])
    
    i+=1
plt.show()

畫QQ圖及正態分布圖

• QQ圖：曲線越接近直線，越接近正態分布，預測效果更好，

train_cols = 6
train_rows = len(df[continuous_cols].columns)
plt.figure(figsize=(4*train_cols,4*train_rows))

i=0
for col in df[continuous_cols].columns:
    i+=1
    ax=plt.subplot(train_rows,train_cols,i)
    sns.distplot(df[continuous_cols][col],fit=stats.norm)
    i+=1
    ax=plt.subplot(train_rows,train_cols,i)
    res = stats.probplot(df[continuous_cols][col], plot=plt)
plt.show()

訓練集的資料和測驗集的資料分布差不多可以將他們整合到一起進行處理

1.1.5查看數值非連續性型資料分布

for i in range(len(non_continuous_cols)):
    print("%s這列的非連續性資料的分布："%non_continuous_cols[i])
    print(df[non_continuous_cols[i]].value_counts())

1.1.6查看非數值型資料分布

for i in range(len(non_numeric_cols)):
    print("%s這列非數值型資料的分布：\n"%non_numeric_cols[i])
    print(df[non_numeric_cols[i]].value_counts())

2. 特征工程

2.1.1 數值非連續性型資料處理

1. policyCode欄位

df['policyCode'].describe()

# 欄位只有一個值，不用了
df.drop('policyCode',axis=1,inplace=True)

2. n13欄位

df['n13'] = df['n13'].apply(lambda x: 1 if x not in [0] else x)
df['n13'].value_counts()

2.1.2 非數值型資料

1. grade欄位

# 非數值型編碼
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
df['grade'] = le.fit_transform(df['grade'])
df['grade'].value_counts()

2. subGrade欄位

from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
df['subGrade'] = le.fit_transform(df['subGrade'])
df['subGrade'].value_counts()

3. employmentLength欄位

# 構造編碼函式
def encoder(x):
    if x[:-5] == '10+ ':
        return 10
    elif x[:-5] == '< 1':
        return 0
    else:
        return int(x[0])
df['employmentLength'] = df['employmentLength'].apply(encoder)
df['employmentLength'].value_counts()

4. issueDate欄位

• 計算離現在多少月就ok了

from datetime import datetime
def encoder1(x):
    x = str(x)
    now = datetime.strptime('2020-07-01','%Y-%m-%d')
    past = datetime.strptime(x,'%Y-%m-%d')
    period = now - past
    period = period.days
    return round(period / 30, 2)
df['issueDate'] = df['issueDate'].apply(encoder1)
df['issueDate'].value_counts()

5. earliesCreditLine欄位

def encoder2(x):
    if x[:3] == 'Jan':
        return x[-4:] + '-' + '01-01'
    if x[:3] == 'Feb':
        return x[-4:] + '-' + '02-01'
    if x[:3] == 'Mar':
        return x[-4:] + '-' + '03-01'
    if x[:3] == 'Apr':
        return x[-4:] + '-' + '04-01'
    if x[:3] == 'May':
        return x[-4:] + '-' + '05-01'
    if x[:3] == 'Jun':
        return x[-4:] + '-' + '06-01'
    if x[:3] == 'Jul':
        return x[-4:] + '-' + '07-01'
    if x[:3] == 'Aug':
        return x[-4:] + '-' + '08-01'
    if x[:3] == 'Sep':
        return x[-4:] + '-' + '09-01'
    if x[:3] == 'Oct':
        return x[-4:] + '-' + '10-01'
    if x[:3] == 'Nov':
        return x[-4:] + '-' + '11-01'
    if x[:3] == 'Dec':
        return x[-4:] + '-' + '12-01'
df['earliesCreditLine'] = df['earliesCreditLine'].apply(encoder2)
df['earliesCreditLine'].value_counts()

df['earliesCreditLine'] = df['earliesCreditLine'].apply(encoder1)
df['earliesCreditLine'].value_counts()

3. 保存檔案

train = df[df['train_test'] == 'train']
test = df[df['train_test'] == 'test']

del test['isDefault']

del train['train_test']
del test['train_test']

train.to_csv('train_process.csv')
test.to_csv('test_process.csv')

4. 資料建模

4.1.1資料查看

# 資料處理
import numpy as np
import pandas as pd

# 資料可視化
import matplotlib.pyplot as plt

# 特征選擇和編碼
from sklearn.preprocessing import LabelEncoder

# 機器學習
from sklearn import model_selection, tree, preprocessing, metrics
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.linear_model import LinearRegression, LogisticRegression, Ridge, Lasso, SGDClassifier
from sklearn.tree import DecisionTreeClassifier

# 網格搜索、隨機搜索
import scipy.stats as st
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import RandomizedSearchCV

# 模型度量（分類）
from sklearn.metrics import precision_recall_fscore_support, roc_curve, auc

# 警告處理 
import warnings
warnings.filterwarnings('ignore')

# 在Jupyter上畫圖
%matplotlib inline

train = pd.read_csv('train_process.csv')
test = pd.read_csv('test_process.csv')
train.shape, test.shape

train.columns,test.columns

# 洗掉Unnamed: 0
del train['Unnamed: 0']
del test['Unnamed: 0']

## 為了正確評估模型性能，將資料劃分為訓練集和測驗集，并在訓練集上訓練模型，在測驗集上驗證模型性能，
from sklearn.model_selection import train_test_split

## 選擇其類別為0和1的樣本 （不包括類別為2的樣本）
data_target_part = train['isDefault']
data_features_part = train[[x for x in train.columns if x != 'isDefault' and 'id']]

## 測驗集大小為20%， 80%/20%分
x_train, x_test, y_train, y_test = train_test_split(data_features_part, data_target_part, test_size = 0.2, random_state = 2020)

x_train.head()

y_train.head()

4.1.1選擇演算法

以下是用到的演算法.

• Logistic Regression
• Random Forest
• Decision Tree
• Gradient Boosted Trees

# 繪制AUC曲線
import time
def plot_roc_curve(y_test, preds):
    fpr, tpr, threshold = metrics.roc_curve(y_test, preds)
    roc_auc = metrics.auc(fpr, tpr)
    plt.title('Receiver Operating Characteristic')
    plt.plot(fpr, tpr, 'b', label = 'AUC = %0.2f' % roc_auc)
    plt.legend(loc = 'lower right')
    plt.plot([0, 1], [0, 1],'r--')
    plt.xlim([-0.01, 1.01])
    plt.ylim([-0.01, 1.01])
    plt.ylabel('True Positive Rate')
    plt.xlabel('False Positive Rate')
    plt.show()

# Logistic Regression
clf1 = LogisticRegression(solver='sag', max_iter=100, multi_class='multinomial')
clf1.fit(x_train, y_train)
## 在訓練集和測驗集上分布利用訓練好的模型進行預測
train_predict = clf1.predict(x_train)
test_predict = clf1.predict(x_test)
from sklearn import metrics

## 利用accuracy（準確度）【預測正確的樣本數目占總預測樣本數目的比例】評估模型效果
print('The accuracy of the Logistic Regression is:',metrics.accuracy_score(y_train,train_predict))
print('The accuracy of the Logistic Regression is:',metrics.accuracy_score(y_test,test_predict))

# 網格搜索
from sklearn.model_selection import GridSearchCV
param_grid = {
              'penalty': ['l2', 'l1'],
              'class_weight': [None, 'balanced'],
              'C': [0, 0.1, 0.5, 1],
                'intercept_scaling': [0.1, 0.5, 1]
             }

clf2 = LogisticRegression(solver='sag')
rfc = GridSearchCV(clf2, param_grid, scoring = 'neg_log_loss', cv=3, n_jobs=-1)
rfc.fit(x_train, y_train)
print(rfc.best_score_)
print(rfc.best_params_)

# Logistic Regression
clf1 = LogisticRegression(solver='sag', max_iter=100, penalty='l2', 
                          class_weight=None, C=0.1, intercept_scaling=0.1)
model = clf1.fit(x_train, y_train)
## 在訓練集和測驗集上分布利用訓練好的模型進行預測
train_predict = clf1.predict(x_train)
test_predict = clf1.predict(x_test)
from sklearn import metrics

## 利用accuracy（準確度）【預測正確的樣本數目占總預測樣本數目的比例】評估模型效果
print('The accuracy of the Logistic Regression is:',metrics.accuracy_score(y_train,train_predict))
print('The accuracy of the Logistic Regression is:',metrics.accuracy_score(y_test,test_predict))

# 畫圖
plot_roc_curve(y_test, model.predict_proba(x_test)[:,1])

# Random Forest

clf1 = RandomForestClassifier()
clf1.fit(x_train, y_train)
## 在訓練集和測驗集上分布利用訓練好的模型進行預測
train_predict = clf1.predict(x_train)
test_predict = clf1.predict(x_test)
from sklearn import metrics

## 利用accuracy（準確度）【預測正確的樣本數目占總預測樣本數目的比例】評估模型效果
print('The accuracy of the Logistic Regression is:',metrics.accuracy_score(y_train,train_predict))
print('The accuracy of the Logistic Regression is:',metrics.accuracy_score(y_test,test_predict))

# 決策樹
clf1 = DecisionTreeClassifier()
model = clf1.fit(x_train, y_train)
## 在訓練集和測驗集上分布利用訓練好的模型進行預測
train_predict = clf1.predict(x_train)
test_predict = clf1.predict(x_test)
from sklearn import metrics

## 利用accuracy（準確度）【預測正確的樣本數目占總預測樣本數目的比例】評估模型效果
print('The accuracy of the Logistic Regression is:',metrics.accuracy_score(y_train,train_predict))
print('The accuracy of the Logistic Regression is:',metrics.accuracy_score(y_test,test_predict))

plot_roc_curve(y_test, model.predict_proba(x_test)[:,1])

# Gradient Boosting Trees

clf1 = GradientBoostingClassifier()
model = clf1.fit(x_train, y_train)
## 在訓練集和測驗集上分布利用訓練好的模型進行預測
train_predict = clf1.predict(x_train)
test_predict = clf1.predict(x_test)
from sklearn import metrics

## 利用accuracy（準確度）【預測正確的樣本數目占總預測樣本數目的比例】評估模型效果
print('The accuracy of the Logistic Regression is:',metrics.accuracy_score(y_train,train_predict))
print('The accuracy of the Logistic Regression is:',metrics.accuracy_score(y_test,test_predict))

plot_roc_curve(y_test, model.predict_proba(x_test)[:,1])

代碼演示結束

三、拓展

感興趣的話還可以做一下特征融合、模型融合，做更好地特征工程使得模型AUC得分更高，