案例:糖尿病患醫療品質

pacman::p_load(caTools, ggplot2, dplyr)
D = read.csv("data/quality.csv")  # Read in dataset
set.seed(88)
split = sample.split(D$PoorCare, SplitRatio = 0.75)  # split vector
TR = subset(D, split == TRUE)
TS = subset(D, split == FALSE)
glm1 = glm(PoorCare ~ OfficeVisits + Narcotics, TR, family=binomial)
summary(glm1)


【A】傳統準確性指標

Fig 12.1 - 混淆矩陣與模型準確性指標

Fig 12.1 - 混淆矩陣與模型準確性指標


Testing Data

預測機率 Predicted Probability (Testing)

混淆矩陣 Confusion Matrix (Testing)

      Predict
Acture FALSE TRUE
     0    23    1
     1     5    3

模型準確性指標 Accuracy Matrices (Testing)

            Train  Test
accuracy    0.808 0.812
sensitivity 0.400 0.375
specificity 0.946 0.958



【B】預測機率分佈、臨界機率、混淆矩陣

Fig 12.2 - 預測機率分佈、臨界機率、混淆矩陣

Fig 12.2 - 預測機率分佈、臨界機率、混淆矩陣


預測機率分佈 (DPP) - Distribution of Predicted Probability (Train)


預測機率分佈 (DPP) - Distribution of Predicted Probability (Test)





【C】作業曲線(ROC)與辨識率(AUC)

ROC - Receiver Operation Curve

AUC - Area Under Curve

[1] 0.77459 0.79948


🗿 練習:
使用TR$MemberID以外的所有欄位,建立一個邏輯式回歸模型來預測PoorCare,並:


Call:
glm(formula = PoorCare ~ InpatientDays + ERVisits + OfficeVisits + 
    Narcotics + DaysSinceLastERVisit + Pain + ProviderCount + 
    MedicalClaims + ClaimLines + StartedOnCombination + AcuteDrugGapSmall, 
    family = binomial, data = TR)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.4950  -0.5389  -0.3037  -0.0612   2.0898  

Coefficients:
                         Estimate Std. Error z value Pr(>|z|)   
(Intercept)              -1.52721    1.24887   -1.22   0.2214   
InpatientDays             0.13978    0.09725    1.44   0.1506   
ERVisits                 -0.32645    0.24856   -1.31   0.1891   
OfficeVisits              0.09852    0.04687    2.10   0.0355 * 
Narcotics                 0.04695    0.04567    1.03   0.3039   
DaysSinceLastERVisit     -0.00389    0.00188   -2.06   0.0391 * 
Pain                      0.00227    0.01538    0.15   0.8827   
ProviderCount             0.02847    0.03330    0.85   0.3927   
MedicalClaims             0.01568    0.02441    0.64   0.5205   
ClaimLines               -0.01055    0.00924   -1.14   0.2535   
StartedOnCombinationTRUE  2.34587    1.66703    1.41   0.1594   
AcuteDrugGapSmall         0.26808    0.10246    2.62   0.0089 **
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 111.888  on 98  degrees of freedom
Residual deviance:  72.527  on 87  degrees of freedom
AIC: 96.53

Number of Fisher Scoring iterations: 6

  【A】 分別畫出TrainingTestingDPP

  【B】 分別畫出TrainingTestingROC

  【C】 分別算出TrainingTestingACCSENSSPEC

      Predict
Acture FALSE TRUE
     0    68    6
     1    14   11
      Predict
Acture FALSE TRUE
     0    23    1
     1     4    4
            Train  Test
accuracy    0.798 0.844
sensitivity 0.440 0.500
specificity 0.919 0.958

  【D】 分別算出TrainingTestingAUC

[1] 0.87568 0.86458

  【E】 跟用兩個預測變數的模型相比,這一個模型有比較準嗎?

  【F】 為什麼它比較準(或比較不準)呢?