r:logistic_regression_analysis
e.g. 1
> Logit(Turnover ~ JS, data=td)
Data Frame: mydata
Response Variable: Turnover
Predictor Variable 1: JS
Number of cases (rows) of data: 99
Number of cases retained for analysis: 98
BASIC ANALYSIS
-- Estimated Model of Turnover for the Logit of Reference Group Membership
Estimate Std Err z-value p-value Lower 95% Upper 95%
(Intercept) -1.8554 0.6883 -2.695 0.007 -3.2044 -0.5063
JS 0.4378 0.1958 2.236 0.025 0.0540 0.8216
-- Odds Ratios and Confidence Intervals
Odds Ratio Lower 95% Upper 95%
(Intercept) 0.1564 0.0406 0.6027
JS 1.5492 1.0555 2.2740
-- Model Fit
Null deviance: 131.746 on 97 degrees of freedom
Residual deviance: 126.341 on 96 degrees of freedom
AIC: 130.3413
Number of iterations to convergence: 4
ANALYSIS OF RESIDUALS AND INFLUENCE
Data, Fitted, Residual, Studentized Residual, Dffits, Cook's Distance
[sorted by Cook's Distance]
[res_rows = 20 out of 98 cases (rows) of data]
--------------------------------------------------------------------
JS Turnover fitted residual rstudent dffits cooks
69 6.00 quit 0.6838 -0.6838 -1.5688 -0.3725 0.08496
7 1.38 stay 0.2225 0.7775 1.7682 0.2877 0.06241
73 5.48 quit 0.6327 -0.6327 -1.4476 -0.2949 0.04889
58 5.43 quit 0.6276 -0.6276 -1.4363 -0.2877 0.04618
12 1.72 stay 0.2493 0.7507 1.6920 0.2486 0.04353
31 1.77 stay 0.2534 0.7466 1.6810 0.2429 0.04117
13 1.96 stay 0.2695 0.7305 1.6393 0.2219 0.03314
1 4.96 quit 0.5783 -0.5783 -1.3332 -0.2239 0.02609
33 4.88 quit 0.5698 -0.5698 -1.3162 -0.2138 0.02353
84 4.66 quit 0.5460 -0.5460 -1.2703 -0.1875 0.01757
63 4.65 quit 0.5449 -0.5449 -1.2682 -0.1863 0.01733
61 2.52 stay 0.3203 0.6797 1.5199 0.1668 0.01693
97 5.59 stay 0.6438 0.3562 0.9554 0.2021 0.01693
70 5.48 stay 0.6327 0.3673 0.9731 0.1985 0.01648
74 2.56 stay 0.3242 0.6758 1.5115 0.1635 0.01615
75 2.57 stay 0.3251 0.6749 1.5095 0.1626 0.01596
67 2.65 stay 0.3329 0.6671 1.4929 0.1563 0.01454
80 5.04 stay 0.5869 0.4131 1.0457 0.1813 0.01431
77 4.46 quit 0.5243 -0.5243 -1.2296 -0.1656 0.01336
39 4.43 quit 0.5210 -0.5210 -1.2235 -0.1625 0.01282
PREDICTION
Probability threshold for classification stay: 0.5
0: quit
1: stay
Data, Fitted Values, Standard Errors
[sorted by fitted value]
[pred_all=TRUE to see all intervals displayed]
--------------------------------------------------------------------
JS Turnover label fitted std.err
24 0.23 quit 0 0.1475 0.08116
88 0.67 quit 0 0.1734 0.08096
48 1.05 quit 0 0.1985 0.07904
66 1.19 quit 0 0.2084 0.07790
... for the rows of data where fitted is close to 0.5 ...
JS Turnover label fitted std.err
14 4.14 stay 0 0.4893 0.06579
27 4.15 stay 0 0.4903 0.06609
64 4.26 quit 1 0.5024 0.06946
83 4.41 stay 1 0.5188 0.07431
39 4.43 quit 1 0.5210 0.07497
... for the last 4 rows of sorted data ...
JS Turnover label fitted std.err
70 5.48 stay 1 0.6327 0.1090
73 5.48 quit 1 0.6327 0.1090
97 5.59 stay 1 0.6438 0.1120
69 6.00 quit 1 0.6838 0.1215
--------------------------------------------------------------------
----------------------------
Specified confusion matrices
----------------------------
Probability threshold for predicting stay: 0.5
Corresponding cutoff threshold for JS: 4.238
Baseline Predicted
---------------------------------------------------
Total %Tot 0 1 %Correct
---------------------------------------------------
1 39 39.8 31 8 20.5
Turnover 0 59 60.2 49 10 83.1
---------------------------------------------------
Total 98 58.2
Accuracy: 58.16
Sensitivity: 20.51
Precision: 44.44
>
e.g. 2
d <- subset(iris, Species == "virginica" | Species == "versicolor") head(d) d$Species <- factor(d$Species) str(d) m <- glm(Species ~ ., data=d, family="binomial") round(fitted(m)[c(1:5, 51:55)],3) round(fitted(m)[c(1:5, 51:55)],2) f <- fitted(m) as.numeric(d$Species) ifelse(f > .5, 1, 0) == as.numeric(d$Species) - 1 is_correct <- ifelse(f > .5, 1, 0) == as.numeric(d$Species) - 1 sum(is_correct) sum(is_correct) / NROW(is_correct) predict(m, newdata=d[c(1,10,55),], type="response") d3 <- read.csv(file="d3.csv") round(predict(m, newdata=d3[c(1:5),], type="response"),2) is_correct <- ifelse(f > .5, 1, 0) == as.numeric(d$Species) - 1 sum(is_correct) sum(is_correct) / NROW(is_correct)
> d <- subset(iris, Species == "virginica" | Species == "versicolor")
> d
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
51 7.0 3.2 4.7 1.4 versicolor
52 6.4 3.2 4.5 1.5 versicolor
53 6.9 3.1 4.9 1.5 versicolor
54 5.5 2.3 4.0 1.3 versicolor
55 6.5 2.8 4.6 1.5 versicolor
56 5.7 2.8 4.5 1.3 versicolor
57 6.3 3.3 4.7 1.6 versicolor
58 4.9 2.4 3.3 1.0 versicolor
59 6.6 2.9 4.6 1.3 versicolor
60 5.2 2.7 3.9 1.4 versicolor
61 5.0 2.0 3.5 1.0 versicolor
62 5.9 3.0 4.2 1.5 versicolor
63 6.0 2.2 4.0 1.0 versicolor
64 6.1 2.9 4.7 1.4 versicolor
65 5.6 2.9 3.6 1.3 versicolor
66 6.7 3.1 4.4 1.4 versicolor
67 5.6 3.0 4.5 1.5 versicolor
68 5.8 2.7 4.1 1.0 versicolor
69 6.2 2.2 4.5 1.5 versicolor
70 5.6 2.5 3.9 1.1 versicolor
71 5.9 3.2 4.8 1.8 versicolor
72 6.1 2.8 4.0 1.3 versicolor
73 6.3 2.5 4.9 1.5 versicolor
74 6.1 2.8 4.7 1.2 versicolor
75 6.4 2.9 4.3 1.3 versicolor
76 6.6 3.0 4.4 1.4 versicolor
77 6.8 2.8 4.8 1.4 versicolor
78 6.7 3.0 5.0 1.7 versicolor
79 6.0 2.9 4.5 1.5 versicolor
80 5.7 2.6 3.5 1.0 versicolor
81 5.5 2.4 3.8 1.1 versicolor
82 5.5 2.4 3.7 1.0 versicolor
83 5.8 2.7 3.9 1.2 versicolor
84 6.0 2.7 5.1 1.6 versicolor
85 5.4 3.0 4.5 1.5 versicolor
86 6.0 3.4 4.5 1.6 versicolor
87 6.7 3.1 4.7 1.5 versicolor
88 6.3 2.3 4.4 1.3 versicolor
89 5.6 3.0 4.1 1.3 versicolor
90 5.5 2.5 4.0 1.3 versicolor
91 5.5 2.6 4.4 1.2 versicolor
92 6.1 3.0 4.6 1.4 versicolor
93 5.8 2.6 4.0 1.2 versicolor
94 5.0 2.3 3.3 1.0 versicolor
95 5.6 2.7 4.2 1.3 versicolor
96 5.7 3.0 4.2 1.2 versicolor
97 5.7 2.9 4.2 1.3 versicolor
98 6.2 2.9 4.3 1.3 versicolor
99 5.1 2.5 3.0 1.1 versicolor
100 5.7 2.8 4.1 1.3 versicolor
101 6.3 3.3 6.0 2.5 virginica
102 5.8 2.7 5.1 1.9 virginica
103 7.1 3.0 5.9 2.1 virginica
104 6.3 2.9 5.6 1.8 virginica
105 6.5 3.0 5.8 2.2 virginica
106 7.6 3.0 6.6 2.1 virginica
107 4.9 2.5 4.5 1.7 virginica
108 7.3 2.9 6.3 1.8 virginica
109 6.7 2.5 5.8 1.8 virginica
110 7.2 3.6 6.1 2.5 virginica
111 6.5 3.2 5.1 2.0 virginica
112 6.4 2.7 5.3 1.9 virginica
113 6.8 3.0 5.5 2.1 virginica
114 5.7 2.5 5.0 2.0 virginica
115 5.8 2.8 5.1 2.4 virginica
116 6.4 3.2 5.3 2.3 virginica
117 6.5 3.0 5.5 1.8 virginica
118 7.7 3.8 6.7 2.2 virginica
119 7.7 2.6 6.9 2.3 virginica
120 6.0 2.2 5.0 1.5 virginica
121 6.9 3.2 5.7 2.3 virginica
122 5.6 2.8 4.9 2.0 virginica
123 7.7 2.8 6.7 2.0 virginica
124 6.3 2.7 4.9 1.8 virginica
125 6.7 3.3 5.7 2.1 virginica
126 7.2 3.2 6.0 1.8 virginica
127 6.2 2.8 4.8 1.8 virginica
128 6.1 3.0 4.9 1.8 virginica
129 6.4 2.8 5.6 2.1 virginica
130 7.2 3.0 5.8 1.6 virginica
131 7.4 2.8 6.1 1.9 virginica
132 7.9 3.8 6.4 2.0 virginica
133 6.4 2.8 5.6 2.2 virginica
134 6.3 2.8 5.1 1.5 virginica
135 6.1 2.6 5.6 1.4 virginica
136 7.7 3.0 6.1 2.3 virginica
137 6.3 3.4 5.6 2.4 virginica
138 6.4 3.1 5.5 1.8 virginica
139 6.0 3.0 4.8 1.8 virginica
140 6.9 3.1 5.4 2.1 virginica
141 6.7 3.1 5.6 2.4 virginica
142 6.9 3.1 5.1 2.3 virginica
143 5.8 2.7 5.1 1.9 virginica
144 6.8 3.2 5.9 2.3 virginica
145 6.7 3.3 5.7 2.5 virginica
146 6.7 3.0 5.2 2.3 virginica
147 6.3 2.5 5.0 1.9 virginica
148 6.5 3.0 5.2 2.0 virginica
149 6.2 3.4 5.4 2.3 virginica
150 5.9 3.0 5.1 1.8 virginica
> d$Species <- factor(d$Species) > str(d) 'data.frame': 100 obs. of 5 variables: $ Sepal.Length: num 7 6.4 6.9 5.5 6.5 5.7 6.3 4.9 6.6 5.2 ... $ Sepal.Width : num 3.2 3.2 3.1 2.3 2.8 2.8 3.3 2.4 2.9 2.7 ... $ Petal.Length: num 4.7 4.5 4.9 4 4.6 4.5 4.7 3.3 4.6 3.9 ... $ Petal.Width : num 1.4 1.5 1.5 1.3 1.5 1.3 1.6 1 1.3 1.4 ... $ Species : Factor w/ 2 levels "versicolor","virginica": 1 1 1 1 1 1 1 1 1 1 ...
> m <- glm(Species ~ ., data=d, family="binomial")
> m
Call: glm(formula = Species ~ ., family = "binomial", data = d)
Coefficients:
(Intercept) Sepal.Length Sepal.Width Petal.Length
-42.638 -2.465 -6.681 9.429
Petal.Width
18.286
Degrees of Freedom: 99 Total (i.e. Null); 95 Residual
Null Deviance: 138.6
Residual Deviance: 11.9 AIC: 21.9
> round(fitted(m)[c(1:5, 51:55)],3) 51 52 53 54 55 101 102 103 104 105 0.000 0.000 0.001 0.000 0.001 1.000 1.000 1.000 1.000 1.000 > round(fitted(m)[c(1:5, 51:55)],2) 51 52 53 54 55 101 102 103 104 105 0 0 0 0 0 1 1 1 1 1
> f <- fitted(m) > as.numeric(d$Species) [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 [32] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 [63] 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 [94] 2 2 2 2 2 2 2 > ifelse(f > .5, 1, 0) == as.numeric(d$Species) - 1 51 52 53 54 55 56 57 58 59 60 61 TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE 62 63 64 65 66 67 68 69 70 71 72 TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE 73 74 75 76 77 78 79 80 81 82 83 TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE 84 85 86 87 88 89 90 91 92 93 94 FALSE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE 95 96 97 98 99 100 101 102 103 104 105 TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE 106 107 108 109 110 111 112 113 114 115 116 TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE 117 118 119 120 121 122 123 124 125 126 127 TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE 128 129 130 131 132 133 134 135 136 137 138 TRUE TRUE TRUE TRUE TRUE TRUE FALSE TRUE TRUE TRUE TRUE 139 140 141 142 143 144 145 146 147 148 149 TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE 150 TRUE
> predict(m, newdata=d[c(1,10,55),], type="response")
51 60 105
1.171672e-05 1.481064e-05 9.999999e-01
> d3 <- read.csv("http://commres.net/wiki/_media/d3.csv")
> round(predict(m, newdata=d3[c(1:5),], type="response"),2)
1 2 3 4 5
1.00 1.00 0.92 0.00 0.00
r/logistic_regression_analysis.txt · Last modified: 2023/12/07 08:00 by hkimscil