Differences

This shows you the differences between two versions of the page.

--- statistical_regression_methods [2020/11/26 08:44] – [Statistical Regression Methods] hkimscil
+++ statistical_regression_methods [2022/05/22 22:06] – hkimscil
@@ Line 5: / Line 5: @@
   - Enter method
   - Selection method
+    - [[:Hierarchical regression]] method Sequential regression method
     - Statistical regression method
       - forward selection: 인들 (predictors) 중 종속변인인 Y와 상관관계가 가장 높은 변인부터 먼저 투입되어 회귀계산이 수행된다. 먼저 투입된 변인은 (상관관계가 높으므로) 이론적으로 종속변인을 설명하는 중요한 요소로 여겨지게 된다. 또한 다음 변인은 우선 투입된 변인을 고려한 상태로 투입된다.
       - backward deletion: 모든 독립변인들이 한꺼번에 투입되어 회귀계산이 시작된다. 이어서 회귀식에 통계학적으로 기여하지 못한다고 판단되는 X변인이 하나씩 제거되면서 회귀계산을 반복적으로 한다.
       - stepwise selection: Forward와 같은 방식으로 회귀계산을 하되, 투입된 변인의 설명력을 계산하여 버릴 것인지 취할 것인지를 결정한다. 각 IV에 대한 t-test를 근거로 그 IV가 significant한 기여를 했는지를 판단하는 것을 말한다.
-    - Sequential regression method
+----
+혹은 아래와 같이 분류하기도 한다
+Multiple Regression
+  - Enter method
+  - Selection method
+    - Hierarchical regression
+    - Stepwise regression -- 이 경우 stepwise 방식은 컴퓨터를 이용하여 criteria를 정하여 독립변인을 고르는 방법을 말한다.
+      - forward
+      - backward
+      - both direction
 See also {{youtube>4Y7PF3Ca3Gk}}
@@ Line 531: / Line 543: @@
 Multiple R-squared: 0.24,   Adjusted R-squared: 0.215
 F-statistic: 9.59 on 6 and 182 DF,  p-value: 0.00000000366 </code>
+===== e.g. SWISS data =====
+<code>A data frame with 47 observations on 6 variables, each of which is in percent, i.e., in [0, 100].
+[,1]	Fertility	Ig, ‘common standardized fertility measure’ 임신율
+[,2]	Agriculture	% of males involved in agriculture as occupation
+[,3]	Examination	% draftees receiving highest mark on army examination
+[,4]	Education	% education beyond primary school for draftees.
+[,5]	Catholic	% ‘catholic’ (as opposed to ‘protestant’).
+[,6]	Infant.Mortality	live births who live less than 1 year.
+All variables but ‘Fertility’ give proportions of the population.</code>
+<code>
+head(swiss)
+fit = lm(Infant.Mortality~., data=swiss)
+summary(fit)
+</code>
+<code>> head(swiss)
+             Fertility Agriculture Examination Education Catholic
+Courtelary        80.2        17.0          15        12     9.96
+Delemont          83.1        45.1           6         9    84.84
+Franches-Mnt      92.5        39.7           5         5    93.40
+Moutier           85.8        36.5          12         7    33.77
+Neuveville        76.9        43.5          17        15     5.16
+Porrentruy        76.1        35.3           9         7    90.57
+             Infant.Mortality
+Courtelary               22.2
+Delemont                 22.2
+Franches-Mnt             20.2
+Moutier                  20.3
+Neuveville               20.6
+Porrentruy               26.6
+> fit = lm(Infant.Mortality~., data=swiss)
+> summary(fit)
+Call:
+lm(formula = Infant.Mortality ~ ., data = swiss)
+Residuals:
+    Min      1Q  Median      3Q     Max
+-8.2512 -1.2860  0.1821  1.6914  6.0937
+Coefficients:
+              Estimate Std. Error t value Pr(>|t|)
+(Intercept)  8.667e+00  5.435e+00   1.595  0.11850
+Fertility    1.510e-01  5.351e-02   2.822  0.00734 **
+Agriculture -1.175e-02  2.812e-02  -0.418  0.67827
+Examination  3.695e-02  9.607e-02   0.385  0.70250
+Education    6.099e-02  8.484e-02   0.719  0.47631
+Catholic     6.711e-05  1.454e-02   0.005  0.99634
+---
+Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
+Residual standard error: 2.683 on 41 degrees of freedom
+Multiple R-squared:  0.2439,	Adjusted R-squared:  0.1517
+F-statistic: 2.645 on 5 and 41 DF,  p-value: 0.03665
+> </code>
+<code>step <- stepAIC(fit, direction="both")
+> step <- stepAIC(fit, direction="both")
+Start:  AIC=98.34
+Infant.Mortality ~ Fertility + Agriculture + Examination + Education +
+    Catholic
+              Df Sum of Sq    RSS     AIC
+- Catholic     1     0.000 295.07  96.341
+- Examination  1     1.065 296.13  96.511
+- Agriculture  1     1.256 296.32  96.541
+- Education    1     3.719 298.79  96.930
+<none>                     295.07  98.341
+- Fertility    1    57.295 352.36 104.682
+Step:  AIC=96.34
+Infant.Mortality ~ Fertility + Agriculture + Examination + Education
+              Df Sum of Sq    RSS     AIC
+- Examination  1     1.320 296.39  94.551
+- Agriculture  1     1.395 296.46  94.563
+- Education    1     5.774 300.84  95.252
+<none>                     295.07  96.341
++ Catholic     1     0.000 295.07  98.341
+- Fertility    1    72.609 367.68 104.681
+Step:  AIC=94.55
+Infant.Mortality ~ Fertility + Agriculture + Education
+              Df Sum of Sq    RSS     AIC
+- Agriculture  1     4.250 300.64  93.220
+- Education    1     6.875 303.26  93.629
+<none>                     296.39  94.551
++ Examination  1     1.320 295.07  96.341
++ Catholic     1     0.255 296.13  96.511
+- Fertility    1    79.804 376.19 103.758
+Step:  AIC=93.22
+Infant.Mortality ~ Fertility + Education
+              Df Sum of Sq    RSS     AIC
+<none>                     300.64  93.220
+- Education    1    21.902 322.54  94.525
++ Agriculture  1     4.250 296.39  94.551
++ Examination  1     4.175 296.46  94.563
++ Catholic     1     2.318 298.32  94.857
+- Fertility    1    85.769 386.41 103.017
+>
+</code>
+<code>> summary(step)
+Call:
+lm(formula = Infant.Mortality ~ Fertility + Education, data = swiss)
+Residuals:
+    Min      1Q  Median      3Q     Max
+-7.6927 -1.4049  0.2218  1.7751  6.1685
+Coefficients:
+            Estimate Std. Error t value Pr(>|t|)
+(Intercept)  8.63758    3.33524   2.590 0.012973 *
+Fertility    0.14615    0.04125   3.543 0.000951 ***
+Education    0.09595    0.05359   1.790 0.080273 .
+---
+Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
+Residual standard error: 2.614 on 44 degrees of freedom
+Multiple R-squared:  0.2296,	Adjusted R-squared:  0.1946
+F-statistic: 6.558 on 2 and 44 DF,  p-value: 0.003215
+></code>
+How about fertility?
+<code>
+head(swiss)
+fit = lm(Fertility ~ ., data=swiss)
+summary(fit)
+</code>
+<code>
+step <- stepAIC(fit, direction="both")
+summary(step)
+</code>
+Then figure out the standardized coefficients, betas.