Differences

This shows you the differences between two versions of the page.

--- multiple_regression [2019/05/21 22:33] – [무엇부터? 라는 문제] hkimscil
+++ multiple_regression [2019/05/23 10:20] – [Why overall model is significant while IVs are not?] hkimscil
@@ Line 331: / Line 331: @@
 see https://www.researchgate.net/post/Why_is_the_Multiple_regression_model_not_significant_while_simple_regression_for_the_same_variables_is_significant
-This problem
+<code>
+RSS = 3:10 #Right shoe size
+LSS = rnorm(RSS, RSS, 0.1) #Left shoe size - similar to RSS
+cor(LSS, RSS) #correlation ~ 0.99
+weights = 120 + rnorm(RSS, 10*RSS, 10)
+##Fit a joint model
+m = lm(weights ~ LSS + RSS)
+##F-value is very small, but neither LSS or RSS are significant
+summary(m)
+</code>
+<code>> RSS = 3:10 #Right shoe size
+> LSS = rnorm(RSS, RSS, 0.1) #Left shoe size - similar to RSS
+> cor(LSS, RSS) #correlation ~ 0.99
+[1] 0.9994836
+>
+> weights = 120 + rnorm(RSS, 10*RSS, 10)
+>
+> ##Fit a joint model
+> m = lm(weights ~ LSS + RSS)
+>
+> ##F-value is very small, but neither LSS or RSS are significant
+> summary(m)
+Call:
+lm(formula = weights ~ LSS + RSS)
+Residuals:
+       2       3       4       5       6       7       8
+.8544  4.5254 -3.6333 -7.6402 -0.2467 -3.1997 -5.2665 10.6066
+Coefficients:
+            Estimate Std. Error t value Pr(>|t|)
+(Intercept)  104.842      8.169  12.834 5.11e-05 ***
+LSS          -14.162     35.447  -0.400    0.706
+RSS           26.305     35.034   0.751    0.487
+---
+Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
+Residual standard error: 7.296 on 5 degrees of freedom
+Multiple R-squared:  0.9599,	Adjusted R-squared:  0.9439
+F-statistic: 59.92 on 2 and 5 DF,  p-value: 0.000321
+>
+> ##Fitting RSS or LSS separately gives a significant result.
+> summary(lm(weights ~ LSS))
+Call:
+lm(formula = weights ~ LSS)
+Residuals:
+   Min     1Q Median     3Q    Max
+-6.055 -4.930 -2.925  4.886 11.854
+Coefficients:
+            Estimate Std. Error t value Pr(>|t|)
+(Intercept)  103.099      7.543   13.67 9.53e-06 ***
+LSS           12.440      1.097   11.34 2.81e-05 ***
+---
+Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
+Residual standard error: 7.026 on 6 degrees of freedom
+Multiple R-squared:  0.9554,	Adjusted R-squared:  0.948
+F-statistic: 128.6 on 1 and 6 DF,  p-value: 2.814e-05
+>
+</code>