Hypothesis testing of the overall regression model - analysis of variance
Hypothesis test of partial regression coefficient—t test
Independent variable screening
Only variables that contribute significantly to the dependent variable are included in the equation
backward elimination
Focus on introducing variables with strong joint effects
Forward introduction method (forward)
Focus on introducing independent variables that have a strong independent effect
Stepwise screening method (stepwise)
When there is no linear correlation between independent variables, the calculation results of the three methods are the same.
The meaning of several coefficients
coefficient of determination
Complex correlation coefficient
adjusted coefficient of determination
Standardized partial regression coefficient
There is no unit of measurement, which eliminates the impact of different units of measurement and degree of variation on the partial regression coefficient.
The absolute value can be used to compare the degree of influence of its corresponding independent variable on the dependent variable.
Coefficient of determination R2
To what extent can the regression equation including all independent variables explain the variability of the dependent variable Y.
The larger the value, the better the regression effect.
Complex correlation coefficient R
Indicates the degree of linear correlation between y and p independent variables
adjusted coefficient of determination
The index after deducting the influence of the number of independent variables on the coefficient of determination, that is, the index is not affected by the number of independent variables.
application
Analysis of influencing factors
Things to note
Sample size estimates
Rules of thumb
Generally, the sample size should be at least 10-20 times the number of independent variables.
Quantification of qualitative variables
Multiple unordered categories (nominal variables)
Dummy variables should be set
Identification and processing of strong influence points
Judgment of strong influence points
When the standardized residual is >3, the record can be identified as a strong influence point.
Treatment method
Check whether the record is due to recording or entry errors. If so, correct those that can be corrected and eliminate those that cannot.
Consider whether the record does not belong to the same group as other records in the database (does the record belong to another subgroup?) If so, it should be deleted
If it does not fall into the above two categories, check whether the fitted model is suitable? Fitting other forms of models should be considered to correct
Robust regression, nonparametric regression
If circumstances permit, the sample size can be increased. The increase in the amount of information can appropriately weaken the role of strong influence points.
multicollinearity
Refers to the linear correlation between multiple independent variables
Performance: linear regression model, especially partial regression coefficients cannot be professionally interpreted
The test result of the entire model is P<ɑ, but the partial regression coefficient of each variable is P>ɑ
An independent variable that is considered statistically significant by professionals, but the test result is not statistically significant.
The values and even signs of the partial regression coefficients of the independent variables are contrary to the actual situation and are difficult to interpret.
The partial regression coefficient is unstable. When an independent variable or a record is added or deleted, the partial regression coefficient of the independent variable changes greatly.
identify
Tolerance
The smaller the tolerance, the more severe the multicollinearity. When the tolerance is <0.1, it indicates serious collinearity.
Variance inflation factor VIF
The reciprocal of tolerance. The larger the variance expansion coefficient, the greater the possibility of collinearity between independent variables; generally speaking, the variance expansion coefficient should not be >=5, and can also be relaxed to >=10
deal with
delete variable
Among the variables with strong correlation, delete the variables with the largest measurement error and the most missing data, which are not very important from a professional perspective.
Use other regression methods
Interactions between explanatory variables
When analyzing, the interaction term can be introduced to analyze
Setting of inspection levels
When the number of variables is small or the study is exploratory
Entry=0.10, Elimination=0.15
When the number of variables is large or confirmatory research
Entry=0.05, Elimination=0.10
Example 13-3 Analysis process
1. Draw a scatter plot (matrix scatter plot)
3. Normality and homogeneity of variances
4. Collinearity diagnosis
5. Linear regression analysis—establish regression equation—analyze influencing factors
Wondershare EdrawMind
