Discussion questions#
PCA gives you clean, orthogonal components that explain most of the variance. Factor Analysis gives you latent constructs that are harder to estimate but more interpretable. If both models fit the data “well,” how do you decide which representation is truer?
You run PCA on your predictors and drop low-variance components. Your model performance gets worse, not better. How could directions with low variance still be important for prediction? What does this say about using ‘variance explained’ as a proxy for ‘importance’?