An in-depth exploration of common regression pitfalls, including non-constant variance of errors, serial correlation, and crucial model specification concerns.
Have you ever run a regression, looked at the residual plots, and scratched your head thinking, “Hmm, there’s something weird going on here?” Well, you’re not alone. Many of us, myself included, have encountered odd patterns in our error terms. These “odd” patterns often point to deeper issues—like heteroskedasticity, autocorrelation, or even model specification mistakes.
In this section, we’ll dig into each of these topics with a friendly, practical approach. We’ll see why these issues matter, how to detect them, and what to do when they crop up. By the end, you’ll be able to interpret your regression output way more confidently (and hopefully avoid a few nightmares in the process).
Before we begin, you might recall from previous sections (see 10.2 on assumptions of linear regression) that the classical linear regression model depends on certain conditions—constant variance of error terms, no correlation among residuals, and proper model specification, among others. Here, we’ll expand on those assumptions and see what happens when they’re violated. Let’s jump right in.
To help visualize how these concerns fit into a broader regression workflow, here’s a simple Mermaid diagram:
graph LR
A["Data <br/>(X variables, Y variable)"] --> B["Initial <br/>Regression Model"]
B --> C["Check for <br/>Heteroskedasticity <br/>(Breusch–Pagan, White)"]
C --> D["Check for <br/>Autocorrelation <br/>(Durbin–Watson, Ljung–Box)"]
D --> E["Review Model <br/>Specification <br/>(Omitted Variables, <br/>Functional Form)"]
E --> F["Refine & Refit <br/>(WLS, NW, ARMA, etc.)"]
This flow reminds me of a project I worked on in grad school. We’d run an ordinary least squares (OLS) model, then systematically test the assumptions, refine the specification, and re-estimate. The loop might go a few rounds, but we always ended up with a more robust model in the end.
Heteroskedasticity is a fancy way of saying that the variance of the error term isn’t constant. In the standard linear regression model, we assume:
meaning each error term \(\epsilon_i\) has the same variance \(\sigma^2\). When heteroskedasticity is present, that variance might change depending on the level of some explanatory variable(s) or as time progresses.
Breusch–Pagan (BP) Test: One of the more common tests. You basically regress the squared OLS residuals on the original independent variables (or a function thereof). A significant result indicates that your residuals are not homoskedastic.
White Test: This test is more general. It doesn’t assume a particular functional form of heteroskedasticity. You often see the White test in software outputs as a test for “general” heteroskedasticity.
Visual Inspection: Always helpful. If you see a clear “funnel shape” in the residual vs. fitted-value plot, you might have a problem. I’ve definitely done that “eyeball” test: if the residuals look like a trumpet, it’s probably heteroskedasticity.
Heteroskedasticity might not necessarily bias your coefficient estimates \(\hat{\beta}\). However, it definitely plays havoc with your standard errors. You might think a coefficient is super significant when in reality those standard errors are artificially small. Or you might be missing statistical significance because your standard errors are inflated. This obviously leads to incorrect inferences when you run t-tests and form confidence intervals.
• Robust Standard Errors: Also called White’s correction or “heteroskedasticity-robust standard errors.” They adjust standard errors without changing your point estimates.
• Weighted Least Squares (WLS): Assign different weights to observations to stabilize the variance of residuals.
• Variable Transformations: Sometimes a log–log or other transformation helps address heteroskedasticity by stabilizing the variance.
• Model Refinement: In some cases, the presence of heteroskedasticity implies missing variables or model misspecification.
Here’s a brief Python snippet (just for fun) on how one might do a Breusch–Pagan test using the statsmodels library:
1import statsmodels.api as sm
2from statsmodels.stats.diagnostic import het_breuschpagan
3
4X = sm.add_constant(X)
5model = sm.OLS(y, X).fit()
6
7bp_test = het_breuschpagan(model.resid, model.model.exog)
8bp_stat = bp_test[0]
9bp_pval = bp_test[1]
10
11print(f"Breusch-Pagan Test Statistic: {bp_stat}, p-value: {bp_pval}")
If the p-value is super low, yep, you have heteroskedasticity issues.
Autocorrelation exists when the error terms \(\epsilon_t\) are correlated with each other over time (or over observations if there’s some natural ordering). In time-series data, it’s really common. There’s often a pattern in how your residuals evolve over time.
• Durbin–Watson Statistic: Classic approach for detecting first-order autocorrelation. Values near 2.0 generally signal no autocorrelation. Values significantly below 2.0 often indicate positive autocorrelation.
• Ljung–Box Q Test: More general test that checks for autocorrelation at multiple lags.
• Residual Plots or ACF/PACF: Plot the autocorrelation function (ACF) of the residuals. If the residuals are random, you shouldn’t see significant spikes at successive lags.
When autocorrelation is present, standard errors of the OLS estimates tend to be understated. Just like with heteroskedasticity, your t-statistics may lie to you, showing significance when there isn’t any. This is especially problematic in time-series contexts where you might incorrectly conclude that an economic indicator or factor is driving your dependent variable.
• Newey–West Standard Errors: These standard errors correct for both heteroskedasticity and autocorrelation up to a certain lag. They’re often called HAC (heteroskedasticity and autocorrelation-consistent) standard errors.
• Other Statistical Approaches: If your data is time-series, consider an ARMA or ARIMA model (Section 12.2) that explicitly accounts for serial correlation in the error process.
• Transformation & Differencing: Sometimes differencing the data (or taking logs) reduces serial correlation if the pattern is related to trend or growth rates.
• Model Specification Adjustments: If a key variable is omitted or the functional form is missing crucial lags, incorporating those might mitigate autocorrelation in the residuals.
Specifying your model well is really the bedrock of good regression analysis. You might have the best standard errors in the world, but if your model is fundamentally off—like omitting key drivers—your results can be misleading. Let’s chat about some common pitfalls.
If you forget to include a relevant variable \(Z\) that influences your dependent variable \(Y\) and is correlated with the included regressors \(X\), the estimated coefficients on \(X\) will be biased. This problem can be huge in financial applications. Imagine analyzing stock returns without controlling for market factors that obviously matter (like overall market returns). The coefficients on the variables you include could be way off.
On the other hand, adding extra, irrelevant variables can raise the variance of your estimated coefficients. Your model might become needlessly complicated, leading to higher standard errors and weaker significance. This is also something to keep in mind for model parsimony.
Sometimes a simple linear model just doesn’t cut it. For instance, a relationship might be quadratic or of the form \(\ln(Y) \sim \ln(X)\). If you treat everything as purely linear, you might see weird patterns in the residuals or spurious significance.
If your data on a key variable is inaccurate, regression coefficients can become inconsistent. In finance, measurement error is especially concerning for variables like intangible asset values or proprietary indicators that are “estimated.” If the measurement error is correlated with the model errors, it can wreak havoc on estimation.
Below are some “lessons learned” from my own experience and from standard econometrics texts (Greene, Enders):
• Always Inspect Residuals: A quick residual plot can tell you if variance looks constant or if there’s a time-related pattern.
• Start Simple, Then Refine: Begin with standard OLS; if something seems off, test for heteroskedasticity and autocorrelation.
• Evaluate Model Specification Thoroughly: Don’t just rely on R². Check whether your theory or intuitive reasoning suggests another factor or a different functional form.
• Use the Right Tools: Weighted least squares, robust standard errors, and specialized time-series models exist for a reason. Use them appropriately.
• Cross-Reference with Theoretical Framework: If your finance or economic theory indicates certain variables matter, double-check they’re included. If theory suggests dynamic relationships, consider lagged terms or difference equations.
Let’s consider a scenario from a hypothetical equity returns model. Suppose you estimate:
At first glance, you find a huge t-statistic for \(\beta_2\). You’re excited—maybe you discovered a new inflation-risk factor for equity returns. But then you plot the residuals and notice a big cyclical pattern over time. A Durbin–Watson test indicates strong serial correlation. Next, you check for heteroskedasticity and find that your residual variance rises in more volatile market periods.
So you decide to do two things:
After refining the specification, you find \(\beta_2\) is actually smaller and no longer as statistically significant when these adjustments are made. Moral of the story: failing to handle these issues can lead you astray.
• Don’t rely on raw OLS outputs at face value. Before you interpret your coefficients or start making trades based on your “super significant” factor, confirm that your model meets the classical assumptions.
• On the CFA exam (especially in item-set or scenario-based questions), watch for prompts that mention a pattern in residuals or a funnel shape in residual plots—this often hints at heteroskedasticity.
• If a question references a Durbin–Watson statistic substantially below 2, suspect autocorrelation.
• Pay attention to how the question frames omitted variable bias: if there’s a mention of a crucial factor left out, expect the test to revolve around how that omission biases your slope estimates.
• Time Management: If you see multiple diagnostic tests referencing heteroskedasticity, autocorrelation, or specification, be systematic. Outline steps: (1) identify the problem, (2) name the correct test, and (3) propose the solution.
• Keep your answers direct and concise in constructed-response questions. Examiners often want to see that you can link the correct symptom (residual pattern, test statistic) with the correct remedy (robust standard errors, WLS, or time-series approach).
Good luck, and remember to keep an eye on those assumption violations. They might just pop up when you least expect them!
Important Notice: FinancialAnalystGuide.com provides supplemental CFA study materials, including mock exams, sample exam questions, and other practice resources to aid your exam preparation. These resources are not affiliated with or endorsed by the CFA Institute. CFA® and Chartered Financial Analyst® are registered trademarks owned exclusively by CFA Institute. Our content is independent, and we do not guarantee exam success. CFA Institute does not endorse, promote, or warrant the accuracy or quality of our products.