Deepen your understanding of polynomial regression, interaction terms, and step functions. Learn how to capture nonlinear relationships, interpret model coefficients, and avoid overfitting in a CFA® Level II context.
So, remember those straight‑line regressions with a single slope and intercept? Well, sometimes reality just doesn’t behave that nicely. In practice, market returns can surge and then plateau, interest rates can accelerate and then level off, and all sorts of financial indicators can bend in ways that a simple linear model can’t handle. That’s where polynomial terms, interaction effects, and step (piecewise) functions come rushing in to save the day.
Polynomial terms let us capture that swoop or curve in the data. Interaction terms reveal all those interesting ways in which one variable’s effect depends on another. Step functions highlight those threshold points—like how a bond’s duration risk might change dramatically when rates cross a certain barrier. In a sense, these techniques help us respect reality’s complexity without having to jump to super-advanced machine learning methods.
Anyway, let’s see how you can fold in polynomial, interaction, and step functions into your regression toolkit. We’ll talk about the mechanics, the why, and the so-what—always tying it back to real financial examples that might pop up in CFA Level II item sets.
Occasionally, you might sense that the effect of X on Y isn’t strictly a straight line. Think about how consumer spending might ramp up slowly during recoveries, accelerate sharply as growth heats up, and then level off once markets saturate or inflation bites. A linear regression might either understate or overstate certain segments of that data. That’s precisely when we fit a polynomial regression by including squared or cubed (or even higher powers) terms of an explanatory variable.
From a model-building perspective, a polynomial regression can be expressed as:
where \( x^2 \) and \( x^3 \) capture the curvature that a simple linear model cannot.
Picture a portfolio manager analyzing the relationship between small-cap stock returns (Y) and a market volatility index (X). If volatility is very low, small-caps might not offer much excitement. As volatility rises, small-cap returns might increase—reflecting risk-on sentiment. But at extremely high volatility levels, small-cap returns might tumble. Including \( x^2 \) might capture that hump-shaped (inverted U) pattern.
It’s tempting to interpret \(\beta_2\) or \(\beta_3\) in a polynomial regression at face value. Careful, though: the effect of an incremental change in X depends on all the terms. The marginal effect of X on Y is given by the partial derivative:
If you’re working through a CFA vignette, it can help to evaluate this derivative at particular values of X. That shows the slope of the relationship—like measuring how returns shift when volatility is at 10, 20, or 30.
Adding \( x \), \( x^2 \), \( x^3 \), and so on can inflate collinearity. (After all, \( x^2 \) is highly correlated with \( x \) in many datasets—both just track “bigger X.”) This can bloat standard errors and complicate significance tests. A common remedy is to center or standardize X before generating powers, effectively letting the intercept capture the mean and reducing correlations among polynomial terms.
You don’t want to keep piling on powers blindly. Test whether an additional polynomial term is justified—maybe by an incremental F‑test, an information criterion (like AIC or BIC), or cross-validation. In practice, if the second‑degree polynomial doesn’t significantly improve R-squared or reduce your out-of-sample error, it might not be worth the extra complexity.
All too often in finance, the effect of one variable depends on the level of another. Let’s say the strength of a momentum strategy (X₁) might differ significantly depending on market liquidity (X₂). In a plain linear model without an interaction term, you’d simply have:
But to capture how \( X_1 \) interacts with \( X_2 \) in a combined effect, you add \( X_1 \times X_2 \) to the mix:
Now, \(\beta_3\) reveals how the slope on \( X_1 \) changes as \( X_2 \) changes, and vice versa.
Suppose you’re modeling a stock’s returns (Y) using its price-to-earnings ratio (X₁) and an overall market sentiment index (X₂). Maybe high P/E is good only when sentiment is bullish. When an interaction term is significant, you’ll see that the relationship between P/E and returns is magnified or even reversed at different levels of sentiment.
The coefficient \(\beta_3\) from \( X_1 X_2 \) doesn’t just stand alone; it modifies the original main effects. If you want the effect of \( X_1 \) on Y at a specific value of \( X_2 \), you’d compute:
Likewise,
This helps you interpret how each predictor’s slope changes in different contexts—something that can pop up in item sets where you must identify the effect of varying levels of another variable.
Markets frequently exhibit threshold behaviors. A stock might exhibit one pattern of returns if its price remains below $50 (value zone, perhaps), but above $50, the market’s done messing around, and the valuation jumps onto a new plateau. Instead of forcing a continuous polynomial curve, piecewise regression (or step functions) can explicitly allow for a different slope or intercept once X crosses a threshold, known as a “knot” or “breakpoint.”
Consider a single knot at \( x = c \). You might fit a model:
In practice, we often implement this by creating a new variable, say \( (X - c)_+ \), that equals \( 0 \) if \( X \le c \) and \( (X - c) \) otherwise. This ensures a continuous function but allows the slope to change after \( X \) passes \( c \).
• Credit Risk: The effect of a firm’s debt-to-equity ratio might shift drastically once it crosses 1.0.
• Regulatory Thresholds: New capital adequacy rules might kick in beyond a certain ratio, changing a bank’s returns or lending volumes.
• Behavioral: Individual investors could react differently to stock price changes below $100 than they do above $100.
In theory, you can set knots at known regulatory or psychological thresholds. Sometimes you do an exploratory analysis: scanning potential breakpoints to see which significantly improves model performance. The risk is data-snooping or overfitting if you try too many breakpoints.
All these expansions—extra polynomial terms, interactions, and step functions—can drastically improve in-sample fit, but that’s not always helpful if the model starts chasing random noise. Overly complex models can yield inflated R-squared values but fail miserably out of sample.
Cross-validation (like k-fold) is your friend. It’s a systematic way to see how well your model generalizes. If you see big discrepancies in performance across folds, or if a simpler model is consistently doing just as well, it might be time to scale back.
Another more formal approach is to consult the AIC (Akaike Information Criterion) or BIC (Bayesian Information Criterion). These penalize complexity, so adding polynomial or step terms that don’t actually help yields higher (worse) AIC/BIC.
Sometimes you just want to see if you managed to iron out the nonlinearity. Beyond standard checks of normality or homoscedasticity, plot the residuals against fitted values (or your primary predictor X). If the original linear model had a U-shaped pattern, a well-specified polynomial or step function model hopefully flattens that out.
If new patterns appear—like wave shapes or abrupt step-like shifts in your residual plot—maybe you need a different type of break, or you still need an additional polynomial term. Even with advanced regression techniques, never skip investigating the leftover structure in your residuals.
• Look for Clues: If the vignette shows a residual plot with obvious curvature, guess that they’re hinting at polynomial terms or step functions.
• Partial Correlations: Before building an overstuffed model, see if the partial correlation between X and Y suggests a polynomial shape. Quick scatterplots or descriptive stats can guide your thinking.
• Interaction Mentions: Watch if the wording hints that the effect of one variable “depends on” or “varies with” another. That’s generally code for an interaction term.
• Focus on Interpretation: CFA exams love asking how to interpret a certain coefficient or how the relationship changes for certain values. If you see an interaction or polynomial, anticipate a question about partial derivatives, marginal effects, or how the slope transitions in different intervals.
• Model Fit Tests: If they provide partial F-tests or AIC/BIC changes, they might want you to identify which terms meaningfully improve the model.
Below is a simple flowchart that illustrates how you might decide between a standard linear model, a polynomial approach, or step functions when diagnosing nonlinearity in your data.
flowchart LR A["Data & Residual Plots"] --> B["Detect Curvature or Threshold?"] B --> C["No: Use Linear Model"] B --> D["Yes: Use Polynomial or Step Function"] D --> E["Check Overfitting <br/> with AIC/BIC or Cross-Validation"] E --> F["Refine Model or Finalize"]
This high-level workflow isn’t set in stone, but it’s a handy reminder that you typically explore the data, look for hints of nonlinearity, and then proceed to more complex forms—always mindful of overfitting.
Polynomial, interaction, and step function methods offer a flexible way to capture relationships that plain old linear regression just can’t handle. Whether you’re dealing with the cyclical nature of market indicators, synergy between two asset pricing factors, or threshold behaviors that abruptly change beyond a specific point, these techniques enlarge your regression toolbox.
Still, you’ll want to be cautious. It’s easy to get carried away adding polynomial terms of every order or slicing the data into infinite steps. Lean on validation methods, keep an eye on potential multicollinearity, and interpret your results carefully. Above all, let your domain expertise guide the construction of these models so that they reflect real-world phenomena rather than chase ephemeral patterns.
Important Notice: FinancialAnalystGuide.com provides supplemental CFA study materials, including mock exams, sample exam questions, and other practice resources to aid your exam preparation. These resources are not affiliated with or endorsed by the CFA Institute. CFA® and Chartered Financial Analyst® are registered trademarks owned exclusively by CFA Institute. Our content is independent, and we do not guarantee exam success. CFA Institute does not endorse, promote, or warrant the accuracy or quality of our products.