Vignette-Based Exercises and Solutions

Overview

Evaluating regression model fit and interpretation can feel like juggling a bunch of spinning plates—especially under tight exam conditions. You’ve got partial sums of squares, R², adjusted R², F-statistics, t-tests, and who knows what else thrown at you. In the CFA Level II exam, these concepts come alive through vignette-style item sets. That’s what we’ll explore here: how to handle data-rich scenarios, extract the important bits, and answer questions swiftly but accurately.

Below, we’ll walk through sample vignettes, show how to parse them, and practice what’s almost a ritual when analyzing any regression output: checking for overall model significance, verifying individual coefficient significance, and connecting statistical significance to real-world financial theories. We’ll also point out typical pitfalls—like mixing up the F-statistic with the t-statistic. I (remember, once, I spent ages computing the wrong ratio because I rushed through the data. Ugh, not fun.)

We’ll wrap up with best practices and a short quiz so you can test your knowledge. Let’s get started.

Understanding the Vignette Approach

CFA exam vignettes are practical scenarios meant to test more than just your memory. They assess your ability to think critically about financial data. In such item sets:

• You’ll receive a (sometimes lengthy) backstory: maybe an analyst wants to forecast next quarter’s stock returns, or a fund manager is measuring the effectiveness of a discount rate model.
• You get numerical outputs (coefficient estimates, standard errors, SSE, SSR, partial F-tests, etc.).
• You then see 4–6 multiple-choice questions that piggyback on that data, requiring integrated understanding of the context and the statistics.

Core Steps in Tackling Vignettes

Because these item sets can be dense, having a consistent process is the key:

1. Read the entire vignette quickly but carefully. Identify the main financial or economic question.
2. Pinpoint critical data: R², adjusted R², SSE (sum of squared errors), SSR (regression sum of squares), MSR (mean square regression), MSE (mean square error), and all relevant degrees of freedom.
3. Note the coefficient estimates and standard errors or p-values. If you need to compute t-statistics or partial F-tests, be sure to have the right formula.
4. Keep track of degrees of freedom for each test.
5. Interpret results in the context of the financial question posed.

Below is a simplified workflow diagram you might find helpful:

    flowchart LR
	   A["Read <br/>and Comprehend Vignette"] --> B["Identify <br/>Key Data"]
	   B --> C["Perform <br/>Calculations"]
	   C --> D["Interpret <br/>Results"]
	   D --> E["Address <br/>Exam Questions"]

Quick Refresh: Key Formulas and Concepts

Before we dive into examples, let’s recall a few key formulas in regression analysis that typically pop up in CFA vignettes:

1. R-Squared:
   The coefficient of determination, which is:

   $$ R^2 = \frac{\mathrm{SSR}}{\mathrm{SST}} $$ where SSR is the regression sum of squares and SST is the total sum of squares.

2. Adjusted R-Squared:
   Adjusted for the number of independent variables \( k \) and sample size \( n \): $$ \bar{R}^2 = 1 - \frac{\mathrm{SSE}/(n-k-1)}{\mathrm{SST}/(n-1)}. $$

3. F-Statistic (overall significance test of the regression):
   $$ F = \frac{\mathrm{MSR}}{\mathrm{MSE}} = \frac{\mathrm{SSR}/k}{\mathrm{SSE}/(n-k-1)}. $$

4. t-Statistic for an Individual Coefficient:
   $$ t = \frac{\widehat{\beta}_j - 0}{\mathrm{SE}(\widehat{\beta}_j)}, $$ often tested against the null hypothesis \(H_0: \beta_j = 0\).

5. Confidence Intervals:
   Typically, a 95% CI for \(\beta_j\) is: $$ \widehat{\beta}j \pm t{\alpha/2,, df} \times \mathrm{SE}(\widehat{\beta}_j). $$

Now let’s jump into some realistic item-set style examples.

Vignette Example 1: Equity Returns Forecast

Scenario Setup

Imagine an analyst, Marta, who is building a multiple regression model to forecast monthly equity returns for a technology exchange-traded fund (ETF). She uses the following independent variables:

1. Market Excess Return (difference between market returns and risk-free rate).
2. SMB Factor (Small minus Big).
3. HML Factor (High book-to-market minus Low book-to-market).

She’s collected 48 months of data (n = 48). A portion of her regression output is shown below:

Other summary data:

• SSR (Regression Sum of Squares) = 410.2
• SSE (Sum of Squared Errors) = 603.5
• Total Observations (n) = 48

Question 1: Evaluate whether the model is overall significant at the 5% level. They give you partial outputs, and you suspect you might need the F-statistic.

Key Steps to Solve:

1. Calculate the degrees of freedom for the regression \((k = 3)\).

2. Calculate degrees of freedom for error \((n - k - 1 = 48 - 3 - 1 = 44)\).

3. Compute MSR and MSE:
   

   $$ \mathrm{MSR} = \frac{\mathrm{SSR}}{k} = \frac{410.2}{3} = 136.73. $$
   
   
   $$ \mathrm{MSE} = \frac{\mathrm{SSE}}{n - k - 1} = \frac{603.5}{44} \approx 13.71. $$

4. Compute the F-statistic:
   

   $$ F = \frac{\mathrm{MSR}}{\mathrm{MSE}} = \frac{136.73}{13.71} \approx 9.98. $$

5. Compare with the critical value \(F_{0.05,3,44}\) or find the p-value for \(F\). Without a table, we approximate the p-value, but given that an F-value near 10 is generally pretty large, it’s likely (in typical exam circumstances) that p < 0.05. So the model is significant overall.

Interpretation: The regression collectively explains a significant amount of the variance in the ETF returns, justifying the use of these factor exposures.

Personal Note: Back when I first computed an F-statistic, I got the DF wrong. I used \(n-1\) for the error term instead of \(n-k-1\). Don’t do that. It leads to an artificially large (or small) F-value.

Question 1 Solution

The overall model is indeed significant at the 5% level. We reject \(H_0\): All \(\beta_j=0\).

Question 2: Are the Market Excess Return and SMB factors individually significant at 5%?

For these, we check:

• \(t_{\alpha=0.05,, df=44}\) is around 2.02.
• For the Market Excess Return:
  
  $$ t = \frac{1.201}{0.300} = 4.00 > 2.02 \quad \Longrightarrow \text{Significant.} $$
• For SMB:
  
  $$ t = \frac{0.640}{0.270} \approx 2.37 > 2.02 \quad \Longrightarrow \text{Significant.} $$
• For HML:
  
  $$ t = \frac{0.150}{0.200} = 0.75 < 2.02 \quad \Longrightarrow \text{Not significant.} $$

Thus, the Market Excess Return and SMB factors appear to have a genuine relationship with the ETF’s returns, while HML may not (at least within the data period used).

Vignette Example 2: Corporate Bond Yield Analysis

Overview

Let’s switch gears to a fixed-income scenario. Suppose Jerry, a research analyst at a bond fund, wants to model corporate bond yield spreads using two economic variables:

1. Unemployment Rate (UR)
2. Consumer Confidence Index (CCI)

He’s got 60 monthly observations (\(n=60\)). Here are partial outputs:

Additional details:

• SSE = 980.0
• SSR = 420.0

Question 1: Compute and interpret \(R^2\) and the adjusted \(R^2\).

• Total sum of squares (SST) = SSR + SSE = 420.0 + 980.0 = 1,400.0.
• \(R^2 = \frac{420.0}{1400.0} = 0.30\). So 30% of the variation in yield spreads is explained by these two factors.

Then adjusted \(R^2\) with \(k=2\) and \(n=60\):

$$ \bar{R}^2 = 1 - \frac{\mathrm{SSE}/(n-k-1)}{\mathrm{SST}/(n-1)} = 1 - \frac{980.0 / (60-2-1)}{1400.0 / (60-1)}. $$

Let’s do the math in steps:

So the adjusted \(R^2 \approx 28%\). The model explains roughly 30% of yield spread variation, dipping slightly to 28% when penalizing for two explanatory variables.

Best Practices for Vignette Questions

• Scan for the Focus: Some item sets emphasize overall significance (the F-test), others highlight individual coefficients. Identify what the question specifically wants.
• Organize the Data: Jot down SSR, SSE, n, k, df, and then systematically compute. Don’t do it in your head. In a rush, it’s easy to slip.
• Think Like a Portfolio Manager: If the variable is significant, how does that tie into investment decisions or risk management strategies?
• Check Sign Consistency: Even if a coefficient is significant, does it have the sign you’d expect? Could economic theory suggest a negative or positive coefficient? This deeper interpretation can show up in more advanced exam questions.
• Time Management: Don’t over-calculate. If you sense the F-value is going to be large (like in the 9–10 range) and your critical p-value is 0.05, you can quickly judge significance without performing super-precise calculations.

Common Pitfalls

• Overlooking the degrees of freedom.
• Mixing up SSE with SSR.
• Using the wrong test statistic (e.g., a t-test formula for a multiple-coefficient F-test).
• Not connecting the regression output to the real-world question posed in the vignette.

Python Snippet: Quick F-Statistic Calculation

Sometimes you might want to replicate or verify a quick calculation. Here’s a basic snippet that demonstrates computing the F-statistic from SSR and SSE:

 1import numpy as np
 2
 3SSR = 410.2
 4SSE = 603.5
 5n = 48
 6k = 3
 7
 8df_reg = k
 9df_resid = n - k - 1
10
11MSR = SSR / df_reg
12MSE = SSE / df_resid
13
14F_stat = MSR / MSE
15print("F-statistic:", F_stat)

When you run this (in an environment that understands Python), you’ll see something around 9.98, matching our earlier manual calculation.

Putting It All Together: An Integrated Vignette

Below is a short integrated vignette bringing in multiple elements—overall significance, partial significance, and a forecast.

Vignette Scenario

• Context: Rachel is building a model to predict next quarter’s commodity price changes. She uses data on inflation, industrial production growth (IPG), and the previous quarter’s commodity price change as independent variables. She has 50 observations, and the regression output is shown:

• SSE = 450
• SSR = 150

Questions

1. Overall Significance: At the 5% level, is the model significant?

   • Steps:
     • \(n = 50, k = 3\).
     • \( \mathrm{MSR} = 150 / 3 = 50\).
     • \( \mathrm{MSE} = 450 / (50-3-1) = 450 / 46 \approx 9.78\).
     • \(F = 50 / 9.78 \approx 5.12\).
     • Compare to \(F_{0.05, 3, 46}\). Typically, that’s in the neighborhood of 2.8, so 5.12 is definitely above it. The model is significant overall.

2. Check Individual Coefficients: Use the t-stat. If \(t_{0.05,46} \approx 2.014\):

   • Inflation: \(t = 0.25/0.12 \approx 2.08\) (significant)
   • IPG: \(t = 0.40/0.15 \approx 2.67\) (significant)
   • Lag(Commodity): \(t = 0.20/0.05 = 4.0\) (significant)
     All variables help explain commodity price changes in Rachel’s data set.

3. Forecast: If next quarter’s inflation forecast is 2%, IPG forecast is 1%, and the last quarter’s commodity price change was 3%, the model’s predicted change is:

   $$ \hat{y} = 0.10 + 0.25(2\%) + 0.40(1\%) + 0.20(3\%) = 0.10 + 0.005 + 0.004 + 0.006 = 0.115 \quad (i.e., 11.5\%). $$

   Note: We often represent inflation or IPG in decimals. If they gave inflation as 2.0 (and it was 2%), you’d adapt accordingly. Always be mindful how data is expressed in the vignette.

Exam Tips and Final Thoughts

1. Organize: Create small tables of SSR, SSE, \(n\), \(k\), and degrees of freedom. Keep a running list.
2. Interpretation: Merely computing an F-statistic or t-value is only half the battle. You must discuss whether it’s “significant” and what that means.
3. Connect: Tie significance back to portfolio or economic implications—often the final question in a vignette might ask, “Which factor best explains the changes in returns?” or “Should the manager rely on the model for next quarter’s forecast?”
4. Pitfalls: Watch for red herrings—irrelevant data that might confuse you.
5. Speed: Hone your mental math (or careful short-hand math) so you don’t get stuck computing.

In real exam conditions, it’s about synergy: your grasp of the content plus your question-reading discipline. You’d be surprised how many mistakes come from misreading the question prompt rather than from lack of knowledge.

References and Further Reading

• CFA Institute’s Learning Ecosystem: Make sure to check official practice vignettes.
• Past Level II Mock Exams: Great resource for seeing how item-set style questions on regression are framed.
• Chapter 2 of this volume (Basics of Multiple Regression and Underlying Assumptions) for a refresher on constructing the model.
• Chapter 3, earlier sections on ANOVA and coefficient testing, for in-depth definitions of SSE, SSR, MSE, and how to interpret them in a real exam setting.

Test Your Knowledge: Evaluating Regression Model Fit Vignette Exercises