Unsupervised Algorithms: PCA, k-Means, Hierarchical Clustering

Introduction

Let’s say you’ve got a humongous dataset—maybe it’s daily returns for hundreds of equities or volumes of client transactions—and you suspect there’s a hidden structure. You can’t see it directly, and you don’t have those neat little target labels telling you which category is which. That’s where unsupervised learning comes to the rescue. Unlike supervised learning, these algorithms do not predict a labeled outcome; they seek patterns or groupings in data without “knowing” the right answer in advance.

From your CFA Level II perspective, unsupervised approaches like Principal Component Analysis (PCA), k-Means Clustering, and Hierarchical Clustering can be powerful tools for factor identification, portfolio construction, client segmentation, and risk management. Below, we’ll walk through each of these techniques in detail, highlight their financial applications, and provide practical tips and best practices to guide you.

Why Unsupervised Techniques Matter in Finance

Earlier in this curriculum, you encountered supervised learning algorithms that predict outcomes (like a stock’s future return). In unsupervised learning, we lack that outcome variable. Instead, we look for underlying patterns or structures:

• PCA helps reduce dimensionality and identify hidden factors or exposures.
• k-Means groups data into clusters based on similarity.
• Hierarchical clustering uncovers nested relationships—think of it like a family tree of securities.

In a real-world financial setting, these methods: • Improve portfolio construction and diversification by revealing uncorrelated factors (PCA).
• Segment clients so that marketing or advisory services can be more targeted (k-Means).
• Spot credit risk or equity groupings at multiple levels of granularity (Hierarchical Clustering).

And, of course, these techniques pop up in the exam and in practical item sets all the time.

Principal Component Analysis (PCA)

Rationale and Core Concepts

Maybe you’ve run across a scenario in which you have 50 different economic indicators or 100 daily returns from a large equity portfolio. There’s simply too much data to analyze directly. PCA addresses this challenge by extracting a smaller set of uncorrelated features from a large dataset, capturing the most variance possible in each successive “principal component.”

Where does PCA come from, mathematically? Let X be your data matrix—say each row is an observation (like daily returns on a set of assets), and each column is a feature (like returns for a specific asset). PCA centers the data (subtracting means of each column), then computes the covariance matrix:

PCA finds the eigenvectors and eigenvalues of \(\Sigma\). Each eigenvector becomes a “principal component,” and the associated eigenvalue measures how much variance is captured by that component. If your first principal component accounts for 40% of the total variance in the data, that’s a big chunk of the story right there.

Practical Steps to Perform PCA

1. Standardize or normalize your data (especially if variables have different scales).
2. Compute the covariance matrix Σ (or sometimes the correlation matrix in finance, if scales differ significantly).
3. Calculate eigenvalues and eigenvectors.
4. Select the top \(k\) principal components that explain a sufficiently large proportion (like 80% to 90%) of the total variance.
5. Project the original data onto these \(k\) components to reduce dimensionality.

Financial Example: Detecting Latent Risk Factors

In portfolio risk management, PCA can reveal “factors” that drive correlated movements among assets. For instance, the first principal component might be a broad market factor, the second might be an industry-specific factor, and so on. Once you identify these factors, you can measure your portfolio’s exposure to each one. That clarity helps you make better asset allocation decisions and manage your risk more intentionally.

Python Snippet (Optional Illustration)

Below is a simple snippet that demonstrates PCA using Python’s scikit-learn. Although you probably won’t write code during the exam, having an intuitive sense of how it’s done can help you interpret item-set outputs.

 1import numpy as np
 2from sklearn.decomposition import PCA
 3
 4# e.g., each row is a different day, each column a different stock return
 5data = np.random.randn(1000, 10)  
 6
 7pca = PCA(n_components=3)
 8pca.fit(data)
 9
10print("Principal components:", pca.components_)
11print("Explained variance ratio:", pca.explained_variance_ratio_)

k-Means Clustering

Intuitive Overview

Picture the dreaded “client segmentation project.” You have client data—assets under management, transaction frequency, risk appetite, everything. You want to split them into distinct clusters so you can tailor your financial advice. With k-Means, you pick the number of clusters k, the algorithm partitions the data, and each cluster has a centroid representing its “center.”

The Algorithmic Steps

k-Means tries to minimize the sum of squared distances of each point to its cluster centroid:

1. Choose k (the desired number of clusters).
2. Randomly initialize k centroids.
3. Assign each data point to the nearest centroid.
4. Recalculate centroids as the mean of all points in each cluster.
5. Repeat steps 3–4 until cluster assignments stabilize or a maximum number of iterations is reached.

Below is a simple visualization in Mermaid, showing how the assignments and centroid updates loop:

    flowchart TB
	    A["Initialize k Centroids"]
	    B["Assign Points to Their <br/> Nearest Centroid"]
	    C["Recalculate Centroids"]
	    D["Convergence? <br/> If No, Repeat"]
	    E["Final Cluster Assignment"]
	
	    A --> B
	    B --> C
	    C --> D
	    D -->|No| B
	    D -->|Yes| E

Financial Use Case: Grouping Stocks by Similar Profiles

Let’s say you want to group securities by their valuation metrics (P/E, P/B), volatility, or returns. By specifying k, you can create segments of “Value Stocks,” “Growth Stocks,” “Stable Dividend Payers,” etc. The resulting clusters help you pick a diversified set or analyze your portfolio’s cluster exposures.

But watch out: you need to preselect k. If you pick too few clusters, you lose detail. Too many, and you might end up with oversegmentation. One common approach is the “elbow method,” which plots the within-cluster sum of squares against different choices of k. You look for the elbow, which is often a good trade-off between cluster separation and interpretability.

Pitfalls and Best Practices

• Initialization sensitivity: k-Means can give different clusters if you start with different random seeds.
• Scaling: Normalize or standardize inputs so one feature doesn’t dominate the distance metric.
• Outliers: A single outlier might shift your centroids significantly. Sometimes it helps to remove outliers or use robust distance metrics.

Hierarchical Clustering

The Hierarchical Approach

What if you have no clue how many clusters you want? Hierarchical clustering solves that. It builds a tree-like structure (a dendrogram) showing how data points cluster together at various similarity thresholds.

• Agglomerative (“bottom-up”) approach: Start with every single observation as its own cluster, then iteratively merge the two closest clusters until only one remains.
• Divisive (“top-down”) approach: Start with all points in one cluster, then split clusters recursively.

Dendrogram Visualization

A dendrogram is a visual representation in which the y-axis might measure the distance (or dissimilarity), and the x-axis shows each data point or cluster. A high-level diagram might look like this:

    graph LR
	    A["Observations 1, 2, 3, ..., N"]
	    B["Merge Closest Observations <br/> Into Clusters"]
	    C["Form Next-Level Clusters from <br/> Similar Groups"]
	    D["Eventually All Observations <br/> Merge Into One Cluster"]
	
	    A --> B
	    B --> C
	    C --> D

You can “cut” the dendrogram at any level to decide how many clusters you want, so there’s no need to prespecify k as you do with k-Means.

Real-World Example: Multi-Level Risk Grouping

In credit analysis, you might use hierarchical clustering to group loans by risk. At a coarse level, you see two or three major clusters (say, “Investment Grade,” “Speculative,” and “High Yield”). If you cut the dendrogram further down, you can differentiate subgroups, like “Technology High Yield” vs. “Emerging Markets High Yield.” This nested structure helps you refine your risk strategy.

Key Points for Exam and Practice

• Linkage Criteria: How do we measure the “distance” between clusters? Single linkage uses the distance between the closest points in each cluster, while complete linkage uses the furthest points’ distance. In finance, “average linkage” can be more robust, balancing extremes.
• Interpretability: The dendrogram makes the analysis visually interpretable. You can see which clusters merge at which distance thresholds.

Comparing the Three Methods

Exam Tips and Pitfalls

So you’re heading into the CFA exam, feeling good about your mastery of unsupervised learning. Here are a few final tips:

• Know the difference between supervised vs. unsupervised algorithms: exam prompts might test your conceptual understanding.
• PCA in the exam context often shows up as factor extraction, so watch for item sets that mention “variance explained” and “loadings.”
• k-Means might appear with questions about how many clusters to form or how to interpret cluster centroids.
• Hierarchical clustering might be tested with a dendrogram. You might need to interpret where the cluster cuts happen or how to choose a linkage criterion.
• Practice reading residual charts or cluster assignment tables quickly; time is precious in the item-set format.
• Watch for outliers: a sneaky exam question might mention an extreme data point that skews your cluster centroid.

Over the years, I’ve found it helpful to keep a mental checklist: 1) Is the data standardized? 2) Have I considered outliers? 3) What’s the right number of principal components or clusters?

References

• Jolliffe, I. (2002). “Principal Component Analysis.” Springer.
• Tan, P., Steinbach, M., & Kumar, V. (2019). “Introduction to Data Mining.”
• Python Data Science Handbook chapters on clustering: https://jakevdp.github.io/PythonDataScienceHandbook/

Test Your Knowledge: Unsupervised Learning in Finance