Skip to content
LearnPY
scikit-learn

Reduce dimensions with PCA

Project many correlated features onto a few directions of variance, and read explained variance before trusting the picture.

12 min readbeginnerscikit-learn, numpy, matplotlib

By the end

  • Explain what principal components represent.
  • Scale features before applying PCA.
  • Read explained variance ratios.
  • Choose a component count from cumulative variance.
  • Use a projection for exploration, not as proof of classes.

Find directions of variance

PCA creates new features that are linear combinations of the originals. The first component points in the direction of greatest variance, the second is the next orthogonal direction, and so on. It is unsupervised: no target is involved.

Python

Four measured features, three species

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Scale before projecting

PCA is sensitive to feature units. A feature measured in thousands can dominate one measured in fractions. Put StandardScaler before PCA unless the features already share a meaningful scale.

Python

Four features become two components

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

The projected rows keep their row order. PCA changes the columns, not which observation each row represents.

Read explained variance

explained_variance_ratio_ reports the fraction of total variance captured by each component. The cumulative sum shows how many components are needed for a target amount of variance, such as 95%.

Python

How much variation each component carries

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Choose a component budget

Fit PCA without a component limit to inspect the full spectrum, then pick the smallest number that reaches your target. Keep the target tied to the task rather than automatically using 95%.

Python

Let the cumulative curve set the budget

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

More components preserve more information but give up the compression and visualization benefits. A smaller projection can also discard noise, but it can discard useful signal too.

Plot the projection, carefully

A two-dimensional projection is useful for seeing broad separation and outliers. Color by a known label only for evaluation; PCA did not use those labels to find the directions.

Python

A projection colored by species after the fact

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Practice

Make it stick

Run the tests as often as you like. Hints and the solution are available when you need them.

Exercise

Build a StandardScaler plus PCA(n_components=2) pipeline and fit_transform iris.data into projected. Store the two explained variance ratios in ratios and their cumulative sum in cumulative.

Your solution

Ready
Loading editor…

Results

Run the tests to check your solution. The first run downloads Python; later runs reuse the session.

Quiz

Check your understanding

01Does PCA use the target labels to choose components?
02Why scale features before PCA?
03What does a cumulative explained variance of 0.95 mean?

Lesson progress