Skip to content
LearnPY
scikit-learn

Predict a number with regression

Fit a continuous target, read coefficients carefully, and evaluate predictions in the target's own units.

14 min readbeginnerscikit-learn, numpy, matplotlib

By the end

  • Fit and predict a continuous target with LinearRegression.
  • Interpret coefficients when features are scaled.
  • Measure MAE, RMSE, and R² on held-out rows.
  • Inspect residuals for structure the metrics hide.
  • Use Ridge regularization when coefficients are unstable.

Start from the target

Regression predicts a continuous quantity: a price, a duration, a demand count, or a disease-progression score. The target determines the metric. An error of 50 means something different for dollars than for days, so always report metrics in the target's units.

Python

A small regression dataset with a continuous target

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Fit a regression pipeline

LinearRegression finds coefficients that minimize squared error on the training rows. A pipeline keeps any feature preparation with the model, so the same learned transformation is used at prediction time.

Python

Fit on training rows, predict held-out rows

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Calling model.score on a regressor returns RA� by default. That is useful, but it is not the whole story. A model can have a respectable RA� and still make errors that are too large for the decision at hand.

Bug hunt

This mean absolute error implementation forgot the absolute value. Fix it so positive and negative misses cannot cancel.

Your solution

Ready
Loading editor…

Results

Run the tests to check your solution. The first run downloads Python; later runs reuse the session.

Read coefficients in context

For a scaled feature, a coefficient estimates the target change for a one-standard-deviation increase in that feature, holding the others fixed. Positive and negative signs describe association, not proof of causation.

Python

The largest coefficients in absolute value

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Measure error in target units

Mean absolute error is the average absolute miss and is easy to explain. Root mean squared error punishes large misses more strongly. R² compares the model's squared error with the squared error of predicting the training mean; 1 is perfect, 0 is that baseline, and values can be negative on held-out data.

Python

Three views of the same regression errors

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

  • MAE is robust and has the same units as the target.
  • RMSE grows faster when a few predictions are very wrong.
  • R² is scale-free but can hide the size of the actual error.

Plot what the metrics miss

A residual is actual minus predicted. Plot residuals against predictions. A random cloud around zero is reassuring; a curve, a funnel, or a cluster suggests the linear model is missing structure.

Python

Look for a pattern, not just a score

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Stabilize with Ridge

Ridge adds a penalty on the size of the coefficients. A larger alpha shrinks them toward zero. This can improve held-out performance when features are correlated or the sample is small, but it also introduces bias.

Python

Compare regularization strengths on the same split

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Do not choose alpha by peeking repeatedly at the test score. Use cross-validation on the training rows, then evaluate the chosen model once on the test set.

Practice

Make it stick

Run the tests as often as you like. Hints and the solution are available when you need them.

Exercise

Fit a StandardScaler plus LinearRegression pipeline on the diabetes training rows. Predict X_test, then store mean absolute error in mae, root mean squared error in rmse, and R² in r2.

Your solution

Ready
Loading editor…

Results

Run the tests to check your solution. The first run downloads Python; later runs reuse the session.

Quiz

Check your understanding

01What does RMSE emphasize compared with MAE?
02What does an R² of 0 mean on held-out data?
03What happens as Ridge alpha increases?

Lesson progress