Skip to content
LearnPY
scikit-learn

Classify with probabilities

Train a binary classifier, read predicted probabilities, and choose a threshold that matches the cost of mistakes.

14 min readbeginnerscikit-learn, numpy

By the end

  • Fit a logistic regression classifier with a pipeline.
  • Distinguish predicted labels from predicted probabilities.
  • Move the decision threshold deliberately.
  • Reason about class imbalance and error costs.
  • Interpret coefficients as log-odds changes.

Predict a category

Classification predicts a discrete label. In binary classification the target has two values, such as malignant versus benign. Accuracy is a starting point, but it can hide a model that ignores the smaller class.

Python

A binary target with named classes

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Fit logistic regression

LogisticRegression models the log-odds of the positive class with a linear combination of features. Standardize the features inside a pipeline so the coefficients and optimizer behave consistently.

Python

Fit on training rows and predict labels

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

max_iter=1000 gives the solver enough iterations to converge on this scaled dataset. If scikit-learn warns about convergence, increase the limit or check the feature scale rather than ignoring it.

Look behind the label

predict_proba returns one probability per class for every row. The columns follow model.classes_ order, so for classes [0, 1] the second column is the probability of class 1.

Python

Probabilities are per class and sum to one

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

The threshold is a decision

predict applies a 0.5 threshold to the positive-class probability. Lowering it catches more positives but also raises false positives. Raising it does the reverse. Choose the threshold from the cost of each mistake, not from habit.

Python

Changing the cutoff changes the decisions

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

The model's ranking can stay the same while the threshold changes. Metrics that use probabilities, such as ROC-AUC, measure ranking quality; metrics that use labels depend on the threshold.

Account for imbalance

When one class is much rarer, the optimizer can favor the majority. class_weight="balanced" gives the classes weights inversely proportional to their frequency. It changes the tradeoff, not the underlying information.

Python

Compare class weighting

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Read coefficients as log-odds

For a scaled feature, a positive coefficient increases the log-odds of the positive class when that feature rises by one standard deviation. Exponentiating the coefficient gives an odds multiplier, not a probability.

Python

Coefficients are additive on the log-odds scale

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Correlated features still split credit. A large coefficient is not proof that changing that feature causes the outcome.

Practice

Make it stick

Run the tests as often as you like. Hints and the solution are available when you need them.

Exercise

Fit a StandardScaler plus LogisticRegression(max_iter=1000) pipeline on the breast-cancer training rows. Store labels in predictions, class probabilities in probabilities, and test accuracy in accuracy.

Your solution

Ready
Loading editor…

Results

Run the tests to check your solution. The first run downloads Python; later runs reuse the session.

Quiz

Check your understanding

01What does predict_proba add beyond predict?
02What should guide the threshold choice?
03What does class_weight="balanced" change?

Lesson progress