Skip to content
LearnPY
scikit-learn

Search for better settings

Tune hyperparameters with cross-validated grid search, keep the test set untouched, and evaluate the winning pipeline once.

14 min readbeginnerscikit-learn, numpy

By the end

  • Explain the difference between parameters and hyperparameters.
  • Tune a pipeline with GridSearchCV.
  • Use the step__parameter naming convention.
  • Read best_params_, best_score_, and best_estimator_.
  • Keep the final test score separate from model selection.

Read the search results

best_params_ is the winning combination, best_score_ is its mean cross-validation score, and best_estimator_ is that pipeline refitted on all training rows. The trailing underscores mark attributes learned during fit.

Python

Inspect the whole search, not just the winner

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Score the winner once on test

After the search, evaluate best_estimator_ on the locked-away test rows. This is the number to report as an estimate of performance on unseen data. Do not go back and tune after seeing it.

Python

Keep the selection score and final score distinct

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

If the final test score is much worse than the cross-validation score, investigate distribution shift, a small test set, leakage, or too many candidate comparisons. Do not simply rerun the search until the gap looks nice.

When search itself needs validation

GridSearchCV chooses a winner using the same rows it scores. That winner's cross-validation score is optimistically biased. For a stricter estimate of the entire tuning procedure, use nested cross-validation: tune in an inner loop and score in an outer loop.

Python

Tune inside each outer fold

Ready
Loading editor…

Output

Press Run to execute this cell. The first run downloads Python; later runs reuse the session.

Practice

Make it stick

Run the tests as often as you like. Hints and the solution are available when you need them.

Quiz

Check your understanding

01Which value is a hyperparameter?
02How do you tune n_neighbors inside a pipeline step named classify?
03When should the test set be used during grid search?

Lesson progress