Settings chosen outside fit
A model parameter is learned from training data, such as a linear regression coefficient. A hyperparameter is chosen before fitting, such as the number of neighbors in KNN or the regularization strength in Ridge. Grid search tries combinations and uses cross-validation to compare them.
Python
A baseline with one hand-picked setting
Output
Press Run to execute this cell. The first run downloads Python; later runs reuse the session.
Search a grid with cross-validation
GridSearchCV evaluates every candidate combination with cross-validation. With a pipeline, prefix a step name and two underscores to the parameter name: classify__n_neighbors reaches the classifier inside the pipeline.
Python
Five candidates, each scored five times
Output
Press Run to execute this cell. The first run downloads Python; later runs reuse the session.
Five candidates with cv=5 means 25 fits, not five. A larger grid multiplies the cost. Start with a small, meaningful range before adding dimensions.
Read the search results
best_params_ is the winning combination, best_score_ is its mean cross-validation score, and best_estimator_ is that pipeline refitted on all training rows. The trailing underscores mark attributes learned during fit.
Python
Inspect the whole search, not just the winner
Output
Press Run to execute this cell. The first run downloads Python; later runs reuse the session.
Score the winner once on test
After the search, evaluate best_estimator_ on the locked-away test rows. This is the number to report as an estimate of performance on unseen data. Do not go back and tune after seeing it.
Python
Keep the selection score and final score distinct
Output
Press Run to execute this cell. The first run downloads Python; later runs reuse the session.
If the final test score is much worse than the cross-validation score, investigate distribution shift, a small test set, leakage, or too many candidate comparisons. Do not simply rerun the search until the gap looks nice.
When search itself needs validation
GridSearchCV chooses a winner using the same rows it scores. That winner's cross-validation score is optimistically biased. For a stricter estimate of the entire tuning procedure, use nested cross-validation: tune in an inner loop and score in an outer loop.
Python
Tune inside each outer fold
Output
Press Run to execute this cell. The first run downloads Python; later runs reuse the session.
Practice
Make it stick
Run the tests as often as you like. Hints and the solution are available when you need them.
Exercise
Using the provided split, build a StandardScaler plus KNeighborsClassifier pipeline and tune classify__n_neighbors over [1, 3, 5, 7] with GridSearchCV(cv=5, scoring="accuracy"). Store the fitted search in search, its winning parameters in best_params, its mean CV score in best_score, and its test accuracy in test_accuracy.
Your solution
ReadyResults
Run the tests to check your solution. The first run downloads Python; later runs reuse the session.
Quiz
Check your understanding
Lesson progress