Skip to main content
Brave Programmer Logo

BraveProgrammer

BraveProgrammer

HomeProjectsBlogsCoursesLessonsAbout

Site footer

BraveProgrammer

Free coding courses, practical tutorials, and real projects from BraveProgrammer. Learn web development with React, Next.js, and TypeScript.

Navigation

  • Home
  • Projects
  • Blogs
  • Courses

Resources

  • About
  • Lessons
  • Teach on BraveProgrammer

© 2026 BraveProgrammer. All rights reserved.

Model Selection – Cr | Master Data Science with Python lesso | BraveProgrammer
  1. Courses
  2. /
  3. Master Data Science with Python

Lesson 53 of 60

·python

Model Selection – Cross‑Validation

20 minutes

Course progress88%

Cross‑Validation (CV)

Cross‑validation provides reliable estimates of model performance on unseen data.

k‑fold cross‑validation

  • Split data into k equal folds.
  • Train on k‑1 folds, validate on the held‑out fold.
  • Repeat k times, each fold becomes validation once.
  • Aggregate scores (mean, std).

Using scikit‑learn cross_val_score

 sklearn.model_selection  cross_val_score
 sklearn.ensemble  GradientBoostingClassifier

model = GradientBoostingClassifier(random_state=)
cv_scores = cross_val_score(model, X, y, cv=, scoring=)
(, cv_scores)
(, cv_scores.mean())
from
import
from
import
42
5
'accuracy'
print
'CV Accuracy scores:'
print
'Mean accuracy:'

Stratified k‑fold (preserves class distribution)

from sklearn.model_selection import StratifiedKFold
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
cv_scores = cross_val_score(model, X, y, cv=skf, scoring='f1')

Nested cross‑validation (hyperparameter tuning + performance estimate)

from sklearn.model_selection import GridSearchCV, cross_val_score

param_grid = {'n_estimators': [100, 200], 'learning_rate': [0.01, 0.1]}
inner_cv = StratifiedKFold(n_splits=3, shuffle=True, random_state=42)
outer_cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)

clf = GridSearchCV(GradientBoostingClassifier(random_state=42), param_grid, cv=inner_cv, scoring='roc_auc')
nested_score = cross_val_score(clf, X, y, cv=outer_cv, scoring='roc_auc')
print('Nested CV ROC‑AUC:', nested_score.mean())

TimeSeriesSplit (for temporal data)

from sklearn.model_selection import TimeSeriesSplit
tscv = TimeSeriesSplit(n_splits=5)
cv_scores = cross_val_score(model, X, y, cv=tscv, scoring='neg_mean_squared_error')

Best practices

  • Shuffle data before splitting unless time series.
  • Use stratification for classification.
  • Keep a final hold‑out test set untouched for the last evaluation.

Info

Cross‑validation can be computationally expensive; consider using fewer folds or smaller subsets for quick prototyping.
Make it stickPractice what you've learned1 quiz · 4 practical tests · Optional activities from Master Data Science with PythonBrowse activities
Previous lessonDimensionality Reduction – Principal Component Analysis (PCA)Continue to lesson 54Hyperparameter Tuning – Grid Search & Randomized SearchBack to Master Data Science with Python curriculum