Model Evaluation Metrics for Classification

Model Evaluation – Classification Metrics

Choosing the right metric depends on the problem context (balanced vs. imbalanced, cost of errors).

Confusion matrix

	Predicted Positive	Predicted Negative
Actual Positive	TP	FN
Actual Negative	FP	TN

Primary metrics

from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, confusion_matrix, roc_auc_score, roc_curve

accuracy = accuracy_score(y_test, y_pred)
precision = precision_score(y_test, y_pred)
recall = recall_score(y_test, y_pred)
f1 = f1_score(y_test, y_pred)
cm = confusion_matrix(y_test, y_pred)

print('Accuracy:', accuracy)
print('Precision:', precision)
print('Recall:', recall)
print('F1:', f1)
print('Confusion Matrix:\n', cm)

ROC curve & AUC (binary)

y_proba = log_reg.predict_proba(X_test)[:,1]
fpr, tpr, thresholds = roc_curve(y_test, y_proba)
auc = roc_auc_score(y_test, y_proba)

plt.plot(fpr, tpr, label=f'AUC = {auc:.2f}')
plt.plot([0,1], [0,1], 'k--')
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC Curve')
plt.legend()
plt.show()

Precision‑Recall curve (useful for imbalanced data)

from sklearn.metrics import precision_recall_curve
prec, rec, thr = precision_recall_curve(y_test, y_proba)
plt.plot(rec, prec)
plt.xlabel('Recall')
plt.ylabel('Precision')
plt.title('Precision‑Recall Curve')
plt.show()

Cross‑validation for robust estimates

from sklearn.model_selection import cross_val_score
scores = cross_val_score(rf, X, y, cv=5, scoring='f1')
print('CV F1 scores:', scores)
print('Mean F1:', scores.mean())

Choosing a metric

• Accuracy: good for balanced classes.
• Precision: when false positives are costly.
• Recall: when false negatives are costly.
• F1: harmonic mean of precision and recall.
• AUC‑ROC: overall ranking capability.