7. Model Metrics – Classification

Let’s talk about a topic that often gets underestimated — classification metrics in machine learning. I know many of you are eager to dive into LLMs and the shiny new world of GenAI. But here’s the truth: without building a strong foundation in traditional ML, your understanding of advanced systems will always remain shallow.

So, stay with me — this is important, and honestly, quite powerful.

When you’re working on classification problems, choosing the right metric is critical. A good model isn’t just about accuracy — it’s about the right kind of correctness based on the problem you’re solving.

In this blog, I’ll walk you through the most commonly used metrics that cover over 90% of real-world classification use cases.

---

📌 1. Accuracy

Definition:
Accuracy is the ratio of correctly predicted observations to the total observations.

Formula:
Accuracy = (TP + TN) / (TP + TN + FP + FN)

Where:

• TP = True Positives
• TN = True Negatives
• FP = False Positives
• FN = False Negatives

Use when:

• Your dataset is balanced.
• You want a quick, high-level measure.

⚠️ Caveat:
Accuracy can be misleading when classes are imbalanced.

---

📌 2. Precision & Recall

Let’s break this down with basic definitions first:

• True Positive (TP): Correctly predicted positive cases
• True Negative (TN): Correctly predicted negative cases
• False Positive (FP): Incorrectly predicted positive cases (Type I error)
• False Negative (FN): Missed positive cases (Type II error)

---

Precision

Definition:
How many of the predicted positives are actually correct?

Formula:
Precision = TP / (TP + FP)

Use when:
False positives are costly — e.g., spam detection, fraud detection.

---

Recall (Sensitivity or TPR)

Definition:
How many actual positives were correctly predicted?

Formula:
Recall = TP / (TP + FN)

Use when:
Missing a positive is costly — e.g., cancer detection, fraud risk analysis.

---

📌 3. F1 Score

Definition:
The harmonic mean of precision and recall. It’s a balance when you care equally about precision and recall.

Formula:
F1 Score = 2 * (Precision * Recall) / (Precision + Recall)

Use when:

• You have an imbalanced dataset.
• You want to balance false positives and false negatives.

---

📌 4. ROC (Receiver Operating Characteristic) Curve

Definition:
Plots the True Positive Rate (Recall) against the False Positive Rate at various threshold levels.

It helps you visualize model performance across different thresholds.

Use when:
You want to understand trade-offs between sensitivity and specificity.

---

📌 5. AUC (Area Under the Curve)

Definition:
AUC measures the entire two-dimensional area under the ROC curve.

Interpretation:

• AUC = 0.5: No discrimination (random)
• AUC = 1.0: Perfect model

Use when:
You want a summary of model performance in ranking positive cases higher than negative.

---

📌 6. Confusion Matrix

Definition:
A matrix layout that lets you see the number of correct and incorrect predictions, broken down by each class.

	Predicted Positive	Predicted Negative
Actual Positive	True Positive (TP)	False Negative (FN)
Actual Negative	False Positive (FP)	True Negative (TN)

Use when:
You want a granular understanding of prediction types — especially useful for presentations and model debugging.

---

🎯 Final Word

Metric selection depends on your success criteria.

• Is it okay to have some false positives? → Use precision
• Is missing a positive a deal-breaker? → Use recall
• Do you want a balance? → Use F1 score
• Need a visual check? → Use ROC & AUC
• Want to debug? → Start with confusion matrix

Don’t blindly go with accuracy. It’s important, but in many real-world problems — especially with imbalanced datasets — it’s the least useful metric.