CART Algorithm 🌲🏷️

CART (Classification and Regression Trees) is one of the most famous algorithms in data science. As the name suggests, it is a "Double Threat"—it can predict categories (Classification) AND numbers (Regression).

1. Binary Splitting (The Great Divider)

Unlike CHAID, CART is a Binary Tree. This means every time it asks a question, it only allows two possible answers.

• The "Fork in the Road": Every node splits into exactly two child nodes (e.g., "Is Income > $50k?" -> Yes vs. No).
• Exhaustive Search: CART looks at every possible place it could "Cut" the data for every single variable and picks the one that splits the labels most clearly.
• Ease of Deployment: Because the logic is Always "Yes/No," CART models are incredibly easy to convert into simple "If-Then" rules for computer code.
• Handling All Data: It is "Data Neutral"—it can use both numbers (Age) and labels (Gender) effortlessly in the same tree.

2. Classification vs. Regression

3. How it decides to split? (The Pursuit of Purity)

For classification, CART uses the Gini Index to measure "Impurity." For regression, it uses Variance Reduction.

• Gini Impurity (Chaos Measure): If a group has 50% "Buyers" and 50% "Non-Buyers," it is "Impure" (Gini = 0.5). CART wants to split this so that one group is 99% Buyers (Pure).
• Recursive Partitioning: The algorithm keeps splitting and re-splitting the groups until it either runs out of data or reaches a "Pure" result (Gini = 0).
• Feature Selection: It automatically identifies which variables are the most "Powerful" by placing them at the very top of the tree.
• Non-Parametric Nature: It doesn't care if your data follows a "Normal Distribution." It just looks at the raw numbers to find a split path.

4. Pruning: The Art of Trimming

A tree that is too big "Overfits," meaning it learns the specific names/dates of your old data instead of the general pattern. CART solves this using Pruning.

• Cost-Complexity Pruning: A mathematical trade-off where the algorithm "Cuts off" branches that add too much complexity without adding enough accuracy.
• The Healthy Tree: Pruning ensures that the final model is small enough for a human to understand but powerful enough to work on brand-new data.
• Validation Testing: The algorithm tests different versions of the "Trimmed" tree on a small slice of hidden data to find the "Perfect Size."
• Rule Simplification: It combines smaller, weak branches back into their parent nodes to reduce the "Noise" in the decision process.

Summary

• CART creates binary decision trees (2 branches only).
• It handles both Labels and Numbers.
• It uses Gini Impurity to find the best split.
• Pruning is used to keep the tree small and efficient.

Quiz Time! 🎯