Overfitting

Overfitting is a phenomenon in Machine Learning in which a Statistical Model or Neural Network fits Training Data too closely, capturing noise or chance patterns that do not generalize to unseen data from the same underlying Probability Distribution. It commonly manifests as low Training Error but substantially higher Generalization Error, which can be estimated using appropriately held-out Validation Data or Test Data. Risk factors include high Model Complexity relative to the amount and quality of available data, insufficient or unrepresentative training examples, noise, and excessive training. Distribution Shift can also impair Generalization, but poor performance caused by a distribution mismatch does not by itself establish overfitting. Mitigation techniques include Regularization, such as L1 Regularization and L2 Regularization, as well as Dropout and Early Stopping; Cross-Validation helps assess generalization and guide Model Selection rather than directly preventing overfitting. In Deep Learning, larger, representative datasets and Data Augmentation can also reduce the risk, but neither guarantees that a model avoids spurious patterns or generalizes well during Inference.