MLE from scratch #2 – Bias, variance, and why training accuracy lies

After building a baseline model, the next failure many teams encounter is not low accuracy, but false confidence. Training metrics look strong. Loss decreases smoothly. Models appear stable. Yet performance degrades in production, generalization fails, and iteration becomes reactive. In many cases, the issue is not the algorithm itself, but a misunderstanding of how model complexity, data, and evaluation interact. This is where the bias–variance framework becomes essential. Not as a theoretical decomposition, but as a practical lens for interpreting error behavior and deciding what kind of change is actually justified. ...

December 26, 2025

MLE from scratch #1 - Linear regression beyond the textbook

In the current AI landscape, classical machine learning models are often considered obsolete. Large-scale deep learning and foundation models dominate most discussions, driven by abundant compute, massive datasets, and rapidly evolving tooling. In many teams, reaching for a large model has become the default response, sometimes before the problem itself is fully understood. What is discussed far less is the cost of this default. Infrastructure expenses grow quickly, data requirements escalate, and system complexity increases in ways that are difficult to reason about. In practice, many production failures attributed to model limitations are symptoms of deeper issues, weak signals, noisy or poorly curated data, and objectives that were never clearly defined. ...

December 20, 2025