Compare MSE, MAE, and Huber loss for regression. When does each break down and how do you choose?
formulate your answer, then —
tldr
MSE is optimal for Gaussian noise but sensitive to outliers. MAE is robust but has constant gradients and predicts the median. Huber is quadratic near zero and linear for large errors — the default robust regression loss. Quantile loss extends MAE to predict arbitrary quantiles, useful when overestimation and underestimation have different costs.
follow-up
- Why does MSE predict the conditional mean while MAE predicts the conditional median?
- How would you tune the δ parameter in Huber loss?
- You are predicting ad revenue per click. Clicks occasionally produce fraudulent high-value events. Which loss would you use and why?