Compare MSE, MAE, and Huber loss for regression. What does each optimize for, and how do outliers affect each?
formulate your answer, then —
tldr
MSE optimizes the conditional mean, amplifies outliers, smooth gradient. MAE optimizes the conditional median, robust to outliers, constant gradient (noisy). Huber combines both: quadratic near zero (smooth), linear for large errors (robust). Pick Huber as default for regression, MAE when labels have heavy tails, MSE when Gaussian errors and you want to penalize large deviations.
follow-up
- How would you pick the δ hyperparameter in Huber loss? Is there a principled way?
- Your target variable is log-normally distributed (house prices). Would you log-transform targets and use MSE, or use MAE on raw prices? What's the trade-off?
- What loss function would you use for quantile regression — predicting the 90th percentile of delivery time instead of the mean?