14. Basic Machine Learning#
Put basic machine learning intro here.
Define regression model versus classification model. Define parametric model versus nonparametric model. Define supervised learning versus unsupervised learning.
Introduce the paradigm of cross-validation.
The definitions of machine learning, statistical learning, and artificial intelligence overlap in most cases. And in some contexts they are indistinguishable.
Machine learning, statistical learning, and artificial intelligence is mostly focused on predictive models \(\hat{y}=f(x|\theta)\) and tuning or estimating the parameters \(\theta\) to minimize some definition of total error in the predictions for \(\hat{y}\).
Highly nonlinear models
Cross-validation
Exotic loss functions
Super robust minimizers (variants of stochastic gradient descent)
Machine learning could have an equally appropriate and nondescriptive name of nonlinear regression modeling. On predictive accuracy, machine learning models outperform structural models and top regression models. However, this accuracy often comes at the cost of interpretability. The estimated parameters in structural models and regression models often have clear interpretations. On the other hand, it is nearly impossible to make a robust claim about the effect of an explanatory variable on a dependent variable in a neural net model.
Recent advances by Athey and others have re-established the elements of interpretation, marginal effects, and causal inference to machine learning models. [include citations here.]
14.1. Footnotes#
The footnotes from this chapter.