Hyperparameter is used in machine learning, where it refers, loosely speaking, to user-set parameters, and in Bayesian statistics, to refer to parameters of the prior distribution.
In a standard linear regression, the predictor coefficients that characterize the relationship between the predictors and an outcome variable are model parameters that are estimated from the data (via least squares). A machine learning model can also have parameters that are “learned” from the data, e.g. node weights in a neural net.
A machine learning algorithm will also have hyperparameters that are set by the user – how many layers to have in a neural net, for example, or how many levels to have in a decision tree. (Actually, this distinction is a bit artificial – often hyperparameter adjustments are embedded in an iterative cross-validation process to optimize them – but you could still imagine setting them by hand in a way that you could not for the core machine learning algorithm.)
In Bayesian statistics, the standard statistical modeling process is integrated with the estimation and specification of a “prior” distribution to incorporate more information about the problem than is provided by the sample of data at hand. Hyperparameters are the parameters of that prior distribution.
In a standard linear regression, the predictor coefficients that characterize the relationship between the predictors and an outcome variable are model parameters that are estimated from the data (via least squares). A machine learning model can also have parameters that are “learned” from the data, e.g. node weights in a neural net.