In predictive modeling, ensemble methods refer to the practice of taking multiple models and averaging their predictions. In the case of classification models, the average can be that of a probability score attached to the classification. Models can differ with respect to algorithms used (e.g. neural net, logistic regression), settings used to configure algorithms (e.g. number of hidden layers in a neural net), variables used, and case weights assigned. Ensemble predictions often outperform all of the constituent models that make up the ensemble.
