A model-based discriminant analysis method that includes variable selection is presented. The discriminant analysis model is fitted in a semi-supervised manner using both labeled and unlabeled data. The method is shown to give excellent classification performance on several high-dimensional multiclass datasets with more variables than observations. The variables selected by the proposed method provide information about which variables are meaningful for classification purposes. A headlong search strategy for variable selection is shown to be efficient in terms of computation and achieves excellent classification performance. In applications to several food classification datasets, our proposed method outperformed default implementations of Random Forests, AdaBoost and Bayesian Multinomial Regression by substantial margins.
Keywords: Headlong search, model-based discriminant analysis, normal mixture models, semi-supervised learning, updating classification rules, variable selection.