A well-known inherent ambiguity in factor models is that factors and factor loadings can only be identified up to an orthogonal rotation. In confirmatory factor analysis (CFA), different options exist for placing constraints aimed at ensuring unique identification of the model parameters. For simple CFA structures, when each observed variable loads on exactly one factor, different sets of constraints are equivalent with respect to model fit. However, this may no longer be the case when some variables are permitted to load on more than one factor. Following a Bayesian approach to factor analysis, we illustrate that naive implementation of rotational constraints can be problematic for Bayesian inference. Dealing with rotational invariance by constraining some loadings to be one or positive may result in nontrivial multimodality in the likelihood and in mode-switching behavior with Markov chain Monte Carlo samplers. We present a simple approach for dealing with rotational invariance in Bayesian confirmatory factor analysis. We demonstrate our approach on simulated data and further illustrate it on a classic bifactor data set.
Keywords: Identifiability constraints, label-switching, Markov chain Monte Carlo, reflection, relabeling, rotational invariance.