We propose a method for comparing density forecasts that is based on weighted versions of the continuous ranked probability score. The weighting emphasizes regions of interest, such as the tails or the center of a variable’s range, while retaining propriety, as opposed to a recently developed weighted likelihood ratio test, which can be hedged. Threshold and quantile based decompositions of the continuous ranked probability score can be illustrated graphically and prompt insights into the strengths and deficiencies of a forecasting method. We illustrate the use of the test and graphical tools in case studies on the Bank of England’s density forecasts of quarterly inflation rates in the United Kingdom, and probabilistic predictions of wind resources in the Pacific Northwest.
KEY WORDS: Continuous ranked probability score; Predictive ability testing; Probabilistic forecast; Proper scoring rule; Quantile; Weighted likelihood ratio test