Humans can categorize with only a few samples despite the numerous features. To mimic this ability, we propose a novel mixture of probabilistic principal component analyzers (mPPCA) model with dimension-reduced category representations, along with a theoretical analysis of rational dimensionality choices in categorization. Tests on the {\tt CIFAR-10H} natural image categorization dataset show that introducing a single principal component for each category effectively improves predictions of human categorization patterns. We further use mPPCA to account for human category generalization with very few samples. In our experiments with visual patterns of varying size and color, combining principal components and the hierarchical prior leads to significantly better predictions of human generalization within and beyond previously learned categories.