Partial Domain Adaptation (PDA) aims to generalize a classification model learned on a labeled source domain to an unlabeled target domain, where the target label space is a subset of the source label space. In PDA tasks, existing methods typically achieve transferability through distribution alignment in a statistical framework, and discriminability through geometric modeling. These two aspects are often treated as separate frameworks, which severs the intrinsic connection between them. To bridge this gap, we propose a unified framework termed Geometry-aware Conditional Alignment (GCA), which is derived from theoretical insights of MCRR. GCA collaboratively achieves conditional alignment and orthogonal discriminability in a unified framework, making the learned features more interpretable in both statistical and geometric aspects. Extensive experiments on four benchmark datasets are conducted to demonstrate the effectiveness of GCA.