Phenotypic drug discovery has attracted widespread attention for its potential in identifying bioactive molecules. Transcriptomic profiling provides a comprehensive reflection of phenotypic changes in cellular responses to external perturbations. In this paper, we propose XTransferCDR, a novel generative framework designed for feature decoupling and transferable representation learning across domains. Given a pair of perturbed expression profiles, our approach decouples the perturbation representations from basal states through domain separation encoders, and then cross transfer them in the latent space. Next, the transferred representations are subsequently used to reconstruct the corresponding perturbed expression profiles via a shared decoder. Such cross-transfer constraint effectively promotes the learning of transferable drug perturbation representations. We conducted extensive evaluations of our model on multiple datasets, including the single-cell transcriptional responses to drugs, and both single and combinatorial genetic perturbations. The experimental results show that XTransferCDR achieved better performance than current state-of-the-art methods, showcasing its potential for advancing phenotypic drug discovery. Our code are publicly available at https://anonymous.4open.science/r/XTransferCDR-34FC.