This work advances \textit{semantic representation learning} to render language representations or models more semantically and geometrically interpretable, and to enable localised, quasi-symbolic, compositional control through deliberate shaping of their latent space geometry. We pursue this goal within a VAE framework, exploring two complementary research directions: \textbf{(i)} Sentence-level learning and control — disentangling and manipulating specific semantic features in the latent space to guide sentence generation, with explanatory text serving as the testbed; and \textbf{(ii)} Reasoning-level learning and control — isolating and steering inference behaviours in the latent space to control NLI. In this direction, we focus on \textit{Explanatory NLI} tasks, in which two premises (explanations) are provided to infer a conclusion. Our objective is to move toward LMs whose internal semantic representations can be systematically interpreted, precisely shaped, and reliably directed.