Small language models (SLMs) struggle with complex reasoning because exploration is expensive under tight compute budgets. We introduce Semantic Diversity – Exploration–Exploitation (SD-E2), a reinforcement learning framework that makes exploration explicit by optimizing semantic diversity in generated reasoning trajectories. Using a frozen sentence-embedding model, SD-E2 assigns a diversity reward that captures (i) the coverage of semantically distinct solution strategies and (ii) their average pairwise dissimilarity in embedding space, rather than surface-form novelty. This diversity reward is combined with outcome correctness and solution efficiency in a -score–normalized multi-objective objective that stabilizes training. On GSM8K, SD-E2 surpasses the base Qwen2.5-3B-Instruct and strong GRPO baselines (GRPO-CFL and GRPO-CFEE) by +26.0, +6.0, and +1.5 percentage points, respectively, while discovering on average 9.8 semantically distinct strategies per question. These results indicate that rewarding semantic novelty yields a more compute-efficient exploration–exploitation signal for training reasoning-capable SLMs. By introducing cognitive adaptation (adjusting the reasoning process structure rather than per-token computation), SD-E2 offers a complementary path to efficiency gains in resource-constrained models.