The existence of multiple, equally accurate models for a given predictive task—the Rashomon set—leads to predictive multiplicity, where models achieve similar global accuracy but diverge in their individual predictions. This inconsistency undermines trust in high-stakes applications, where reliable decision-making at the individual level is critical. Existing reconciliation methods attempt to resolve this issue by enforcing global agreement across models, but often fall short in ensuring consistency for specific instances. We first introduce Rocile (Robust Reconciliation), a model-agnostic framework that systematically reduces ensemble disagreement by iteratively adjusting model predictions through a momentum-based batching procedure. Rocile guarantees convergence to a globally consistent ensemble, yet, like other global methods, it overlooks localized inconsistencies that impact individual predictions. To address this limitation, we propose AdaRocile, an extension of Rocile that incorporates local calibration into the reconciliation process. For each test instance, AdaRocile identifies a context-sensitive neighborhood using an adaptive nearest-neighbor strategy, computes empirical correction terms for each model based on residuals in the local neighborhood, and applies Rocile to reconcile the locally adjusted predictions into a globally coherent output. The reconciled predictions are then distilled into a single, transparent decision rule for real-world deployment. Empirical results on high-stakes benchmarks, such as COMPAS and Adult, show that AdaRocile significantly improves the accuracy-reliability trade-off, reducing local calibration error by up to 27.1% over the global-only reconciliation, while driving key multiplicity metrics (variance, ambiguity, discrepancy, and disagreement rate) to near zero, maintaining global performance on key benchmarks. AdaRocile delivers both interpretability and individual-level reliability, offering a scalable, practical pipeline for building trustworthy and aligned AI systems.