The common occurrence of occlusion-induced incompleteness in point clouds has made point cloud completion (PCC) a highly-concerned task in the field of geometric processing. Existing PCC methods typically produce complete point clouds from partial point clouds in a coarse-to-fine paradigm, with the coarse stage generating entire shapes and the fine stage improving texture details. Though diffusion models have demonstrated effectiveness in the coarse stage, the fine stage still faces challenges in producing high-fidelity results due to the ill-posed nature of PCC. The intrinsic contextual information for texture details in partial point clouds is the key to solving the challenge. In this paper, we propose a high-fidelity PCC method that digs into both short and long-range contextual information from the partial point cloud in the fine stage. Specifically, after generating the coarse point cloud via a diffusion-based coarse generator, a mixed sampling module is developed to introduce short-range contextual information from partial point clouds into the fine stage. In this module, a surface freezing mechanism is presented to safeguard points from noise-free partial point clouds against disruption. As for the long-range contextual information, a similarity modeling module is designed to derive similarity with rigid transformation invariance between points, so as to conduct effective matching of geometric manifold features globally. In this way, the high-quality components present in the partial point cloud can serve as valuable references for refining the coarse point cloud with high fidelity. Extensive experiments covering synthetic and real-scanned data have demonstrated the superiority of the proposed method over SOTA competitors. $\textbf{Our code will be available after acceptance.}$