Enabling object detectors to recognize out-of-distribution (OOD) objects is essential for building reliable AI systems. A primary obstacle stems from the fact that models lack supervision signals from unknown data, which can produce overconfident predictions on OOD objects. Despite previous progress that estimates OOD uncertainty based on the detection model and in-distribution (ID) samples, we explore using pre-trained vision-language representations for object-level OOD detection. We first discuss the limitations of applying image-level CLIP-based OOD detection methods to object-level scenarios. Building upon these insights, we propose RUNA, a novel framework that leverages a dual encoder architecture to capture rich contextual information and employs a regional uncertainty alignment mechanism to distinguish ID from OOD objects effectively. We introduce a few-shot fine-tuning approach that focuses on aligning region-level semantic representations to enhance further the model's ability to discriminate between similar objects. Our experiments demonstrate that RUNA significantly outperforms state-of-the-art methods in object-level OOD detection, particularly in challenging scenarios with diverse and complex object instances.