Deep learning have achieved remarkable success in image recognition, yet their inherent opacity poses challenges for deployment in critical domains. Concept-based interpretations aim to address this by explaining model reasoning through human-understandable concepts. However, existing post-hoc methods and ante-hoc concept bottleneck models (CBMs), suffer from limitations such as unreliable concept relevance, non-visual or labor-intensive concept definitions, and model or data-agnostic assumptions. This paper introduces $\textbf{P}$ost-hoc $\textbf{C}$oncept $\textbf{B}$ottleneck $\textbf{M}$odel via $\textbf{Re}$presentation $\textbf{D}$ecomposition ($\textbf{PCBM-ReD}$), a novel pipeline that synergizes the strengths of both paradigms. PCBM-ReD automatically extracts visual concepts from a pre-trained encoder, employs multimodal large language models (MLLMs) to label and filter concepts based on visual identifiability and task relevance, and selects an independent subset via reconstruction-guided optimization. Leveraging CLIP’s visual-text alignment, it decomposes image representations into linear combination of concept text embeddings to fit into the CBMs abstraction. Our extensive experiments across 11 image classification tasks demonstrate that PCBM-ReD achieves state-of-the-art accuracy, narrows the performance gap with end-to-end models, and exhibits better interpretability. The code will be released.