Large Language Models (LLMs) have been extensively employed in Legal Judgment Prediction (LJP) in recent years. However, existing LLM-based methods often fail to effectively simulate the cognitive processes of human judges, particularly in keyword extraction, leading to suboptimal predictions. Inspired by cognitive science, we propose KnowJudge, a knowledge-driven framework, which explicitly models the cognitive process of legal decision-making, leveraging keyword extraction and precedent-based enhancement to guide LLMs in structured legal reasoning. By integrating external legal knowledge tailored to fact descriptions, it refines keyword identification and selects relevant case precedents, thereby mitigating ambiguity in legal judgment. Unlike conventional methods that rely on fine-tuning, KnowJudge improves performance purely through cognitive-process simulation. Experiments on five benchmarks show that KnowJudge outperforms baseline methods, including both general and legal LLMs.