Large Language Model (LLM) agent systems have advanced rapidly, driven by their strong generalization in zero-shot settings. To further enhance reasoning and accuracy on complex tasks, Multi-Agent Debate (MAD) has emerged as a promising framework that engages multiple LLM agents in structured debates to encourage diverse reasoning. However, triggering MAD for every input instance is inefficient, as it incurs substantial computational (token) cost and may even degrade accuracy by overturning correct single-agent answers. To address these limitations, we propose intelligent Multi-Agent Debate (iMAD), a token-efficient framework that selectively triggers MAD only when it is likely to be beneficial (i.e., correcting an initially wrong answer) in the zero-shot setting. To achieve this goal, iMAD learns generalizable model behaviors to make accurate debate decisions in the zero-shot setting. Specifically, it first prompts a single agent to produce a structured self-critique response, from which we extract over 40 interpretable linguistic and semantic features capturing hesitation cues. A lightweight classifier, based on Multi-Layer Perceptron and trained using our proposed FocusCal loss, then determines whether to trigger MAD, enabling robust zero-shot decisions without dataset-specific tuning. We evaluate iMAD on six (visual) question answering datasets against five competitive baselines. iMAD significantly reduces token usage (by up to 92%) while also improving final answer accuracy (by up to 13.5%).