The approximate number system (ANS) enables humans to rapidly estimate numerical quantities without relying on counting, yet the computational processes underlying the ANS remain mysterious. Here, we propose a mechanistic model for the ANS, built on the core idea that when presented with a stimulus array, observers first form a template representation through ensemble averaging and subsequently engage in a template-matching process, whereby each array item is compared to the template. With a limited set of plausible assumptions about the inherently noisy computational processes, our model naturally accounts for Weber’s law, the main hallmark of number psychophysics, systematic numerosity underestimation and the coherence illusion. We further test two novel predictions about stimulus factors that modulate the strength of the coherence illusion and demonstrate high fidelity between predicted and obtained data. This model offers new insight into the computational mechanisms by which ANS representations are derived from visual input.