The emergence of LM-based judging reward modeling, represented by generative reward models, has enabled cost-effective and efficient RLAIF by utilizing synthetic preferences generated from language models as an alternative to human annotations. For the first time, we reveal that such reward modeling shares fundamental formal consistency with Natural Language Inference, the core task in natural language understanding. Accordingly, we innovatively redefine this reward modeling paradigm as a Natural Language Inference task, which implies that we can enhance reward modeling capabilities by scaling the comprehension boundaries of language models. Through performance evaluation, we discover that masked language models incorporating contextual explanations demonstrate superior comprehension boundaries compared to current mainstream autoregressive models. Thus, we propose ESFP-RM, a lightweight two-stage reward model that leverages Explanation based Slot Framework for Prediction. Experimental results demonstrate that ESFP-RM, despite its lighter architecture, significantly outperforms existing LM-based judging reward modeling frameworks, including generative reward models in both RLHF and OOD tasks. By providing more stable and generalizable reward signals for large model alignment, ESFP-RM emerges as a more promising reward modeling paradigm compared to generative reward modeling.