
Premium content
Access to this content requires a subscription. You must be a premium user to view this content.

Would you like to see your presentation here, made available to a global audience of researchers?
Add your own presentation or have us affordably record your next conference.
Existing multilingual benchmarks suffer from inconsistent translation quality across tasks and languages, compromising evaluation reliability and reproducibility. Such benchmarks usually involve either long translation with manual verification or automated process which fails to preserve original context and coherence. In this work we present a fully automated translation framework supporting both proprietary and open models that enables scalable multilingual benchmark and dataset translation. The framework can also be applied to general dataset translation, facilitating broader multilingual model development. We demonstrate that test-time scaling methods like Best-of-N and Fusion-of-N are effective for machine translation when applied correctly. Our newly proposed multi-round ranking method T-RANK leverages iterative refinement to identify and correct translation errors, helping to avoid repeating mistakes from existing multilingual benchmarks with minimal human intervention. This approach delivers higher-quality benchmark translations more efficiently than traditional methods, reducing both time and cost while enabling more accurate multilingual evaluation. We evaluate our methods across several Eastern European languages, translating popular benchmarks and datasets, and comparing results with existing translations. Our findings show that the integrated methods produce higher-quality translations with more accurate evaluation results and improved text quality. Our framework addresses a critical bottleneck in multilingual AI development by accelerating multilingual development and making high-quality benchmark localization accessible and scalable.
