A Multimodal Evaluation Pipeline for Mathematical Expression Recognition: Comparisons of Datasets, Metrics, and Models
Published in 19th International Conference on Document Analysis and Recognition (ICDAR 2025), 2025
This paper presents PiE-MER, a multimodal evaluation pipeline for printed mathematical expression recognition. The proposed approach goes beyond LaTeX string similarity by converting predictions into several complementary representations, including MathML, graph-based structures, and normalized images.
The goal is to provide a broader and more robust evaluation of recognition systems across multiple datasets and output modalities. The study compares five representative models across eight datasets and shows that classical text-only metrics such as BLEU and Levenshtein are not sufficient to fully capture syntactic, semantic, and visual accuracy.
Recommended citation: François Wieckowiak, Véronique Eglin, Tony Bonnet, Stéphane Bres, and Laetitia Rousseau. (2025). "A Multimodal Evaluation Pipeline for Mathematical Expression Recognition: Comparisons of Datasets, Metrics, and Models." 19th International Conference on Document Analysis and Recognition (ICDAR 2025).
Download Paper