Bilson, S; Cox, M G; Pustogvar, A; Thompson, A (2025) A metrological framework for uncertainty evaluation in machine learning classification models. Metrologia, 62 (6). 064001 ISSN 0026-1394

Preview

Text eid10475.pdf - Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract

Machine learning (ML) classification models are increasingly being used in a wide range of applications where it is important that predictions are accompanied by uncertainties, including in climate and earth observation, medical diagnosis and bioaerosol monitoring. The output of an ML classification model is a type of categorical variable known as a nominal property in the International Vocabulary of Metrology (VIM). However, concepts related to uncertainty evaluation for nominal properties are not defined in the VIM, nor is such evaluation addressed by the guide to the expression of uncertainty in measurement (GUM). In this paper we propose a metrological conceptual uncertainty evaluation framework for nominal properties. This framework is based on probability mass functions and summary statistics thereof, and it is applicable to ML classification. We also illustrate its use in the context of two applications that exemplify the issues and have significant societal impact, namely, climate and earth observation and medical diagnosis. Our framework would enable an extension of the GUM to uncertainty for nominal properties, which would make both applicable to ML classification models.

Item Type:	Article
Subjects:	Mathematics and Scientific Computing > Modelling
Divisions:	Data Science
Identification number/DOI:	10.1088/1681-7575/ae1bae
Last Modified:	03 Jul 2026 10:56
URI:	https://eprintspublications.npl.co.uk/id/eprint/10475