Partarrieu, I X; Smith, N; Harris, P (2022) What Am I Measuring? Using Experimental Design to Understand and Improve Measurement Pipelines, with an Example Application to Radiomics Measurements. NPL Report. MS 35

Preview

Text (NPL Report MS 35) MS35.pdf - Published Version Download (2MB) | Preview

Abstract

Measurement pipelines are becoming more and more complex as technology improves. Additionally, competition between measurement providers means that technological specifications may differ significantly between instruments designed to be measuring the same underlying measurand at a similar resolution. These trends have been identified as contributing to the reproducibility challenge, as often it is difficult to establish whether measurements are different due to differences in the measurand or in the measurement pipeline. Here, we discuss a method for determining the relative sensitivity of a measurement to changes in the underlying measurand and to changes in factors defining the pipeline. We apply the method to radiomics, which is a machine learning methodology used in medical imaging to calculate a large number of image features from patient scans, in an attempt to extract information that might be of prognostic or predictive value. We show that our method can be used to give a clear prioritisation order for standardisation of those factors by quantifying their effect on the measurement. The method can also be used to understand the comparability between radiomic studies, as it makes it possible to determine whether the differences in factor levels will affect a measurand. Importantly, the method can determine the effect of the interactions of factors effecting the measurement.

Item Type:	Report/Guide (NPL Report)
NPL Report No.:	MS 35
Keywords:	Design of Experiments, Sensitivity Analysis, Radiomics, Measurement, Repeatability, Reproducibility, Replicability
Subjects:	Mathematics and Scientific Computing > Measurement Uncertainties
Divisions:	Data Science
Identification number/DOI:	10.47120/npl.MS35
Last Modified:	23 Mar 2022 09:00
URI:	https://eprintspublications.npl.co.uk/id/eprint/9383