Ground Truth – Is it True in Healthcare?
The CEO Corner
Wayne Moore
2/14/2023
“Ground truth” is a term commonly used in statistics and machine learning, although it had its beginnings
in Meteorology and refers to the correct or “true” answer to a specific problem or question. In the field of AI it is held as a “gold standard” that can be used to compare and evaluate model results. For example, in a medical image modality classification system, the algorithm learns to classify each image into a set of classes (e.g., MRI, CT, Ultrasound, or heart, liver, breast using a single modality). The algorithm then “trains”/learns by using data with ground truth labels, indicating the true class of each object. When the developer is certain that the model is ready for validation, it is applied to a series of unreviewed images. An ideal model will generate a prediction identical to the ground truth label for every source image. Specifically, it will know, within probability bounds, that an image of a liver is a liver, and an image of a tumor is a tumor. The key elements (and challenges) in designing an appropriate AI model are collecting enough relevant and high-quality data across a broad cross-section of patient demographics (i.e., enough data to cover all classes and outlier cases that will be input to the algorithm). In healthcare usage great care must be taken to ensure that data bias is not designed into the algorithm thereby rendering it useless: e.g., in ultrasound inputting breast images that do not include dense breast examples. The FDA places a great deal of weight on data collection and if outliers, and other variants, have been considered when developing the AI tool. We are still at the nascent stage of AI development in healthcare and our knowledge concerning machine learning is still not well known. Machine Learning (ML) knowledge development begins with the input data and within healthcare this means clinically specific applications over a potentially wide demographics with several tiers of outliers – no easy task to be sure. We will get there but it will take a decade before folks will be comfortable with machine-based diagnosis; in the same manner as being comfortable with a pilotless jetliner!
Until Next Month,
Wayne