Easier said than done: unexpected hurdles to preparing ∼1,000 cranial CT scans for data collection from an online digital repository

Abstract

Background

As science becomes more open and accessible, researchers are increasingly encouraged—and sometimes required—to share their digital data on public repositories. While this promotes transparency and reusability, it can also introduce challenges. We highlight one such challenge by detailing our experience processing computerized tomography (CT) scans of 985 baboon skulls downloaded from MorphoSource, part of a quantitative genetic study of craniodental variation in the pedigreed baboon colony from the Southwest National Primate Research Center. When importing DICOM files into 3D Slicer, 182 of the 985 scans (18.5%) generated an “inconsistent slice spacing” error. When prompted, 3D Slicer “corrected” this by regularizing the slice spacing. However, this led to a mismatch between the slice spacing reported on MorphoSource and the spacing adjusted by 3D Slicer. Methods

To determine which slice spacing was accurate, we compared Prosthion-Basion (PR-BA) distances measured directly from physical skulls (using calipers and a Microscribe) with those derived from the CT models. We ran paired t-tests to determine whether there were differences between them. Our comparison sample included five randomly selected skulls from the error group and fifteen ramdon skulls from the error-free group (which exhibited various slice spacings when scanned). Results

For scans without the slice spacing error, there was strong agreement between physical and digital measurements, indicating metadata accuracy. For error-generating scans, measurements based on 3D Slicer’s corrected spacing and Amira-Avizo both aligned well with the physical data. In contrast, manually overriding the spacing to match the MorphoSource metadata led to overestimations of the PR-BA distance. Conclusion

Although the discrepancy was straightforward to describe, resolving it required over 250 person-hours across 8 months. Accessing physical specimens, conducting repeated measurements, and cross-validating with multiple tools made the process labor-intensive. Nonetheless, this effort avoided a 3–5% measurement bias in nearly 20% of our sample and allowed inclusion of these scans in downstream semi-automated data collection. We urge researchers to thoroughly understand the digital datasets they work with and resist the temptation to ignore apparent errors during import. We also recommend that funding bodies provide support for the extensive time needed to validate and process digital imagery, both for data generators and users. Finally, we highlight the need for public repositories to implement stronger quality control. If a data import check similar to 3D Slicer’s had been applied during data submission, the inconsistency between manually entered metadata and embedded DICOM information might have been caught and corrected at the time of upload.