14 March 2019
In our recent paper, Fung et al. [1], we investigated the validity of automated FreeSurfer protocols in a group of routine clinical brain MRI scans. One structure of interest was the entorhinal cortex, in which we compared manual segmentation against automated segmentation generated by the ex vivo protocols of Fischl et al. [2] and Augustinack et al. [3] found in FreeSurfer version 6.
The standard volume statistics generated from this cytoarchitecturally defined protocol are found in the automatically generated stats files (BA_exvivo.stats and BA_exvivo.thresh.stats); however, during the auditing of our study data, we noted that there was a mismatch in the “annot labels” of the volume statistics with the “probabilistic labels”. Specifically, the structure labelled as “entorhinal cortex” in the annot stats file corresponded more closely to the perirhinal cortex probabilistic label, while the structure labelled as “perirhinal label” in the annot stats file corresponded more closely to the entorhinal cortex probabilistic label. Given our observation that the probabilistic labels were consistent with the expected anatomical locations of these two structures, as a workaround to the mislabelling in the stats files, the annot statistical output that was labelled as “perirhinal cortex” was used in our analysis in place of the FreeSurfer automated ex vivo “entorhinal” segmentation.
We contacted the developers of FreeSurfer and they have acknowledged that this systematic mis-labelling was an issue previously unknown to them and agreed that the structures are mislabelled within the annot stats file. The developers have advised to use the work-around of the probabilistic labels for analysis while they work on an update to resolve the error with the annot labelling. At the time of writing we are unaware of available software updates to rectify this problem.
In sum, the aim of this letter is twofold: (1) to bring broad collegiate attention to the currently unaddressed anomalies in the automated stats output found in FreeSurfer v.6, and (2) to highlight the vital importance of researcher vigilance when evaluating the assumptions surrounding the outputs generated by automated algorithms, which are possibly too often taken at face value.
Yi Leng Fung1, Kelly E.T. Ng1, Simon J. Vogrin2, Catherine Meade2, Michael Ngo2, Steven J. Collins2,3, Stephen C. Bowden1,2
1 School of Psychological Sciences, University of Melbourne, Victoria, Australia
2 Centre for Clinical Neuroscience and Neurological Research, St Vincent’s Hospital, Victoria, Australia
3 Department of Medicine, University of Melbourne, Victoria, Australia
Correspondence to:
Yi Leng Fung, MPsych (Clinical Neuropsychology)/PhD Candidate, School of Psychological Sciences, University of Melbourne, Victoria, Australia. E-mail: yfung@student.unimelb.edu.au
References
[1] Fung YL, Ng KET, Vogrin SJ, Meade C, Ngo M, Collins SJ, Bowden SC (2019) Comparative Utility of Manual versus Automated Segmentation of Hippocampus and Entorhinal Cortex Volumes in a Memory Clinic Sample. J Alzheimers Dis, doi: 10.3233/JAD-181172.
[2] Fischl B, Stevens AA, Rajendran N, Yeo BT, Greve DN, Van Leemput K, Polimeni JR, Kakunoori S, Buckner RL, Pacheco J, Salat DH, Melcher J, Frosch MP, Hyman BT, Grant PE, Rosen BR, van der Kouwe AJ, Wiggins GC, Wald LL, Augustinack JC (2009) Predicting the location of entorhinal cortex from MRI. Neuroimage 47, 8-17.
[3] Augustinack JC, Huber KE, Stevens AA, Roy M, Frosch MP, van der Kouwe AJ, Wald LL, Van Leemput K, McKee AC, Fischl B, Alzheimer's Disease Neuroimaging Initiative (2013) Predicting the location of human perirhinal cortex, Brodmann's area 35, from MRI. Neuroimage 64, 32-42.
