Inter-rater Variability in Assessing Hippocampal Atrophy Using Scheltens Scales

Authors: A. Kadlecová 1,2;  M. Vyhnálek 1,2;  J. Laczó 1,2;  R. Anděl 2,3;  K. Sheardová 4;  B. Urbanová 1;  Z. Nedělská 1,2;  D. Hudeček 2;  I. Gažová 1,2;  J. Lisý 5;  D. Hořínek 2,6;  J. Hort 1,2
Authors‘ workplace: Kognitivní centrum, Neurologická klinika 2. LF UK a FN v Motole, Praha 1;  Mezinárodní centrum klinického výzkumu, FN u sv. Anny v Brně 2;  School of Aging Studies, University of South Florida, Tampa, Florida, USA 3;  Neurologická klinika, Mezinárodní centrum klinického výzkumu, FN u sv. Anny v Brně 4;  Klinika zobrazovacích metod 2. LF UK a FN v Motole, Praha 5;  Neurochirurgická klinika 1. LF UK a ÚVN – Vojenská fakultní nemocnice Praha 6
Published in: Cesk Slov Neurol N 2013; 76/109(5): 603-607
Category: Original Paper

Děkujeme za finanční podporu grantu IGA NT 11225- 4 a projektu FNUSA‑ ICRC (no. CZ.1.05/ 1.1.00/ 02.0123) z Evropského fondu regionálního rozvoje.


Recently, a great emphasis has been placed on early diagnosis of Alzheimer’s disease (AD). The new diagnostic criteria for AD involve new methods such as detection of structural and metabolic changes in the brain. These include examination of hippocampal volume. Scheltens et al. introduced a visual rating scale for hippocampal atrophy assessment. Our aim was to determine inter-rater variability and to test practical application of this scale.

MRI scans of 70 elderly persons with cognitive impairment and persons classified as cognitively normal were assessed by eight investigators. The investigators had a different degree of experience with Scheltens visual rating scale. Following a brief training, the investigators were asked to practice on 20 MRI brain scans in coronal plane. Correct scores were disclosed to all investigators. Subsequently, the investigators evaluated 70 study MRI scans. They were unaware of the participants’ cognitive status. The variability was calculated for all investigators together and after dividing them into “experienced” (n = 4) and “inexperienced” (n = 4) group, where “inexperienced” refered to no previous knowledge of this visual rating scale. Results: Inter-rater agreement (kappa) was very high among all investigators for the right (K = 0.87) and left (K = 0.88) hippocampus. After dividing the raters into experienced and inexperienced, the inter-rater variability continued to be high in both groups for the right and left hippocampus.

Scheltens visual rating scale is a simple and easy to use tool for hippocampal atrophy assessment even for inexperienced investigators. The final assessment variability was relatively low. We can recommend this visual rating scale for routine clinical use.

Key words:
atrophy – hippocampus – visual scale – dementia – Alzheimer disease

The authors declare they have no potential conflicts of interest concerning drugs, products, or services used in the study.

The Editorial Board declares that the manu­script met the ICMJE “uniform requirements” for biomedical papers.


1. Scarpini E, Scheltens P, Feldman H. Treatment of Alzheimer’s disease: current status and new perspectives. Lancet Neurol 2003; 2(9): 539– 547.

2. Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox NC et al. The dia­gnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging‑ Alzheimer’s Association workgroups on dia­gnostic guidelines for Alzheimer’s disease. Alzheimers Dement 2011; 7(3): 270– 279.

3. Hort J, O’Brien JT, Gainotti G, Pirttila T, Popescu BO, Rektorova I et al. EFNS Scientist Panel on Dementia. EFNS guidelines for the dia­gnosis and management of Alzheimer’s disease. Eur J Neurol 2010; 17(10): 1236– 1248.

4. Dubois B, Feldman HH, Jacova C, Dekosky ST, Barberger‑ Gateau P, Cummings J et al. Research criteria for the dia­gnosis of Alzheimer’s disease: revising the NINCDS‑ ADRDA criteria. Lancet Neurol 2007; 6(8): 734– 746.

5. Hort J, Glosová L, Vyhnálek M, Bojar M, Škoda D, Hladíková M. The liquor tau protein and beta amyloid in Alzheimer’s disease. Cesk Slov Neurol N 2007; 70/ 103(1): 30– 36.

6. Hort J, Bartos A, Pirttilä T, Scheltens P. Use of cerebrospinal fluid bio­markers in dia­gnosis of dementia across Europe. Eur J Neurol 2010; 17(1): 90– 96.

7. Nedelska Z, Andel R, Laczó J, Vlcek K, Horinek D, Lisy J et al. Spatial navigation impairment is proportional to right hippocampal volume. Proc Natl Acad Sci U S A 2012; 109(7): 2590– 2594.

8. Venneri A, Gorgoglione G, Toraci C, Nocetti L, Panzetti P, Nichelli P. Combining neuropsychological and structural neuroimaging indicators of conversion to Alzheimer’s disease in amnestic mild cognitive impairment. Curr Alzheimer Res 2011; 8(7): 789– 797.

9. Benoit M, Robert PH. Imaging correlates of apathy and depression in Parkinson’s disease. J Neurol Sci 2011; 310(1– 2): 58– 60.

10. Heckers S, Konradi C. Hippocampal pathology in schizophrenia. Curr Top Behav Neurosci 2010; 4: 529– 553.

11. Horínek D, Varjassyová A, Hort J. Magnetic resonance analysis of amygdalar volume in Alzheimer’s disease. Curr Opin Psychiatry 2007; 20(3): 273– 277.

12. Cavallin L, Bronge L, Zhang Y, Oksengård AR, Wahlund LO, Fratiglioni L et al. Comparison between visual assessment of MTA and hippocampal volumes in an elderly, non‑demented population. Acta Radiol 2012; 53(5): 573– 579.

13. Scheltens P, Leys D, Barkhof F, Huglo D, Weinstein HC, Vermersch P et al. Atrophy of medial temporal lobes on MRI in “probable“ Alzheimer’s disease and normal ageing: dia­gnostic value and neuropsychological correlates. J Neurol Neurosurg Psychiatry 1992; 55(10): 967– 972.

14. Korf ES, Wahlund LO, Visser PJ, Scheltens P. Medial temporal lobe atrophy on MRI predicts dementia in patients with mild cognitive impairment. Neurology 2004; 63(1): 94– 100.

15. Koedam EL, Lehmann M, van der Flier WM, Scheltens P, Pijnenburg YA, Fox N et al. Visual assessment of posterior atrophy development of a MRI rating scale. Eur Radiol 2011; 21(12): 2618– 2625.

16. Kapeller P, Barber R, Vermeulen RJ, Adèr H, Scheltens P, Freidl W et al. European Task Force of Age Related White Matter Changes. Visual rating of age‑related white matter changes on magnetic resonance imaging: scale comparison, interrater agreement, and correlations with quantitative measurements. Stroke 2003; 34(2): 441– 445.

17. Weintraub D, Doshi J, Koka D, Davatzikos C, Siderowf AD, Duda JE et al. Neurodegeneration across stages of cognitive decline in Parkinson disease. Arch Neurol 2011; 68(12): 1562– 1568.

18. Diniz PR, Velasco TR, Salmon CE, Sakamoto AC, Leite JP, Santos AC. Extratemporal damage in temporal lobe epilepsy: magnetization transfer adds information to volumetric MR imaging. AJNR Am J Neuroradiol 2011; 32(10): 1857– 1861.

19. Hayashi K, Kurioka S, Yamaguchi T, Morita M, Kanazawa I, Takase H et al. Association of cognitive dysfunction with hippocampal atrophy in elderly Japanese people with type 2 diabetes. Diabetes Res Clin Pract 2011; 94(2): 180– 185.

20. Laakso MP, Partanen K, Riekkinen P, Lehtovirta M, Helkala EL, Hallikainen M et al. Hippocampal volumes in Alzheimer’s disease, Parkinson’s disease with and without dementia, and in vascular dementia: an MRI study. Neurology 1996; 46(3): 678– 681.

21. Scheltens P, Launer LJ, Barkhof F, Weinstein HC, van Gool WA. Visual assessment of medial temporal lobe atrophy on magnetic resonance imaging: interobserver reliability. J Neurol 1995; 242(9): 557– 560.

22. Wu Y, Du H, Storey P, Glielmi C, Malone F, Sidharthan S et al. Comprehensive brain analysis with automated high‑resolution magnetization transfer measurements. J Magn Reson Imaging 2012; 35(2): 309– 317.

23. Gronenschild EH, Habets P, Jacobs HI, Mengelers R,Rozendaal N, van Os J et al. The effects of FreeSurfer version, workstation type, and Macintosh operating system version on anatomical volume and cortical thickness measurements. PLoS One 2012; 7(6): e38234.

24. Wattjes MP, Henneman WJ, van der Flier WM, de Vries O, Träber F, Geurts JJ et al. Dia­gnostic imaging of patients in a memory clinic: comparison of MR imaging and 64- detector row CT. Radiology 2009; 253(1): 174– 183.

Paediatric neurology Neurosurgery Neurology

Article was published in

Czech and Slovak Neurology and Neurosurgery

Issue 5

2013 Issue 5

Most read in this issue
Forgotten password

Don‘t have an account?  Create new account

Forgotten password

Enter the email address that you registered with. We will send you instructions on how to set a new password.


Don‘t have an account?  Create new account