Response to Vos et al: Modifiable Risk Factors for Prevention of Dementia in Midlife, Late Life and the Oldest-Old: Validation of the LIBRA Index

17 November 2017

To the Editor:

We would like to thank Vos and colleagues for valuable research on the topic of modifiable risk factors for dementia [1]. However, we feel three topics need to be addressed in order to interpret the results with due nuance.

First, the authors state that the aim of their research is “to test the ability of the LIBRA index to assess the prevention potential by investigating the predictive validity” and “C-statistics were calculated as a measure of predictive accuracy”, indicating this as the main analysis. The c-statistics for predictive models with the LIBRA-index in the three different age groups are respectively 0.53, 0.56 and 0.50, reflecting the predictive accuracy of the model to predict who will get dementia and who will not. Provided that rolling a dice equals a c-statistic of 0.50, and that c-statistics usually are down-sized when models are applied to external datasets, the performance of the models is hardly convincing.

Second, each age group is split into tertiles of the LIBRA index, and the survival probabilty for progression to dementia is plotted against time. We have two comments on this analysis. The first is the chosen time frame. Average follow-up is 7.2 years, yet the time frame in the Kaplan-Meier plots extends until 15 years. This practice is questionable, especially in the higher age groups, because this exceeds life expectancy for that group. Furthermore, the large stepwise changes with longer follow-up in these plots suggest there were very few people left in the analysis after approximately 8 years, resulting in imprecise or inaccurate estimates [2]. The divergence between the three risk groups clearly only arises after this timepoint. We believe it would be interesting to see results of Log-Rank tests for plots limited to 8 years of follow-up, and it would be necessary to add ‘number at risk’-tables to these data. Moreover, the data show that the LIBRA index is not successful in discriminating between all three risk groups, especially in higher age groups, as indicated by the Cox models. It would be interesting to identify risk groups based on maximum discrimination instead of tertiles, to see if those groups would perform better.

Third, the authors state that based on this research, it can be concluded that elderly people should attempt to obtain a LIBRA index score as low as possible, in order to avoid dementia. We would like to point out that decreasing LIBRA scores for individuals does not necessarily decrease incidence. Assuming the causal relation between those risk factors and dementia (the study merely shows associations), eliminating any of those risk factors as component causes does not necessarily mean that remaining risk factors do not constitute sufficient cause for dementia [3]. Moreover, achieving a lower LIBRA scale involves cardiovascular risk management, which in a recent large trial in older age was unsuccessful in reducing dementia incidence [4]. This does not support the notion that reducing a LIBRA score in elderly would lower dementia risk. Such actions may be more fruitful when started at much earlier age (the ‘historical’ midlife 40-50 years).

To conclude, given the limited predictive accuracy of the LIBRA index and the limited ability to discriminate risk groups (which may also be distorted by the long time frame in the analysis), we think this work would benefit from more cautious conclusions.

Anke Richters, MSc1,2, Jurgen A.H. Claassen, MD, PhD1,2
1Donders Institute for Brain, Cognition and Behaviour, Department of Geriatric Medicine, Radboud University Medical Center, Nijmegen, The Netherlands
2Radboud Alzheimer Centre, Nijmegen, The Netherlands

References
[1] Vos SJB, van Boxtel MPJ, Schiepers OJG, Deckers K, de Vugt M, Carrière I, Dartigues JF, Peres K, Artero S, Ritchie K, Galluzzo L, Scafato E, Frisoni GB, Huisman M, Comijs HC, Sacuiu SF, Skoog I, Irving K, O'Donnell CA, Verhey FRJ, Visser PJ, Köhler S (2017) Modifiable risk factors for prevention of dementia in midlife, late life and the oldest-old: validation of the LIBRA Index. J Alzheimers Dis 58, 537-547.
[2] Rich JT, Neely JG, Paniello RC, Voelker CC, Nussenbaum B, Wang EW (2010) A practical guide to understanding Kaplan-Meier curves. Otolaryngol Head Neck Surg 143, 331-336.
[3] Rothman KJ (1995) Causes. 1976. Am J Epidemiol 141, 90-95; discussion 89.
[4] van Dalen JW, Moll van Charante EP, Caan MWA, Scheltens P, Majoie CBLM, Nederveen AJ, van Gool WA, Richard E (2017) Effect of long-term vascular care on progression of cerebrovascular lesions: magnetic resonance imaging substudy of the PreDIVA Trial (Prevention of Dementia by Intensive Vascular Care). Stroke 48, 1842-1848.

Comments

Submitted by Stephanie Vos, PhD on

We would like to thank Mrs. Richtes and Dr. Claassen for the critical appraisal of our paper [1]. However, we feel there may be some misunderstandings about our aim, methods, and presented results that we would like to address below.

Mrs. Richtes and Dr. Claassen indicate that the predictive performance of the models is hardly convincing based on the C-statistics. We agree that the C-statistics are relatively low (ranging from 0.50 to 0.57), as was described in the discussion of our paper. DESCRIPA is a large but heterogeneous multicenter study so the predictive performance could be underestimated and may be higher in more homogeneous cohorts. Moreover, LIBRA was developed as a summary score of modifiable risk scores for dementia [2] based on the existing literature with weights coming from risk estimates of existing meta-analyses [3] rather than taking estimates from a single cohort study. This is in line with previous recommendations [4]. As such, it lacked external validation whether it can indeed predict dementia risk at the population level. We used Cox proportional hazard models as our main analyses and showed that the continuous LIBRA index as well as groups based on LIBRA tertiles predicts average risk for dementia. LIBRA therefore has predictive (criterion) validity. Based on our results and low C statistics, the index may however not be suitable for predictive accuracy purposes but it was not our aim to use or test the LIBRA index as a diagnostic or prognostic tool for an individual patient. LIBRA was rather developed to identify groups of individuals within the population with room for health improvement that can participate in prevention interventions. Hence, we think that LIBRA might be interesting for public health purposes by targeting groups at increased environmental risk, while this might not necessary translate to large benefits at the individual level (known as the prevention paradox).

Mrs. Richtes and Dr. Claassen question the chosen time frame and suggest restricting analyses to 8 years of follow-up instead of 16 years. As requested, we have specified the number of individuals at risk at each follow-up in Table 1, showing that a relatively large group of individuals was available after 8 years of follow-up. Still, the cumulative hazard plot (not Kaplan Meier plot of survival probabilities) allows one to examine progression rates up to 8 years only. We think it is important to stress that the LIBRA score was assessed at baseline only and predicted dementia risk over a maximum of 16 years later. Our cumulative hazard plots showed that the hazard for dementia as a function of higher LIBRA scores is increasing proportionally with increasing survival time. The stepwise increase is not due to unusual low numbers but to the fact that not all included studies reported a date of dementia diagnosis but rather a year of dementia diagnosis at study-specific intervals up to 16 years after baseline. It may be important to mention that Cox regression takes into account the changing risk of the event over time in the population (changing baseline hazard) by calculating the survival probability based on the risk set available on the moment of the event (the instantaneous rate per time-click). Therefore Cox regression is an appropriate method to use when studying an aging population, where the failure event is more likely to occur with increasing observation time. We now also ran analyses with only 8 years of follow-up (Table 2). These analyses showed that LIBRA was superior in predicting dementia after a longer follow-up (+8 years) in younger individuals (late life) while for the oldest-old a shorter follow-up may be sufficient (≤8 years). In midlife, the age group in which LIBRA was originally developed, LIBRA appeared to predict dementia equally well, though significance testing was inconclusive at ≤8 years follow-up (few events). Mrs. Richtes and Dr. Claassen also question the use of tertiles. We used tertiles for visualization purposes, easy interpretation of results, and higher power in the age subgroups. One of our previous validation studies used quintiles and showed a similar trend of results [5].

Mrs. Richtes and Dr. Claassen state that lowering LIBRA scores does not necessarily decrease dementia incidence. We agree that only a prevention trial aiming at reducing the LIBRA score and testing the effect on dementia risk could provide direct evidence for this. While the cited preDIVA trial was generally negative, it is interesting that it did find a treatment effect in subgroups with previously untreated hypertension and in those without cardiovascular disease [6]. The age range was 70-78 years, so these people fall within our late-life group. Recent consensus papers suggest that targeting LIBRA factors such as hypertension to reduce cardiovascular risk, physical activity and mental stimulation can be recommended to a wider public for healthy brain ageing [7,8]. We fully agree that such preventative measure should start in midlife or even earlier. Further validation studies and prevention trials will help in defining the age range and disease stage in which prevention interventions might be most successful. For now we believe that LIBRA can be a useful tool for raising people’s awareness and for the identification of groups of individuals who might benefit most from primary prevention strategies through lifestyle change or health management.

S.J.B. Vos and S. Köhler

References

[1] Vos SJB, van Boxtel MPJ, Schiepers OJG, Deckers K, de Vugt M, Carrière I, Dartigues JF, Peres K, Artero S, Ritchie K, Galluzzo L, Scafato E, Frisoni GB, Huisman M, Comijs HC, Sacuiu SF, Skoog I, Irving K, O'Donnell CA, Verhey FRJ, Visser PJ, Köhler S (2017) Modifiable risk factors for prevention of dementia in midlife, late life and the oldest-old: validation of the LIBRA Index. J Alzheimers Dis 58, 537-547.
[2] Deckers K, van Boxtel MPJ, Schiepers OJG, de Vugt M, Muñoz Sánchez JL, Anstey KJ, Brayne C, Dartigues JF, Engedal K, Kivipelto M, Ritchie K, Starr JM, Yaffe K, Irving K, Verhey FR, Köhler S (2015) Target risk factors for dementia prevention: a systematic review and Delphi consensus study on the evidence from observational studies. Int J Geriatr Psychiatry 30, 234-246.
[3] O’Donnell CA, Browne S, Pierce M, McConnachie A, Deckers K, van Boxtel MP, Manera V, Köhler S, Redmond M, Verhey FR, van den Akker M, Power K, Irving K; In-MINDD Team (2015) Reducing dementia risk by targeting modifiable risk factors in mid-life: study protocol for the Innovative Midlife Intervention for Dementia Deterrence (In-MINDD) randomised controlled feasibility trial. Pilot Feasibility Stud 1, 40.
[4] Tang EY, Harrison SL, Errington L, Gordon MF, Visser PJ, Novak G, Dufouil C, Brayne C, Robinson L, Launer LJ, Stephan BC (2015) Current developments in dementia risk prediction modelling: An updated systematic review. PLoS One 10, e0136181.
[5] Schiepers OJ, Köhler S, Deckers K, Irving K, O'Donnell CA, van den Akker M, Verhey FR, Vos SJ, de Vugt ME, van Boxtel MP (2017) Lifestyle for Brain Health (LIBRA): a new model for dementia prevention. Int J Geriatr Psychiatry, doi: 10.1002/gps.4700.
[6] Moll van Charante EP, Richard E, Eurelings LS, van Dalen JW, Ligthart SA, van Bussel EF, Hoevenaar-Blom MP, Vermeulen M, van Gool WA (2016) Effectiveness of a 6-year multidomain vascular care intervention to prevent dementia (preDIVA): a cluster-randomised controlled trial. Lancet 388, 797-805.
[7] Downey A, Stroud C, Landis S, Leshner AI, For the National Academies of Sciences Engineering and Medicine (2017) Preventing Cognitive Decline and Dementia: A Way Forward. National Academies Press (US), Washington, DC.
[8] Livingston G, Sommerlad A, Orgeta V, Costafreda SG, Huntley J, Ames D, Ballard C, Banerjee S, Burns A, Cohen-Mansfield J, Cooper C, Fox N, Gitlin LN, Howard R, Kales HC, Larson EB, Ritchie K, Rockwood K, Sampson EL, Samus Q, Schneider LS, Selbæk G, Teri L, Mukadam N (2017) Dementia prevention, intervention, and care. Lancet, doi: 10.1016/S0140-6736(17)31363-6.

Table 1. Number of individuals at risk at each follow-up visit.

Follow-up visit (in years) Number at risk
1 9387
2 8977
3 8452
4 7474
5 7005
6 6324
7 5441
8 5039
9 2908
10 1387
11 999
12 996
13 962
14 670
15 542
16 37

Table 2.Prediction for dementia stratified by age groups and by risk groups for dementia based on up to 8 years of follow-up versus more than 8 years of follow-up data.

Midlife

Late life

Oldest-old

Index

Risk group

Hazard ratio

p-value

Hazard ratio

p-value

Hazard ratio

p-value

LIBRA

In subset with
≤8 years follow-up

Low

Reference

Reference

Reference

Intermediate

1.63 (0.77-3.45)

L: p=0.201
H: p=0.563

1.09 (0.83-1.43)

L: p=0.525
H: p=0.070

0.91 (0.67-1.23)

L: p=0.527

H: p=0.085

High

1.92 (0.90-4.12)

L: p=0.092

0.85 (0.64-1.14)

L: p=0.282

0.68 (0.49-0.95)

L: p=0.024

LIBRA

In subset with >8 years follow-up

Low

Reference

Reference

Reference

Intermediate

1.55 (0.95-2.55)

L: p=0.081
H: p=0.355

1.34 (1.00-1.79)

L: p=0.047
H: p=0.111

1.02 (0.64-1.64)

L: p=0.922

H: p=0.328

High

1.87 (1.10-3.17)

L: p=0.020

1.74 (1.23-2.46)

L: p=0.002

1.32 (0.80-2.17)

L: p=0.282

Results are Hazard ratios with 95% confidence intervals for individuals with up to 8 years of follow-up versus more than 8 years of follow-up. L, low risk group; H, high risk group.