Recent evidence suggests that music-based interventions can be beneficial in maintaining cognitive, emotional, and social functioning in persons with dementia (PWDs). Our aim was to determine how clinical, demographic, and musical background factors influence the cognitive and emotional efficacy of caregiver-implemented musical activities in PWDs. In a randomized controlled trial, 89 PWD-caregiver dyads received a 10-week music coaching intervention involving either singing or music listening or standard care. Extensive neuropsychological testing and mood and quality of life (QoL) measures were performed before and after the intervention (n = 84) and six months later (n = 74). The potential effects of six key background variables (dementia etiology and severity, age, care situation, singing/instrument playing background) on the outcome of the intervention were assessed. Singing was beneficial especially in improving working memory in PWDs with mild dementia and in maintaining executive function and orientation in younger PWDs. Music listening was beneficial in supporting general cognition, working memory, and QoL especially in PWDs with moderate dementia not caused by Alzheimer's disease (AD) who were in institutional care. Both music interventions alleviated depression especially in PWDs with mild dementia and AD. The musical background of the PWD did not influence the efficacy of the music interventions. Our findings suggest that clinical and demographic factors can influence the cognitive and emotional efficacy of caregiver-implemented musical activities and are, therefore, recommended to take into account when applying and developing the intervention to achieve the greatest benefit.