Machine Learning for Enhancing Dementia Screening in Ageing Deaf Signers of British Sign Language

Autor/a: LIANG, Xing; WOLL, Bencie; KAPETANIOS, Epaminondas; ANGELOPOULOU, Anastasia; AL BATAT, Reda
Año: 2020
Editorial: Proceedings of the 9th Workshop on the Representation and Processing of Sign Languages, pages 135–138
Tipo de código: Copyright
Soporte: Digital


Medios de comunicación y acceso a la información » Nuevas Tecnologías


Real-time hand movement trajectory tracking based on machine learning approaches may assist the early identification of dementia in ageing deaf individuals who are users of British Sign Language (BSL), since there are few clinicians with appropriate communication skills, and a shortage of sign language interpreters. In this paper, we introduce an automatic dementia screening system for ageing Deaf signers of BSL, using a Convolutional Neural Network (CNN) to analyse the sign space envelope and facial expression of BSL signers recorded in normal 2D videos from the BSL corpus. Our approach involves the introduction of a sub-network (the multi-modal feature extractor) which includes an accurate real-time hand trajectory tracking model and a real-time landmark facial motion analysis model. The experiments show the effectiveness of our deep learning based approach in terms of sign space tracking, facial motion tracking and early stage dementia performance assessment tasks.