Using Artificial Intelligence and Augmented Reality to Assist Dementia Patients
Winner of the Junior Academy Innovation Challenge Spring 2024: “Wearables”
Sponsored by The New York Academy of Sciences
Published August 14, 2024
By Nicole Pope
Academy Education Contributor
Team members: Riya K. (India) (Team Lead), Shreeniket B. (United States), Sysha R. (India), Prakul P. (India), Tisha S. (India), Medha T. (United States)
Dementia is a neurodegenerative condition that affects 55 million people worldwide and 1 in 10 people older than 65 in the United States, according to the World Health Organization. Alzheimer’s disease is the best-known form of this debilitating condition, which leads to memory loss and confusion. It gradually erodes individuals’ ability to perform simple functions or even recognize familiar faces.
The impact of dementia is not limited to the individuals affected by the condition. Caregivers, who are often family members, bear the burden of providing support, often at great personal cost.
Harnessing the power of wearable technology, the members of this enterprising team composed of high-schoolers from the U.S. and India, devised the VisionXcelerate glasses – an innovative device designed to provide personalized assistance to dementia patients and help them perform every-day tasks independently, thus lightening the burden of carers. Some of the sensors are contained in an eyeglass chain that also ensures users do not lose the device. The students reached out to patients, hospitals and nursing homes to identify specific needs.
“After contacting 90 dementia patients in total, I gained new research and critical thinking abilities. I was able to identify their needs by observing them and analyzing their responses,” explains Sysha, who also learned about coding while working on the image/facial recognition for the glasses. “I have gained more empathy for others by trying to find a solution for people suffering from Alzheimer’s.”
“Wonder Glasses”
The VisionXcelerate smart glasses and their numerous features, including collecting user health data, are the result of an intense group effort, with each team member contributing different skills and strengths. “Being the Team Lead for this project, I’ve witnessed firsthand how collaboration, hard work, new ideas, and obstacles have shaped our journey,” explains Team Lead Riya. “Each member of our team has contributed significantly, from ideation to prototyping. It’s been inspiring to see everyone’s passion and commitment to the project.”
The students used both Artificial Intelligence (AI) and Augmented Reality (AR) to develop the various functions of their “wonder glasses”, which provide real-time assistance, and help dementia patients overcome some of the challenges posed by memory loss and help wearers identify objects and faces. “I was mainly involved in the research and design aspects of the project, particularly surrounding the technologies we wanted to implement in our design and their practicality/viability,” explains Shreeniket, whose approach was partly shaped by observing elderly family members.
For example, the device includes a virtual personal assistant that prompts users to take medication at specific times and to eat or drink at regular intervals, in a voice that mimics the patients’ loved ones. This helps to foster trust and build emotional connection.
VirtualXcelerate also features an app that allows caregivers to monitor the patients and their activities from afar. “The experience of working with teammates sitting in different corners of the world has helped me gain new insights and think about solutions from a completely new perspective,” explains Tisha.
A Rewarding Journey
Working across time zones was not always smooth sailing. “There were a few hiccups. But at the end of the day, everything came together beautifully,” says Prakul. “Despite the obstacles, the journey was rewarding. It pushed me out of my comfort zone, taught me valuable skills in teamwork, problem-solving, and project management, and allowed me to contribute to a meaningful cause.”
This experience gave team members a new understanding of dementia and how it affects patients and those around them. “I learned about the difficulty dementia patients face on a daily basis, and how our solution was impactful to them,” says Medha. “During this process, I helped contribute to researching certain features on the glasses, such as the reminders. Additionally, I sketched out each feature on the glasses, in a variety of angles to demonstrate the importance of location. Overall, this experience was an outstanding opportunity for me.”
As dementia sufferers are prone to falling, the glasses have detection systems to alert caregivers in case of a mishap while GPS tracking and geofencing allows them to locate patients who have wandered beyond a safe area and are too confused to find their way home. These security features not only contribute to dementia patients’ well-being, but also provide carers and relatives peace of mind.
With user feedback, the team plans to continue to refine their solution and add more elements that will further enhance the quality of life of people living with dementia, help them retain their independence longer, and ease the pressure on caregivers.
Read about the other winner from the Spring 2024 Junior Academy Innovation Challenge:
Developing Circular Textile Practices Through Recyclable Fabrics and Reducing Color Dye Pollution
