EMOTI: Emotion Mobile Observation Tracking Intervention

Addressing the Challenge

This research project has allowed for the construction of the EMOTI app, support Therapist Control Portal (TCP) and Automated Cloud Computing Platform (ACCP) which utilises low-cost emotion tracking application programming interfaces (API’s). The app is capable of tracking a person’s emotions via facial expression landmarks, speech-to-text sentiment analysis, and voice acoustics processing. During the initial proposal the focus was solely on face tracking and facial expression tracking analytics, however the team have added additional functionality in the forms of speech, audio, pitch and sentiment analysis which has added enhanced the value to the outcomes captured from the studies completed.

People

The project is led by Professor Philip Breedon with a multidisciplinary team which includes Paul Watts, lead software developer, NTU; Dr Luke Siena, Product Design, NTU; Professor David Crundall, Psychology, NTU; Dr Mike Vernon, Psychology, NTU; Dr Bill Byrom, Signant Health and Dr Chris Griffiths, Northampton Hospitals NHS Foundation Trust.

Making a Difference

Existing apps/systems provide mental health management tools; these are solely reliant on information input by the patient. Other face/emotion tracking systems exist but don’t focus on mental health solutions. EMOTI captures objective measures that can be analysed statistically by capturing facial/emotion/acoustic data and provide validated results demonstrating a step change in patient capture data analysis and addressing a gap in the market. Successful implementation of EMOTI could allow the application to become an essential part of the depression/mental health service provision. The initial impact achieved via the EMOTI application will be the ability to screen for risk of relapse from treatment and provide an early alert system allowing psychotherapists to be alerted at early stages of relapse, allowing for urgent action. Currently no method exists in NHS practice for daily screening and automated reporting of a patient’s emotional states from home.