Multimodal Human Analysis for Predicting Potential Threat

Project ID: SST_2_9

Violent crimes and possession of weapons in prohibited areas are still major issues in the world today. According to research, weapons are used in approximately a quarter of all violent incidents and most homicides in the US and the UK, with the most frequent weapon of choice being either firearms or knives depending on the region. Also, self-harm by individuals is another aspect of crime on the rise. This provides powerful motivation to research towards the development of advanced technological solutions to detect and prevent incidents and provide safety to the public.

However, despite many advances in technology, the development of systems capable of predicting a potentially harmful person before they commit a crime remains a challenge. This stems mainly from the limitations associated with the data available and predictive Artificial Intelligence (AI)/Computational Intelligence (CI) models. Research has shown that using computer vision information for the analysis of facial features, e.g., use of facial action unit detectors, is useful in predicting a person’s emotions. However, this is not sufficient for predicting their intents and subsequent actions.

This project will focus on the research development of a framework for human intent/action prediction using computer vision information. The project will combine different modes of information such as facial features, motion, and posture of humans in analysing intent and predicting actions to determine a potential threat/crime in public spaces. Emphasis will be on the AI/CI models to be developed to enhance the accuracy of the framework. Several theoretical tools and models exist for predictive analysis such as probabilistic models and deep learning. However, these have limitations when handling multimodal information, hence, the need for further research to develop a robust model.

The objectives of the proposed study are:

1. Investigate and compare available solutions for multimodal human action analysis.
2. Design the data collection framework needed for extraction of multimodal data for human analysis using computer vision devices, etc.
3. Design the framework using advanced AI/CI techniques for the analysis, detection, and prediction of patterns of human actions.
4. Implement and test the framework, while analysing its performance with closely related existing solutions.
5. Investigate the commercial suitability of deploying the research development to real-world scenarios.

The project will be expected to contribute significantly to the advancement in applied AI/CI in safety and security applications, as well as publication of high-quality research papers. The project team have expertise and vast experience in developing AI solutions used in human behaviour monitoring and understanding. References to their works can be found here https://scholar.google.co.uk/citations?user=uRybDEcAAAAJ&hl=en.

Supervisory Team:

Dr David Adama (Director of Studies) [ECR] - David Adama | Nottingham Trent University

Dr Doratha Vinkemeier (Co-Supervisor) [ECR] - Doratha Vinkemeier | Nottingham Trent University

Prof. Ahmad Lotfi (Co-Supervisor) - Ahmad Lotfi | Nottingham Trent University

The proposed work is closely related to Dr David Adama’s expertise in human behaviour monitoring and analysis using computational intelligence methods, for which he has published internationally recognised research papers. Dr Adama’s work is in this area can be found here https://scholar.google.co.uk/citations?user=uRybDEcAAAAJ&hl=en.