New technology tracks eye gaze and body pose to help pupils engage in learning
New technology which tracks the eye gaze and body pose of children to understand and improve their engagement with learning has been designed in collaboration with Nottingham Trent University (NTU).
The innovative digital system, calledMaTHiSiS, understands the emotional state of the student, such as whether they are engaged, frustrated or bored. It also assesses the student’s skill level and presents them with personalised electronic materials to continue their learning.
Part of a European-wide project, the UK study has focused on students with learning disabilities and autism. NTU was the lead partner in understanding the requirements of these students so that their needs could be factored into the design of the system, such as any visual, physical or cognitive impairments they might experience, and their learning goals.
The University also had a significant input into computer methods designed to understand the emotional state of learners. These include tracking their interaction with the system, such as their mouse movements, or patterns and speed of typing, and relating this to whether they are engaged or not.
MaTHiSiS challenges preconceived ideas of the abilities of students with a learning disability and can reveal skills, knowledge and abilities that teachers had not otherwise understood or realised.
The UK partners, including NTU, University of Nottingham and University of East London, worked together to evaluate the results of pilots which took place in schools across Italy, Spain and the UK.
David Brown, Professor of Interactive Systems for Social Inclusion at Nottingham Trent University, said: “The MaTHiSiS system moves beyond current practice in using AI in Education where learner-facing tools use formative assessments and diagnostic tests to personalise the learning experience. We also seek to understand the emotional state of learners as this will have a great impact on the quality of their learning experience.”
The development of the MaTHiSiS system relied heavily on the input of teachers, teaching assistants and students from several European countries so that the end product was something the teachers and students could benefit from.
Funded by the Horizon 2020 Framework Programme of the European Union and developed in collaboration with 18 European partners, the study involved 34 teachers and 67 children between the ages of 6 – 16 who all took part in analysing and testing the efficiency of the system.
Findings from the pilot showed that teachers frame MaTHiSiS as a change in their teaching practice and compared it with the introduction of other ground-breaking technologies. They felt that MaTHiSiS will free them up to be able to give more support and encouragement to the learner and target their support to those learners who needed it most in a busy classroom where one to one support is at a premium.
Gosia Kwiatkowska, acting director of the RIX Centre, University of East London, said: “We are getting more diverse students in our classrooms and that requires diverse teaching. Our project addressed the real issue of teachers not having enough capacity to attend to each child’s individual learning needs and as a result some of the students are left out and are not supported to develop their full potential.
“A system that can detect a learner’s state of flow can provide a wonderful addition to any classroom. I can see that in the future there will be lots of personalised technologies available to teachers and students which will make learning much more effective, engaging and fun. The addition of tailored learning materials mean that students can continue their schoolwork at home.”
Professor of Health Psychology and Learning Disabilities at the University of Nottingham, Penny Standen, said: “MaTHiSiS is successful at monitoring the emotional state of the learner, especially their level of engagement. The significance of this is that engagement is considered to be the single best predictor of learning for children with intellectual disabilities.
“The sensitivity with which it monitored the learner’s progress, made it particularly suited to learners with special needs where achievements are made in small steps so aren’t always picked up by the monitoring derived from national curriculum levels.”
MaTHiSiS is for tutors, parents or caregivers, learners and teachers.
The MaTHiSiS partners are currently investigating ways of rolling out the system for use in education and industrial training. They are also proposing ways in which the system can be used as a research tool in other projects, and extending the methods developed in this project for use in other AI Tools in Education to predict the engagement of pupils with a learning disability to provide personalised learning pathways.