Cooperative Perception in Robotic Teams for a Sustainable Environment

Project ID: SST_2_10

Sustainability in agriculture and forestry is currently an important societal issue, as was made clear at the 2021 UN Climate Change Conference. In fact, two of the main goals of the conference include curtailing deforestation and making agriculture more resilient. As for the former, wildfires have become a major problem in the past few decades, as climate change has pushed the geography of potential wildfire outbreaks to a broader range of regions in the world, while as for the latter, the intensive use of water and chemicals have crippled the sustainability of arable land. The lack of specialised labour willing to work in both industries has further compounded these issues, making the need for automation, such as the introduction of robotic solutions, an inevitability.

Despite many advances in robotics, the development of fully autonomous robotic solutions for precision agriculture and forestry is still in a very early stage. This stems from many different challenges, but in particular due to limited perception capabilities, and reasoning and planning under a high level of uncertainty. Artificial perception for robots operating in outdoor natural environments has been studied for several decades. Nevertheless, despite many years of research, as described in surveys over time, a substantial number of problems have yet to be robustly solved. An even greater challenge presents itself when considering Multi-Robot Systems in this context. Cooperative robotic perception requires that multiple robots autonomously and collaboratively extract semantic information from multimodal data collected over wide areas to build a globally consistent probabilistic semantic map. These robots must perform cooperative active perception, by using the current model to autonomously decide under uncertainty the next target viewpoints, either to explore unknown areas or to update information of previously explored areas. This also requires that robots coordinate their individual actions to optimise team performance in its collective task.

This project will focus on the research and development of a distributed framework for cooperative perception for robotic teams for a sustainable environment. The objectives of the proposed study are to tackle cooperative active perception in teams of unmanned terrestrial and aerial robots operating in large outdoor areas (e.g. forests and agricultural areas), which involve the combination of data from 3D LiDAR sensors with different vision sensors, including RGB, stereo and multispectral cameras. A deep learning approach will be used to extract domain-specific information required to perform cooperative perception. Overall, this project is expected to significantly contribute to artificial intelligence, cognitive robotics, and automation in precision forestry and agriculture.

This project is suited for candidates that are concerned about climate change and a sustainable future and wish to make a difference, who have a background in computing/computer science, and who are excited about the field of robotics.

Supervisory Team:

Dr. João Filipe Ferreira -- https://www.ntu.ac.uk/staff-profiles/science-technology/joao-filipe-ferreira

Dr. David Adama -- https://www.ntu.ac.uk/staff-profiles/science-technology/david-adama

Prof. Ahmad Lotfi -- https://www.ntu.ac.uk/staff-profiles/science-technology/ahmad-lotfi