QU develops intelligent crowd control systems for FIFA World Cup Qatar 2022™

Qatar University Research Team

In collaboration with the Supreme Committee for delivery and Legacy, Qatar University (QU) College of Engineering has developed intelligent crowd management and control systems, including multiple components for crowd counting, face recognition, and abnormal event detection (AED).

The QU research team, led by Prof. Sumaya Al-Maadeed as the lead principal investigator in the study, includes Dr. Noor Al-Maadeed, Associate Dean of Graduate Studies for Academic Affairs and Associate Professor of Computer Engineering, Dr. Khalid Abualsaud, Lecturer of Computer Engineering, Prof. Amr Mohamed, Professor of Computer Engineering, Prof. Tamer Khattab, Professor of electrical engineering and Acting Director of Excellence in Teaching and Learning Center, Dr. Yassine Himeur, Post-doctoral Researcher, and Dr. Omar Elharrouss. Post-doctoral Researcher, Najmath Ottakath, Master's student.

The security and safety of FIFA World Cup Qatar 2022 players, spectators, and other people present at the event is at the center of the attention of the Organization Committee. Typically, security risks increase multi-fold, considering the large scale of the event and the significant number of fans expected to attend it (more than 1.5 million fans). Thus, the security of the FIFA World Cup Qatar 2022 is challenging due to the increasing number of possible threats and use of technology.

Using edge technologies, such as AI, drone-based computer vision, and ICT provides promising perspectives to ensure the security and safety of the players, spectators, and other people attending the event and helps efficiently manage crowds and identify people and abnormal events.

This project aims to use AI and drone-based video surveillance to optimize crowd management and control in FIFA World Cup 2022 sporting facilities. Typically, the team’s effort focuses on (i) ensuring stadium safety and security plans where the fans do not need to think about their security, (ii) detecting abnormal events, (iii) tracking and recognizing suspicious persons using face recognition tools, (iv) maintaining the privacy of the fans by using edge-computing platforms where recorded data are processed locally.

Crowd management at the World Cup stadiums and their perimeters is crucial to ensure the safety and smoothness of the World Cup events due to the inherent occlusion and density of the crowd inside and outside the stadiums. FIFA World Cup 2022 will rely on the deployment of cutting-edge technologies, such as surveillance drones, ICT, and AI, to optimize crowd management. In this respect, the QU research team has first developed a crowd counting system from drones’ data, which exploits the dilated and scaled neural networks to extract pertinent features and density crowd estimations.

Additionally, a new dataset for crowd counting in sports facilities named Football Supporters Crowd Dataset (FSC-Set) is introduced. It includes 6000 images labeled manually and representing various types of scenes, containing thousands of people gathering in or around the stadiums.

The research team’s effort has also focused on developing a face recognition system, which considers faces under pose variations using a multitask convolutional neural network (CNN). Specifically, a cascade structure was employed to combine a pose estimation approach and a face identification module. The CNN-based pose estimation approach has been trained on three categories of face images, including left side, frontal, and right side captures. Moving on, three CNN architecture, namely VGG-16+PReLU left, VGG-16+PReLU front, and VGG-16+PReLU right, have been deployed to identify faces based on the estimated pose. Additionally, a skin-based face segmentation scheme, based on structure-texture decomposition and a color-invariant description, has been introduced to remove useless face information (e.g., background content). Empirical evaluations have been conducted on four popular face recognition datasets, where the proposed system has outperformed related state-of-the-art schemes.

Recently, using drone-based video surveillance, abnormal event detection (AED) is receiving increasing attention due to its reliability and cost-effectiveness. Typically, drones augmented with cameras can spot violent behaviors in crowds during sports events. They can monitor crowds in the perimeter of stadiums and/or other public venues during the FIFA World Cup 2022. To that end, the research team, led by Prof. S. Al-Maadeed, has developed a novel AED, which aims at learning abnormal actions using both normal and abnormal segments. It enables to avoid the annotation of anomalous events in training video sequences to reduce the computational cost and hence be easily implemented on drones.

Therefore, abnormal events are learned using a deep multiple instance ranking scheme, which leverages weakly annotated training video sequences. Put simply; training annotations are put on whole videos instead of specific clips.

Source: Qatar University