Project details
- Funder: Ministry of Universities and Research (MUR)
- Program: Progetti di Ricerca di Interesse Nazionale (PRIN) 2017
- Year of activity: 2020 - 2024
Abstract
To goal of PREMIER is to devise a new class of techniques capable of distinguishing fake from original videos. The new techniques will be developed by following a hybrid approach whereby forgery detectors based on deep learning are coupled with model-based methods, so to cope with the problems raising when deep learning is used within a multimedia forensics scenario. Looked-for solutions will aim at decreasing the amount of data needed for training, at providing ways to interpret the results of the forensic analysis, and at improving the security of the developed tools in the presence of an informed forger aiming at evading fake media detection.
The PREMIER framework
The scientific goals of PREMIER pertain to advances in the field of video forensics. Videos strongly contribute to the viral diffusion of information through web channels, and can play a fundamental role in the digital life of individuals and societies. The forensic analysis of videos therefore represents a crucial tool to preserve the dependability of digital media.
Although promising approaches have been explored in the latest years, research in video forensics is much less advanced than for still images and the proposed solutions suffer from several shortcomings. This includes the need of large amounts of data for training, the difficulty of interpreting the outcoming results, and the vulnerability to adversarial attacks, which particularly affect AI-based techniques employing deep neural networks.
PREMIER will pursue novel hybrid approaches whereby data-driven AI techniques are enriched with a model-based signal-processing view point, with the goal of obtaining forensic detectors that are more interpretable and secure. The research activity is organized in several workpackages, addressing the different project goals and closely interacting through the development of the project.
WP1: Model-based solutions for AI in data-limited conditions The use of model-based priors is explored with the goal of relaxing the need of large amounts of training data and orienting the training process towards task-relevant features. This includes applying smart pre-processing and regularization strategies, and studying conditions of training/testing data mismatching.
WP2: Interpretability tools for AI-based detectors AI-based detectors are enriched by tools for the visualization of the deep network outcomes, such as temporal and spatial heat-maps quantifying the importance of single pixels and areas. This eases the interpretation of the learning process and allows for the informed design of new architecture based on signal processing concepts.
WP3: Authenticity and integrity verification in adversarial settings The security of AI-based detectors is assessed against malicious actions performed by attackers. Different threat models are analyzed, which define the possible interplay between attacker and defender. Countermeasures to be adopted either at training or inference phase are developed, with the goal of increasing security.
WP4: Data collection for authenticity and integrity verification of visual data The need of representative data for training and testing the hybrid detectors is supported by a continuous collection of datasets composed of original and fake videos. This will aid the development of enhanced solutions and provide the scientific community with extensive data corpora for benchmarking.
WP5: Data collection for authenticity and integrity verification of visual data A dissemination and raising awareness plan is implemented to promote the project activities among the scientific audience and the general public.
This includes holding presentations/demos at scientific conferences, co-organizing workshops and special sessions at established events, and co-editing peer review journals. Consortium members also regularly hold training lectures and participate in dissemination events and panels, recognized by local and national press. Moreover, a web-based demonstrator showcasing the developed forensic tools is designed throughout the project.
Research Team
Principal investigator: Prof. Mauro Barni (University of Siena)
LESC Members
Publications
- Pengpeng Yang, Daniele Baracchi, Massimo Iuliani, Dasara Shullani, Rongrong Ni, Yao Zhao, and Alessandro Piva, "Efficient Video Integrity Analysis Through Container Characterization.", IEEE Journal of Selected Topics in Signal Processing, 2020
- Sara Mandelli, Fabrizio Argenti, Paolo Bestagini, Massimo Iuliani, Alessandro Piva, and Stefano Tubaro, "A Modified Fourier-Mellin Approach For Source Device Identification On Stabilized Videos.", ICIP, 2020
- Massimo Iuliani, Marco Fontani, and Alessandro Piva, "A Leak in PRNU Based Source Identification - Questioning Fingerprint Uniqueness.", IEEE Access, 2021
- Chiara Albisani, Massimo Iuliani, and Alessandro Piva, "Checking PRNU Usability on Modern Devices.", IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021
- Daniele Baracchi, Dasara Shullani, Massimo Iuliani, Damiano Giani, and Alessandro Piva, "Camera Obscura: Exploiting in-camera processing for image counter forensics.", Forensic Science International: Digital Investigation, 2021
- Pasquale Ferrara, Massimo Iuliani, and Alessandro Piva, "PRNU-Based Video Source Attribution: Which Frames Are You Using?", Journal of Imaging, 2022
- Dasara Shullani, Daniele Baracchi, Massimo Iuliani, and Alessandro Piva, "Social Network Identification of Laundered Videos Based on DCT Coefficient Analysis.", IEEE Signal Processing Letters, 2022
- Sebastiano Verde, Cecilia Pasquini, Federica Lago, Alessandro Goller, Francesco G. B. De Natale, Alessandro Piva, and Giulia Boato, "Multi-Clue Reconstruction of Sharing Chains for Social Media Images.", IEEE Transactions on Multimedia, 2023
- Simone Magistri, Daniele Baracchi, Dasara Shullani, Andrew D. Bagdanov, and Alessandro Piva, "Towards Continual Social Network Identification.", International Workshop on Biometrics and Forensics (IWBF), 2023
- Daniele Baracchi, Dasara Shullani, Massimo Iuliani, and Alessandro Piva, "FloreView: An Image and Video Dataset for Forensic Analysis.", IEEE Access, 2023