Project details
- Funder: Defense Advanced Research Projects Agency (DARPA)
- Program: Media Forensics (MediFor)
- Year of activity: 2016 - 2020
Abstract
The FENCE project aimed at designing a set of reliable forensic tools based on the visual analysis of image physical properties, to detect and locate several kinds of image manipulations working in a completely automatic way. The core technical challenge of FENCE was to build a computational architecture capable of assessing the physical integrity of an image included in a very large dataset, with no manual input or any other a priori information about its visual content.
The FENCE framework
Any forensic software for physical integrity verification can be seen as composed of two main steps: the extraction of some characteristics from the investigated image and the subsequent integrity assessment. The key idea is that the image physical analysis can be structured into four sequential layers, namely:
- Scene Classification - the image is classified according to specific properties of the scene and light source (e.g., outdoor/indoor, seaside/city, …).
- Characteristic Detection - salient parts of the image are detected/localized (e.g., faces, buildings, lines, ….).
- Integrity Indicator - physical image clues are extracted from the detected characteristics and then aggregated into an integrity indicator.
- Integrity Score – integrity score is computed on the basis of the integrity indicator and other available information.
Each of the first three layers of the framework produces, in addition to the expected output, an uncertainty indicator (UI), that provides a quantitative measure of the reliability of the each output and possibly an explanation of the factors that influenced the result. In the last layer, the UIs are aggregated together with the integrity indicator, in order to compute the integrity score and corresponding report. The proposed framework carries out the performance evaluation of the overall process in a very flexible way, allowing the result analysis even at layer level. For instance, the evaluation of a module can be conducted by switching all of the available algorithms implemented for a given layer, thus facilitating the understanding of the influence of an algorithm on the following steps or on the overall system performance.
Research Team
Principal investigator: Prof. Alessandro Piva
LESC Members
- Dr. Daniele Baracchi
- Dr. Alessia De Rosa
- Dr. Marco Fontani
- Dr. Massimo Iuliani
- Dr. Dasara Shullani
DINFO Members
- Prof. Carlo Colombo
- Dr. Marco Fanfani
- Dr. Fabio Pazzaglia
Publications
- Massimo Iuliani, Dasara Shullani, Marco Fontani, Saverio Meucci, and Alessandro Piva, "A Video Forensic Framework for the Unsupervised Analysis of MP4-Like File Container.", IEEE Transactions on Information Forensics and Security, 2019
- Massimo Iuliani, Marco Fanfani, Carlo Colombo, and Alessandro Piva, "Reliability assessment of principal point estimates for forensic applications.", Journal of Visual Communication and Image Representation, 2017
- Dasara Shullani, Omar Al Shaya, Massimo Iuliani, Marco Fontani, and Alessandro Piva, "A Dataset for Forensic Analysis of Videos in the Wild.", TIWDC, 2017
- Omar Al Shaya, Pengpeng Yang, Rongrong Ni, Yao Zhao, and Alessandro Piva, "A New Dataset for Source Identification of High Dynamic Range Images.", Sensors, 2018
- Massimo Iuliani, Marco Fontani, Dasara Shullani, and Alessandro Piva, "Hybrid reference-based Video Source Identification.", Sensors, 2019
- Benjamin Hadwiger, Daniele Baracchi, Alessandro Piva, and Christian Riess, "Towards Learned Color Representations for Image Splicing Detection.", IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019
- Marco Fanfani, Fabio Bellavia, Massimo Iuliani, Alessandro Piva, and Carlo Colombo, "FISH: Face intensity-shape histogram representation for automatic face splicing detection.", Journal of Visual Communication and Image Representation, 2019
- Fabio Bellavia, Massimo Iuliani, Marco Fanfani, Carlo Colombo, and Alessandro Piva, "Prnu Pattern Alignment for Images and Videos Based on Scene Content.", ICIP, 2019
- David Vázquez-Padín, Marco Fontani, Dasara Shullani, Fernando Pérez-González, Alessandro Piva, and Mauro Barni, "Video Integrity Verification and GOP Size Estimation Via Generalized Variation of Prediction Footprint.", IEEE Transactions on Information Forensics and Security, 2020
- Marco Fanfani, Massimo Iuliani, Fabio Bellavia, Carlo Colombo, and Alessandro Piva, "A vision-based fully automated approach to robust image cropping detection.", Signal Processing: Image Communication, 2020
- Pengpeng Yang, Daniele Baracchi, Massimo Iuliani, Dasara Shullani, Rongrong Ni, Yao Zhao, and Alessandro Piva, "Efficient Video Integrity Analysis Through Container Characterization.", IEEE Journal of Selected Topics in Signal Processing, 2020
- Massimo Iuliani, Marco Fontani, and Alessandro Piva, "A Leak in PRNU Based Source Identification - Questioning Fingerprint Uniqueness.", IEEE Access, 2021