VFRAME (Visual Forensics and Metadata Extraction) is a collection of open source computer vision tools designed specifically for human rights investigations that rely on large video datasets.
Human rights researchers often rely on videos shared online to document war crimes, atrocities, and human rights violations. Manually reviewing these videos is expensive, does not scale, and can cause vicarious trauma. As an increasing number of videos are posted, a new approach is needed.
VFRAME is currently working with Syrian Archive, an organization dedicated to documenting war crimes and human rights violations, to develop computer vision tools to address these challenges.
Specifically VFRAME is developing tools to detect evidence of illegal munitions including A02.5-RT/M, ShOAB-0.5, PTAB-1M; filter graphic content to reduce vicarious trauma; efficiently search for related visual media in large datasets (over 1M videos); and a web annotation platform to construct custom datasets for training computer vision models.
Throughout development we will document the process and release a report along with open-sourced code in Fall 2018.
Below are previews of the current tools in development.
Object detection is the ability to classify and localize an object in a video frame or image. Object detection algorithms can be trained to detect custom objects, such as the AO-2.5RT cluster munition seen in these videos.
The AO-2.5RT/RTM is a specific type of cluster munition that appears frequently in videos from the Syrian conflict. The AO-2.5RT/RTM is banned by the Convention on Cluster Munitions (CCM), which "prohibits the use, transfer, and stockpiling of cluster bombs, a type of explosive weapon which scatters submunitions ('bomblets') over an area."
Researchers at Syrian Archive have already identified evidence of this munition in dozens of videos. But manually reviewing videos does not scale. Currently there are over 1.500.000 videos waiting to be reviewed. More videos contaning evidence of the AO2.5RT/RTM may still exist in these unreviewed videos.
Using computer vision to automatically detect cluster munitions will significantly accelerate researcher's work to locate evidence of cluster munitions in this large dataset.
This project is under active development. Code will be released in October and demos in November 2018.