VFRAME partners with Tech 4 Tracing to gain insight and expertise into the munitions used in conflict zones in order to create 3D models and capture sample data for developing and enhancing computer vision object detection technologies.
Tech 4 Tracing (T4T) is an international, US-based non-profit working towards building partnerships with human rights investigation communities to enhance and improve their abilities to understand and address illicit arms flows, bring perpetrators of armed violence to justice, and contribute to safer societies.
During the winter/spring of 2022 VFRAME and T4T partnered to develop a 3D model of the 9N235 submunition that has frequently appeared in the war in Ukraine. This model is being used to create synthetic training data for object detection algorithms and high-accuracy 3D printed replicas for benchmarking purposes.
The 3D models are used to render synthetic training datasets for developing object detection algorithms. The 3D environment can be manipulated to change the lighting conditions, camera zoom, alter the terrain, or simulate dirt and corrosion. Rendered synthetic data can be automatically annotated, saving huge amounts of time compared to conventional, manual image annotation.
Sample video showing a simple 3D rendered scene with the 9N235 submunition
After training on the 3D rendered synthetic data, the object detection algorithm needs to be tested on real world data. However, gaining real images of munitions can be difficult due to the dangerous circumstances where they appear. Or they may be scarce, with only a few dozen images available. Ideally the algorithm should be tested on thousands of diverse images.
To overcome this challenge, VFRAME produces photo-realistic replicas using 3D printing. The painted replicas are placed in staged scenes to simulate the conditions of target including weather, lighting, and terrain. These staged images are then used as benchmark data to evaluate how well the object detection algorithms work. Below are several early examples of the (in progress) 9N235 3D printed replica.
The 3D print is painted to match the metallic properties of the original object, stencils are applied, and then the object is placed in settings to simulate the real terrain. Here the object is covered in dirt and placed in a grassy terrain.
Together, the 3D rendered and 3D printed data sources enable scalable development of highly specific, dangerous, and rare objects such as cluster munitions. This approach overcomes the limitation of data scarcity and could enable a workflow for developing computer vision algorithms to detect virtually any visually unique munition.
In September VFRAME will release a public version of the 9N235 submunition object detection neural network based on the methods described above. The detector will be open sourced under the MIT license. Please check back on VFRAME.io on September 1 for more details and a web demo.