MIT and Adobe Develop New Machine Learning Technique for Object Identification

Current material selection methods struggle to accurately identify all pixels representing the same material.

Some methods focus on the entire object, which can consist of any number of materials, while others use broad labels for predefined sets of materials.

The MIT and Adobe researchers’ technique dynamically evaluates all the pixels in an image to determine material similarity, even taking into account the shape of objects and varying lighting conditions.

To overcome the challenge of a shortage of finely labeled datasets to train their machine learning models, the researchers created their own indoor scene synthetic dataset. It consists of 50,000 images with more than 16,000 materials randomly applied to objects.

However, when they first tested their model on real images, it didn’t perform very well due to a distributional shift between real-world and synthetic data. To solve this, they used trained computer vision models that had learned from millions of real images, incorporating prior knowledge into their models.