Check out our implementation at https://github.com/untitled-ai/slot_attention.
Here’s a question for you, intelligent reader: how many distinct objects do you see in the upper-left image?
This is not a hard question for us humans. However, unsupervised machine learning models have a surprisingly difficult time answering this: they struggle to understand which pixels constitute an object by human standards. And while there's been success in training supervised models to complete this task 1, it’s largely unsolved on the unsupervised side 2.
You can see the issue here. It's important that machines actually understand what objects are, because countless elementary tasks that humans can do require this basic capability.
Enter Object-Centric Learning with Slot Attention. This nifty technique introduces a novel way to extract objects from the original input without any training or supervision. After an image is processed (for example, through a CNN) into feature representations, a Slot Attention module forces "slots" to compete over these features, using an iterative attention mechanism similar to the attention mechanism popular in other domains of machine learning 3. At each iteration, slots compete to explain parts of input features, and then update their representation.
It’s exciting that unsupervised models can now determine objects from complex scenes, and can even do so without using depth, motion, or other helpful signals.
While slot attention is a step in the right direction toward unsupervised object understanding, there are plenty of interesting open questions:
By reimplementing this paper and sharing our code, we hope we can allow others to help answer these questions, and more.
Note: We did not fully reproduce the result of the paper due to time and GPU constraints, so drop a ticket on our Github page if you have any issues. If you have thoughts on the model, where you think it’d be useful, etc., please shoot us a note at firstname.lastname@example.org. We're happy to chat!
Object-Centric Learning with Slot Attention, https://arxiv.org/abs/2006.15055.
Slot Attention in Tensorflow, https://github.com/google-research/google-research/tree/master/slot_attention.