Permutation Invariance of GCNs

#1

Hi,

I was thinking about the issue of the GCN with respect to the permutation of the atoms.
Most GCNs (for example teh simple AXW propagation) are not invariant to permutation. Different ordering of atoms leads to other activation which could in theory produce a different output.

Most of cases this is avoided because we can use some sort of canonicalization of the molecules which makes sure that the molecule is always represented in the same manner.
However, when molecules have a similar stuctures but completely different canonicalization they will have a different ordering and thus the activation will differ greatly across the network.

Most GCNs are still performing really well on chemical datasets. Am I missing something with regards to the invariance of the models or do they perform so well despite the issue of invariance? Because despite the invariance the non-linear layers allow the model to map precisely molecules to properties.
Would,if problems of invariance/equivariance , lead to even better models?

Maybe this is complete gibberish what I am writing, but I was thinking the whole day about this issue. Could be that I am already overthinking it. But maybe someone has some thoughts to share. Or has something to point out where I went wrong.

1 Like
#2

We currently have a canonicalization hack in the DeepChem loader class that ensures we canonicalize the SMILES strings (https://github.com/deepchem/deepchem/blob/master/deepchem/data/data_loader.py#L68).

I think you’re making an important point here that we should do a better job of handling equivariance/invariance in the models. This could perhaps be a data augmentation technique we add that could help models learn invariance themselves. This might be a really interesting technique to experiment with

#3

I think most GCN rely on some sort of canonicalization to avoid the issue of equivariance (or the lack thereof). The more I read about it, the more I am surprised that GCNs work as well as they do on chemical data.

There is a really interesting paper https://arxiv.org/pdf/1812.09902.pdf which proposes a permutation invariant model and is also applied to some chemical datasets. Unfortunately none of which are included in DeepChe.

1 Like
#4

By construction, GCNs are permutation equivariant, as permuting the input node features (and the adjacencies) leads to a permuted node feature matrix at every layer. However, most models utilize a permutation invariant pooling layer, like a sum or max, to obtain graph features, which makes the entire construction permutation invariant. This is also mentioned by Thomas Kipf here.

If you think about it, this is equivalent to the translation equivariance that CNNs offer by construction, except the presence of invariant or equivariant layers across image classification and object detection.

There is really no need of a canonicalization to the best of my knowledge. If you find sources doing so, I’d be happy to check them out.

2 Likes
#5

Hi thats true you are absolutely right. I realized that I have made a mistake in my thinking.

2 Likes