VICReg

While many recently proposed self-supervised learning algorithms prevent a collapse of the embedding-space implicitly through various methods like contrasting samples in a batch [SimCLR] or clustering [SwAV], VICReg explicitly regularizes the embedding-space with three terms:

• The Variance of the embeddings should be above a given threshold.
• Euclidian distance between embeddings of different views of the same image enforces Invariance to given augmentations.
•  Minimizing the Covariance of the embeddings decorrelates the components, thus efficiently making use of the given space.