Variance-Invariance-Covariance Regularization for Self-Supervised Learning - Deepstash

Bite‑sized knowledge

to upgrade

your career

Ideas from books, articles & podcasts.

published 3 ideas

Variance-Invariance-Covariance Regularization for Self-Supervised Learning

arxiv.org

Self-Supervised Learning

Labelled data is expensive, which makes benefiting from the current success in supervised learning unfeasible for smaller companies.

However, good representations can be learned without any task-specific information from raw data.

In self-supervised learning, labels are generated art...

4

STASHED IN:

6

Quality of Representations

Good representations are expressive and make efficient use of the given dimensionality.

We want the representations to be variant to contextual changes that are essential to a task and invariant to changes related to factors that we cannot control nor care about. 

While these inv...

1

STASHED IN:

6

VICReg

While many recently proposed self-supervised learning algorithms prevent a collapse of the embedding-space implicitly through various methods like contrasting samples in a batch [SimCLR

1

STASHED IN:

6

0 Comments

Discover and save more ideas by creating a

FREE

Deepstash account.

Develop a

reading habit

, save

time

and create an amazing

knowledge library

.

GET THE APP: