Quality of Representations - Deepstash
Quality of Representations

Quality of Representations

Good representations are expressive and make efficient use of the given dimensionality.

We want the representations to be variant to contextual changes that are essential to a task and invariant to changes related to factors that we cannot control nor care about. 

While these invariances can largely be enforced through task-specific data augmentation, the efficient use of the dimensionality of the representations has to be achieved through clever algorithms.

One way to look at this quality is through the variance of the representations, which shall not collapse for a given class of inputs. 

7

37 reads

CURATED FROM

IDEAS CURATED BY

laurenz

Deep Learning, Software Development, Self-Improvement

The idea is part of this collection:

Centers of Progress

Learn more about artificialintelligence with this collection

The historical significance of urban centers

The impact of cultural and technological advances

The role of urban centers in shaping society

Related collections

Similar ideas to Quality of Representations

VICReg

VICReg

While many recently proposed self-supervised learning algorithms prevent a collapse of the embedding-space implicitly through various methods like contrasting samples in a batch [SimCLR] or clustering [

The 4 Requirements of a Change Leader

  • Institute policies to shape the future, including the use of “organized abandonment” and “organized improvement” to keep directing your resources to the top opportunities.
  • Use systematic innovation by regularly scanning the environment, and proactively converting changes to opportu...

Productivity: The 24 Hours Of The Day

Productivity: The 24 Hours Of The Day

  • We all have the same number of hours given to us for a day. It’s up to us how we use those hours.
  • Productivity means simplifying our work, getting it done in a faster, smarter and efficient way, with the idea of the saved time being used for doing ...

Read & Learn

20x Faster

without
deepstash

with
deepstash

with

deepstash

Personalized microlearning

100+ Learning Journeys

Access to 200,000+ ideas

Access to the mobile app

Unlimited idea saving

Unlimited history

Unlimited listening to ideas

Downloading & offline access

Supercharge your mind with one idea per day

Enter your email and spend 1 minute every day to learn something new.

Email

I agree to receive email updates