CLIP is a gigantic leap forward, bringing many of the recent developments from the realm of natural language processing into the mainstream of computer vision: unsupervised learning, transformers, and multimodality to name a few. The burst of innovation it has inspired shows its versatility.
And this is likely just the beginning. There has been scuttlebutt recently about the coming age of "foundation models" in artificial intelligence that will underpin the state of the art across many different problems in AI; I think CLIP is going to turn out to be the bedrock model for computer vision.
MORE IDEAS ON THIS
Unfortunately, for many hyper-specific use-cases (eg examining the output of microchip lithography) or identifying things invented since CLIP was trained in 2020 (for example, the unique characteristics of CLIP+VQGAN creations), CLIP isn't capable of performing well out of the box for all problem...
We've used CLIP along with GANs to convert text into images; there's no reason we can't go in the other direction and create rich captions for images with creative usage of CLIP (possibly along with a language model like GPT-3).
As an extension of image similarity, we've used CLIP to track objects across frames in a video . It uses an object detection model to find items of interest then crops the image and uses CLIP to determine if two detected objec...
Because CLIP doesn't need to be trained on specific phrases, it's perfectly suited for searching large catalogs of images. It doesn't need images to be tagged and can do natural language search.
Yurij Mikhalevich has already created
This is in contrast to traditional computer vision models which disregard the context of their labels (in other words, a "normal" image classifier works just as well if your labels are "cat" and "dog" or "foo" and "bar"; behind the scenes it just converts them into a numeric identifier with no pa...
OpenAI originally evaluated CLIP as a zero-shot image classifier. They compared it against traditional supervised machine learning models and it performed nearly on par with them without having to be trained on any specific dataset.
One challenge with traditional approaches to image classi...
Apple's Neuralhash semantic image similarity algorithm has been in the news a lot recently for how they're applying it to scanning user devices for CSAM. We showed h...
#engineering, #machinelearning and #crypto
More like this
Explore the World’s
Take Your Ideas
Just press play and we take care of the words.
No Internet access? No problem. Within the mobile app, all your ideas are available, even when offline.
2 Million Stashers
Don’t look further if you love learning new things. A refreshing concept that provides quick ideas for busy thought leaders.
Best app ever! You heard it right. This app has helped me get back on my quest to get things done while equipping myself with knowledge everyday.
This app is LOADED with RELEVANT, HELPFUL, AND EDUCATIONAL material. It is creatively intellectual, yet minimal enough to not overstimulate and create a learning block. I am exceptionally impressed with this app!
Great interesting short snippets of informative articles. Highly recommended to anyone who loves information and lacks patience.
Great for quick bits of information and interesting ideas around whatever topics you are interested in. Visually, it looks great as well.
I have only been using it for a few days now, but I have found answers to questions I had never consciously formulated, or to problems I face everyday at work or at home. I wish I had found this earlier, highly recommended!
Even five minutes a day will improve your thinking. I've come across new ideas and learnt to improve existing ways to become more motivated, confident and happier.
Brilliant. It feels fresh and encouraging. So many interesting pieces of information that are just enough to absorb and apply. So happy I found this.
Read & Learn
Access to 200,000+ ideas
Access to the mobile app
Unlimited idea saving & library
Unlimited listening to ideas
Downloading & offline access
Supercharge your mind with one idea per day
Enter your email and spend 1 minute every day to learn something new.
I agree to receive email updates