How to Use Massive AI Models (Like GPT-3) in Your Startup
Curated from: future.a16z.com
Ideas, facts & insights covering these topics:
5 ideas
·462 reads
4
Explore the World's Best Ideas
Join today and uncover 100+ curated journeys from 50+ topics. Unlock access to our mobile app with extensive features.
The journey to this point began a little more than 9 years ago with a 2012 submission, called AlexNet, to the annual ImageNet LS
- Within a year, startups began springing up to replicate AlexNet
- Fast forward to 2022: Neural networks have changed every aspect of machine intelligence in software systems we all use daily, from recognizing our speech to recommending what's in our news feed
- Today's systems still employ neural networks - but at a vastly different scale
- Recent systems for understanding and generating human language, such as OpenAI's GPT-3, were trained on supercomputer-scale resources
- These new systems require tens of millions of dollars in computation
14
122 reads
Pre-trained neural networks give smaller teams a leg up
- A neural network is first trained on a large general-purpose dataset using significant amounts of computational resources, and then fine-tuned for the task at hand using a much smaller amount of data and compute resources.
- The use of pre-trained networks has exploded in recent years as the industrialization of machine learning has taken over many fields and data has increased.
14
108 reads
The risks of foundation models: size, cost, and outsourced innovation
- One of the risks associated with foundation models is their ever-increasing scale.
- Pre-training on a large general-purpose dataset is no guarantee that the network will be able to perform a new task on proprietary data.
- Dataset alignment and recency of training data can matter immensely depending on the use case.
14
69 reads
Cloud APIs are easier, but outsourcing isn't free
- The cost of API calls, data storage, and cloud instances will scale along with usage.
- Many companies that have used cloud APIs for machine learning today may eventually attempt to transition to self-hosted or self-trained models to gain more control over their machine learning pipelines.
15
72 reads
Be strategic and keep an eye on the big AI labs
- Cloud APIs bring their own problems long-term. It's important to have a strategic exit plan so these APIs do not control your product destiny.
- Keep tabs on what is coming out of the big corporate AI labs.
15
91 reads
IDEAS CURATED BY
Aiden Mejia's ideas are part of this journey:
Learn more about startup with this collection
Understanding machine learning models
Improving data analysis and decision-making
How Google uses logic in machine learning
Related collections
Similar ideas
6 ideas
How to Use Massive AI Models (Like GPT-3) in Your Startup
future.a16z.com
4 ideas
What will applied AI look like in 2022?
venturebeat.com
1 idea
Hands-On Transfer Learning with Python
Dipanjan Sarkar
Read & Learn
20x Faster
without
deepstash
with
deepstash
with
deepstash
Personalized microlearning
—
100+ Learning Journeys
—
Access to 200,000+ ideas
—
Access to the mobile app
—
Unlimited idea saving
—
—
Unlimited history
—
—
Unlimited listening to ideas
—
—
Downloading & offline access
—
—
Supercharge your mind with one idea per day
Enter your email and spend 1 minute every day to learn something new.
I agree to receive email updates