How to Use Massive AI Models (Like GPT-3) in Your Startup - Deepstash

Bite-sized knowledge

to upgrade

your career

Ideas from books, articles & podcasts.

created 6 ideas

A16Z

How to Use Massive AI Models (Like GPT-3) in Your Startup

How to Use Massive AI Models (Like GPT-3) in Your Startup

future.a16z.com

STASHED IN:

365 reads

Expensive to train?

Some neural networks are very expensive to train. This led to the popularization of an approach known as pre-training, whereby a neural network is first trained on a large general-purpose dataset using significant amounts of computational resources, and then fine-tuned ...

The use of pre-trained networks allows a startup, for example, to build a product with much less data and compute resources than would otherwise be needed if starting from scratch. This approach is also becoming popular in academia, where researchers can quickly fine-tune a pre-trained network fo...

Foundational models

Pre-training has continued to evolve with the emergence of foundation models such as BERT, GPT, DALL-EC...

One of the risks associated with foundation models is their ever-increasing scale. Neural networks such as Google’s T5-11b (open sourced in 2019) already require a cluster of expensive GPUs simply ...

Dataset alignment can also be a challenge for those using foundation models. Pre-training on a large general-purpose dataset is no guarantee that the network will be able to perform a new task on proprietary data. The network may be so lacking in context or biased based on its pre-training, that ...

The opportunities and risk around using hosted and pre-trained models has led many companies to leverage cloud APIs in the “experimentation phase” to kickstart product development.

Once a company has determined it has a product-market fit, it...

7 Reactions

Comment

It's time to

READ

LIKE

A PRO!

Jump-start your

reading habits

, gather your

knowledge

,

remember what you read

and stay ahead of the crowd!

Takes just 5 minutes a day.


TRY THE DEEPSTASH APP

+2M Installs

4.7 App Score