A Step By Step Guide To AI Model Development - Deepstash
A Step By Step Guide To AI Model Development

A Step By Step Guide To AI Model Development

Curated from: datasciencecentral.com

Ideas, facts & insights covering these topics:

11 ideas

·

1.55K reads

15

Explore the World's Best Ideas

Join today and uncover 100+ curated journeys from 50+ topics. Unlock access to our mobile app with extensive features.

A Guide to Artificial Intelligence Model Development

A Guide to Artificial Intelligence Model Development

Machine Learning (ML) is an integral part of Data Science (DS), that helps computers learn from data. 

This process of learning from data through machine learning techniques contributes to Artificial Intelligence (AI).

Here is a quick overview and simplified steps for developing AI Models.

38

338 reads

Steps Involved in AI Model Development

Steps Involved in AI Model Development

  1. Identification of Business Case
  2. Collection of Data
  3. Preparation of Data
  4. Building the Model
  5. Testing the Model
  6. Validate the Model
  7. Deploy the Model
  8. Govern the Model

39

335 reads

1. Identification of Business  Case

1. Identification of Business Case

Ask the right questions.

  • What results are you expecting from the process?
  • What processes are being used ?
  • What are the KPI's that can help track success?
  • What resources are required?
  • How do you break down the problem?

Based on your answers, you will fall under one of the categories:

  1. Classification - helps categorize something into type-a/type-b/multi-type
  2. Regression - helps predict a definite number of defined parameters. Eg: Covid-19 Projections, Holiday Seasons Sales Projections, etc.
  3. Recommendation - analyze past data and identify patterns. Eg: video recommendations, target sales ads, etc.

37

163 reads

2. Collection of Data

2. Collection of Data

ML Models are only as accurate as the data fed to them.

Important to identify the right set & format of data to ensure accuracy & relevance of the model.

Ask relevant questions:

  • Data required to solve the problem. Eg. Customer data, Inventory
  • Quantity of the data we have, and required
  • Collection & Storage Mechanism
  • Training Methods & Data Pipelines

Common formats of data are

  1. Structured - fixed rows & columns. Eg. Inventory, sales
  2. Unstructured - like variable texts, images, videos
  3. Static - historical - sales & call history
  4. Streaming - continuous, real time - web site traffic

37

115 reads

3. Preparation of Data

3. Preparation of Data

It is critical, though time consuming to clean the available data and transforming it into required formats. 

Involves segmentation of  data sets into training, testing and validation.

  • Process the available data
  • Clean the data set for erroneous and Irrelevant data
  • Transform the data into required format
  • Enhance and augment the dataset if applicable

37

113 reads

4. Building the Model

4. Building the Model

  • Define the features of the model.
  • Use the same features for both Training and Testing the model
  • Collaborate closely with Subject Matter Experts 
  • Be wary of "Curse of Dimensionality" - do not use unnecessary and irrelevant features that reduces the model accuracy
  • Choose suitable Algorithms
  • Consider Model interpretability - Predictions and Decisions should be clearly explainable

For Improving Results:

  • Tune the hyperparameters or algorithm parameters like no. of trees of Random Forest or no. of layers in a Neural Network
  • Use pre-trained models, reuse them to build new models
  • Version each iteration

37

93 reads

5. Testing the Model

5. Testing the Model

Primary objective of Model Testing is to improve results and minimize changes in model behavior post-deployment in real world.

Carry out multiple experiments using Training, Validation and Testing Datasets.

If model performs poorly on Training data improve the model by

  • selecting a better algorithm,
  • increasing quality of data, or
  • feeding more data

If model performs poorly on Testing data - there might be overfitting, i.e. too closely fit with limited no. of data points. Then, add more data to the model.

38

71 reads

6. Validate the Model

6. Validate the Model

After testing the model with different datasets, validate the model performance using the business parameters defined in step 1.

Analyze if KPI's and Business Objectives of the model are achieved. If not, consider changing the model, or improving the quality and quantity of the data.

37

80 reads

7. Deploy the Model

7. Deploy the Model

After successful validation against all defined parameters, deploy the model onto planned infrastructure - cloud, edge, or on-premises environment.

Before deployment, consider the following

  • plan to continuously measure and monitor the model performance
  • define a baseline to measure future iterations of the model
  • keep iterating the model to improve model performance with changing data

37

79 reads

8. Govern the Model

8. Govern the Model

When a model is deployed in real-world, the data fed to it is dynamic.

There can also be changes in technology, business goals or drastic real world changes like the pandemic. 

It is crucial to analyze how these changes affect the model, so you can reiterate.

Consider monitoring the model for:

  • deviations  from the pre-defined accuracy of the model
  • irregular decisions or predictions
  • drifts in the data affecting the model performance

Continuously keep iterating the model to keep up with the changes in data, technology and business.

37

73 reads

Further Considerations

Further Considerations

AI models need time to be developed.

A smooth and successful model development involves a combined effort from data engineers, data scientists, ML engineers and DevOps engineers.

Proper resource allocation, project planning and management is crucial to meet the business goals and objectives.

37

99 reads

IDEAS CURATED BY

madspeak

Lifelong Learner. Audible Fan. Deep Generalist.

CURATOR'S NOTE

The terminologies might change, the technologies might evolve, but this is the future. And that Future is already here!

Madhu Kalale's ideas are part of this journey:

The Podcasting Ecosystem

Learn more about technologyandthefuture with this collection

The importance of networking in podcasting

How to grow your podcast audience

How to monetize your podcast

Related collections

Read & Learn

20x Faster

without
deepstash

with
deepstash

with

deepstash

Personalized microlearning

100+ Learning Journeys

Access to 200,000+ ideas

Access to the mobile app

Unlimited idea saving

Unlimited history

Unlimited listening to ideas

Downloading & offline access

Supercharge your mind with one idea per day

Enter your email and spend 1 minute every day to learn something new.

Email

I agree to receive email updates