A Step By Step Guide To AI Model Development

Ask the right questions.

• What results are you expecting from the process?
• What processes are being used ?
• What are the KPI's that can help track success?
• What resources are required?
• How do you break down the problem?

Based on your answers, you will fall under one of the categories:

1. Classification - helps categorize something into type-a/type-b/multi-type
2. Regression - helps predict a definite number of defined parameters. Eg: Covid-19 Projections, Holiday Seasons Sales Projections, etc.
3. Recommendation - analyze past data and identify patterns. Eg: video recommendations, target sales ads, etc.

ML Models are only as accurate as the data fed to them.

Important to identify the right set & format of data to ensure accuracy & relevance of the model.

Ask relevant questions:

• Data required to solve the problem. Eg. Customer data, Inventory
• Quantity of the data we have, and required
• Collection & Storage Mechanism
• Training Methods & Data Pipelines

Common formats of data are

1. Structured - fixed rows & columns. Eg. Inventory, sales
2. Unstructured - like variable texts, images, videos
3. Static - historical - sales & call history
4. Streaming - continuous, real time - web site traffic

It is critical, though time consuming to clean the available data and transforming it into required formats. 

Involves segmentation of  data sets into training, testing and validation.

• Process the available data
• Clean the data set for erroneous and Irrelevant data
• Transform the data into required format
• Enhance and augment the dataset if applicable

• Define the features of the model.
• Use the same features for both Training and Testing the model
• Collaborate closely with Subject Matter Experts 
• Be wary of "Curse of Dimensionality" - do not use unnecessary and irrelevant features that reduces the model accuracy
• Choose suitable Algorithms
• Consider Model interpretability - Predictions and Decisions should be clearly explainable

For Improving Results:

• Tune the hyperparameters or algorithm parameters like no. of trees of Random Forest or no. of layers in a Neural Network
• Use pre-trained models, reuse them to build new models
• Version each iteration

Primary objective of Model Testing is to improve results and minimize changes in model behavior post-deployment in real world.

Carry out multiple experiments using Training, Validation and Testing Datasets.

If model performs poorly on Training data improve the model by

• selecting a better algorithm,
• increasing quality of data, or
• feeding more data

If model performs poorly on Testing data - there might be overfitting, i.e. too closely fit with limited no. of data points. Then, add more data to the model.

After testing the model with different datasets, validate the model performance using the business parameters defined in step 1.

Analyze if KPI's and Business Objectives of the model are achieved. If not, consider changing the model, or improving the quality and quantity of the data.

After successful validation against all defined parameters, deploy the model onto planned infrastructure - cloud, edge, or on-premises environment.

Before deployment, consider the following

• plan to continuously measure and monitor the model performance
• define a baseline to measure future iterations of the model
• keep iterating the model to improve model performance with changing data

When a model is deployed in real-world, the data fed to it is dynamic.

There can also be changes in technology, business goals or drastic real world changes like the pandemic. 

It is crucial to analyze how these changes affect the model, so you can reiterate.

Consider monitoring the model for:

• deviations  from the pre-defined accuracy of the model
• irregular decisions or predictions
• drifts in the data affecting the model performance

Continuously keep iterating the model to keep up with the changes in data, technology and business.

AI models need time to be developed.

A smooth and successful model development involves a combined effort from data engineers, data scientists, ML engineers and DevOps engineers.

Proper resource allocation, project planning and management is crucial to meet the business goals and objectives.