Define the business problem you are trying to solve.<ul><li>What results are you expecting from the process?</li><li>What processes are used to solve this problem?</li><li>Do you see AI improving the current process?</li><li>What are the key performance indicators (KPIs) that will help track progress?</li><li>What resources will be needed?</li><li>Consider how to break down the problem into iterative sprints.</li></ul> Once you have answers, then identify how you can solve the problem using AI.

Identify the business problem

This step is the most time-consuming, with ML engineers spending around 80% of the AI model development time in this stage. A significant amount of time is spent cleaning the data and transforming it into the required format. Things to consider include:<ul><li>Transforming the data into the required format.</li><li>Cleaning the data set of inaccurate and irrelevant data.</li><li>Enhance and augment the data set if the quality is low.</li></ul>

Preparing the data

Ask questions, such as.<ul><li>What data is needed to solve the business problem?</li><li>What quantity of data is required?</li><li>Do you have enough data to build a model?</li><li>Do you need more data to extend the existing data?</li><li>How is the data obtained, and where is it stored?</li><li>Can you use pre-trained data?</li></ul> Consider if your model will operate in real-time to determine if you need to create data pipelines to feed the model. Consider what form of data is required:<ul><li>Structured data in the form of rows and columns.</li><li>Unstructured data, such as images.</li><li>Static data, such as previous sales data.</li><li>Streaming data.</li></ul>

Identifying and collecting data

While the model is trained and tuned using the training and validation data set, the model will behave differently when used in the real world, which is fine. The main objective is to minimise the change in model behaviour when it is deployed. Three data sets are used when experiments are carried out: training, validation, and testing.<ul><li>If the model performs poorly on the training data, select a better algorithm, increase data quality, or feed more data into the model.</li><li>If the model does not perform well on testing data, the model may not extend the algorithm, and more data needs to be added.</li></ul>

Model testing

Analyse if the KPIs and the business objective of the model are achieved. If the parameters are not met, consider changing the model or improving the quality and quantity of the data. Before deployment:<ul><li>Ensure to measure and monitor the model performance continuously.</li><li>Define a baseline to measure future iterations of the model.</li><li>Keep iterating the model to improve model performance.</li></ul> When all the defined parameters are met, deploy the model into the intended infrastructure.

Model deployment

In 2019, near 87% of data science projects did not get into production. However, due to COVID -19, most companies have scaled up their AI adoption and increased their AI investment. In 2020, almost 50 % of enterprises employed an ML model. But to completely harness the power of AI, multiple models need to be created and deployed.

AI adoption

AI model development involves multiple stages that interconnect to each other. <ol><li>Identify the business problem. Instead of asking how to improve your artificial intelligence, ask how to improve your business.</li><li>Identify and collect data. Identifying the correct data is vital to ensure model accuracy and relevance.</li><li>Preparing the data.</li><li>Model building and training.</li><li>Model testing. The model is trained and tuned using the training and validation data sets.</li><li>Model deployment. Once the model is tested with different datasets, you will have to validate model performance using the parameters from Step 1.</li></ol>

The AI Model development lifecycle

At this step, all the requirements have been collected for the solution modelling to proceed. ML engineers will define the features of the model, taking the following into account:<ul><li>Use the same features for training and testing the model to avoid inaccurate results.</li><li>Consider working with Subject Matter Experts to direct you on what features would be necessary for the model.</li><li>Be wary of using multiple features that might be irrelevant to the model.</li></ul> Once the features are defined, choose the most suitable algorithm.

Model building and training

In 2019, Venturebeat reported that almost 87% of data science projects do not get into production. Redapt, an end-to-end technology solution provider, also re…

Model building and training

Similar ideas to Model building and training

Transfer learning and fine tuning