Step 7 - Monitor Model

Step Seven- Monitor Model. In this section will provide overview of his step seven monitor model of our data science methodology. Step seven monitor model will be covering what resources and guardrails should be in place for continuous monitoring and learning of your AAII system for production use. How do you monitor your model for incorporating user feedback and incorporating fee learning. We will start with the key resources we will be needing for model management and model monitoring.

First major user of an area model End user or expert. End user and experts are business professionals who interact with the system to enter use case data and use it for making decisions. Experts and managers are individuals who review and approve these decisions.

Second major user is Data Scientist. Data scientist analyze past extraction results and user feedback to look for patterns and improvement areas. They may utilize a variety of statistical techniques to organize and learn from past data to make specific recommendation to the librarian. Librarian is a key resource for model management. Model librarians collect, collate and organize data models training samples and test results. They may utilize a combination of human inputs and technical processes to organize the model in the library.

Step Seven- Monitor Model overview. We now cover couple of this of using experimental design for continuous improvement. What is the overall process and who is engaged in managing the continuous improvement process. As we talk, there are four million users associated with the process.

Data scientist and user experts and model librarian. The data scientist trains the model and prepares it for deployment and publishing. And user uses the model and provide feedback. Expert analysis end user feedback to enhance test data and to retrain the model to improve the overall model performance. Librarian manages multiple models and tracks the progress of challengers to decide which model to use in a production setting. Hope this process gives you a good understanding of how the continuous improvement can be established as a process and executed regularly covering a large number of models.

Step Seven- Monitor Model. Here are the key task, define model users, their roles and responsibility. Monitor model performance for continuous feedback and improvement. Collect user feedback, implement continuous improvement and manage model lifecycle. There are key deliverables from the process, feedback reports, continuous improvement dashboard and wanted improvement results.