Challenges
One of the major challenges faced by data scientists is to understand in what condition models perform poorly. Performance optimization becomes easier if underlying conditions can be presented with subsequent data and visual interpretation on top of that.
Solutions
Model performance evaluation and identification of issues where the model performs poorly. We generated analytics and insight that helped the AI technology company to retrain the model under certain conditions to increase their health score.
Outcome of our model observability were
- Health score
- Generating list of issues based on set threshold for certain rules like F1 score, precision, recall etc
- Confusion matrix
- Performance metrics and anomaly counts visualisation
Result
With our model observability reports, data scientists can clearly analyse the issues and conditions. Further, training data prepared for those conditions and models were retrained with a new set of data. This improved the model performance.