AI Total: Analyzing Security ML Models with Imperfect Data in Production

Development of new machine learning models is typically done on manually curated data sets, making them unsuitable for evaluating the models’ performance during operations, where the evaluation needs to be performed automatically on incoming streams of new data. Unfortunately, pure reliance on a fully automatic pipeline for monitoring model performance makes it difficult to understand if any obser...