Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference

On-device machine learning (ML) moves computation from the cloud to personaldevices, protecting user privacy and enabling intelligent user experiences.However, fitting models on devices with limited resources presents a majortechnical challenge: practitioners need to optimize models and balance hardwaremetrics such as model size, latency, and power. To help practitioners createefficient ML models, we designed and developed Talaria: a model visualizationand optimization system. Talaria enables practitioners to compile models tohardware, interactively visualize model statistics, and simulate optimizationsto test the impact on inference metrics. Since its internal deployment twoyears ago, we have evaluated Talaria using three methodologies: (1) a loganalysis highlighting its growth of 800+ practitioners submitting 3,600+models; (2) a usability survey with 26 users assessing the utility of 20Talaria features; and (3) a qualitative interview with the 7 most active usersabout their experience using Talaria.