Automatic evaluation of end-to-end dialog systems with adequacy-fluency metrics

•Automatic metric for evaluating natural language generated sentences for dialog systems.•Integration of adequacy and fluency information to jointly evaluate semantic and syntactic information.•Higher correlation (up to 12.8%) with human evaluations than the best current objective metrics at a system and sentence level.