Using online games to capture, generate, and understand natural language

While the game industry has excelled at simulating combat, dynamically generating social interaction and natural language dialogue has proven intractable. The gaming landscape today, with millions of people playing games together online, provides opportunities to radically rethink our approach to developing conversational, socially intelligent characters. This talk will present a data-driven, human-machine collaborative approach to automating characters using data recorded from online human-human interactions, including a crowd-sourced annotation framework, and a new real-time planning system driven by thousands of annotated human gameplay traces. The approach will be demonstrated with examples from three novel games: The Restaurant Game has recorded over 16,000 people playing as customers and waitress, Improviso is currently recording players on the set of a low-budget science fiction movie, and Mars Escape has recorded hundreds of online human-robot interactions for eventual transfer to a real robot.