His research interests lie in the area of artificial intelligence, including deep learning, speech, natural language, computer vision, information retrieval, and knowledge representation & management. He has authored/co-authored more than 100 papers and one book, and has published in ACL, CVPR, SIGIR, WWW, CIKM, NIPS, ICLR, IEEE TASLP, Proc. IEEE, IEEE SPM and other venues. He and colleagues developed the MSR-NRC-SRI entry and the MSR entry that won No. 1 in the 2008 NIST Machine Translation Evaluation and the 2011 IWSLT Evaluation (Chinese-to-English), respectively, and the MSR image captioning system that won the 1st Prize in the MS COCO Captioning Challenge 2015.