基本信息
views: 1198
Career Trajectory
Bio
I have invented much of the current revolution in large language models. Some of my inventions include:
Transformer (2017)
(personally designed the multi-head attention, the residual architecture, and coded up the first better-than-SOTA working implementation)
Sparsely-gated Mixture of Experts (2016)
Mesh-Tensorflow (2018) - first practical system for training giant Transformers on supercomputers.
T5 (2019)
Major contributor to Google’s LaMDA dialog system, a project led by Daniel De Freitas, my now co-founder at Character AI.
Transformer (2017)
(personally designed the multi-head attention, the residual architecture, and coded up the first better-than-SOTA working implementation)
Sparsely-gated Mixture of Experts (2016)
Mesh-Tensorflow (2018) - first practical system for training giant Transformers on supercomputers.
T5 (2019)
Major contributor to Google’s LaMDA dialog system, a project led by Daniel De Freitas, my now co-founder at Character AI.
Research Interests
Papers共 87 篇Author StatisticsCo-AuthorSimilar Experts
By YearBy Citation主题筛选期刊级别筛选合作者筛选合作机构筛选
时间
引用量
主题
期刊级别
合作者
合作机构
JOURNAL OF MACHINE LEARNING RESEARCH (2023): 240:1-240:113
arXiv (Cornell University) (2022)
JOURNAL OF MACHINE LEARNING RESEARCH (2022): 120:1-120:39
CoRR (2022)
Annual Conference on Neural Information Processing Systems (2022): 6010-6022
CoRRpp.1044-1044, (2022)
semanticscholar(2021)
Cited0Views0Bibtex
0
0
Load More
Author Statistics
Co-Author
Co-Institution
D-Core
- 合作者
- 学生
- 导师
Data Disclaimer
The page data are from open Internet sources, cooperative publishers and automatic analysis results through AI technology. We do not make any commitments and guarantees for the validity, accuracy, correctness, reliability, completeness and timeliness of the page data. If you have any questions, please contact us by email: report@aminer.cn