These days I lead product development teams in several key areas of the Databricks business, including exploratory data analysis, machine learning, and data ingestion. Today, Databricks partners with thousands of organizations to help them take advantage of large scale data processing and machine learning.
I dropped out of a PhD in 2013 to co-found Databricks. At Berkeley, I was one of the founding committers of Apache Spark and published several papers in the area of distributed systems, with a focus on resource management. From 2015-2017 I led the entire engineering team @Databricks, growing from 35 to 80 engineers across low level systems, networking, machine learning, and data visualization, and application engineering.