Patentability: Improving Acceptance Prediction of US Patent Applications using Ensemble Modeling

Patents are an essential part of modern innovation, allowing for certification and protection when an idea is useful and new. However, what determines a successful patent application in the United States has not been thoroughly explored on a mass-scale by machine learning. Suzgun et al. have presented the task of binary acceptance prediction, where the model predicts whether applications were accepted or rejected. In our paper, we re-establish these results on baseline models, and then improve on them with original ensemble models to determine an application’s utility and novelty. Namely, by first ensembling models that look at different parts of a patent (Abstract, Claims) and also ensembling different types of models (Naive Bayes, DistilBERT), we can better model an application’s utility and achieve 62.65% accuracy on acceptance prediction. Additionally, we confirm patterns from Suzgun et al. that Transformer models are not able to significantly outperform Naive Bayes in the prediction task, and discuss hypotheses why. Finally, we provide a visual understanding of how our models focus on various parts of text in its prediction using integrated-gradient analysis. With our improvements to the acceptance prediction task and deepened understanding of a model’s focusing, we grow the information on determining what makes a patent application successful, which is useful especially for patent office efficiency and for groups that are underrepresented in the patent domain.