A Comparison of Generative Models for Sequence Design

Recent work has explored variants of the cross entropy method for black-box optimization of discrete structures such as DNA sequences. Using multiple rounds of calls to the black-box function, a generative sequence model is trained to place high probability on sequences that achieve high function values. These works employ sophisticated deep generative models, which, however, have high sample complexity and require extensive tuning. On the other hand, simple generative models, such as hidden Markov models, have achieved widespread success in computational biology applications. In response, we evaluate the performance of simple generative models when used within the cross entropy method. We find that simple generative models are competitive with more sophisticated models on two synthetic optimization tasks inspired by biological sequence design.