A stochastic approximation approach to fixed instance selection.

•We introduce a new model-agnostic fixed instance selection methodology is based on simultaneous perturbation stochastic approximation.•We present extensive computational experiments across 43 diverse datasets and 4 different classifiers.•Our methodology provides a statistically significant improvement over random sampling in over 90% of tests at a 5% level of significance.•Our methodology outperforms Fast Condensed Nearest Neighbours when selecting the same number of instances on average.