Validating new Automated Computer Vision workflows to traditional Automated Machine Learning

BSTRACT This paper presents some experiments to validate the design of an Automated Computer Vision (AutoCV) library for applications in scientific image understanding. AutoCV attempts to define a search space of algorithms used in common image analysis workflows and then uses a fitness function to automatically select individual algorithmic workflows for a given problem. The final goal is a semi-automated system that can assist researchers in finding specific computer vision algorithms that work for their specific research questions. As an example of this method the researchers have built the SEE-Insight tool which uses genetic algorithms to search for image analysis workflows. This tool has been used to implement an image segmentation workflow (SEE-Segment) and is being updated and modified to work with other image analysis workflows such as anchor point detection and counting. This work is motivated by analogous work being done in Automated Machine Learning (AutoML). As a way to validate the approach, this paper uses the SEE-Insight tool to recreate an AutoML solution (called SEE-Classify) and compares results to an existing AutoML solution (TPOT). As expected the existing AutoML tool worked better than the prototype SEE-Classify tool. However, the goal of this work was to learn from these well-established tools and possibly identify one of them that could be modified as a mature replacement for the core SEE-Insight search algorithm. Although this drop-in replacement was not found, reproducing the AutoML experiments in the SEE-Insight framework provided quite a few insights into best practices for moving forward with this research.