Multi-instrument Image Correlation for In Situ Planetary Science on Mars 2020

The Mars 2020 Perseverance rover is equipped with 23 cameras that aid in science operations and investigations [1]. The cameras on Perseverance image from the scale of tens of meters down to tens of microns, enabling detailed visual analysis of the martian surface across spatial scales. Some of these cameras are co-boresighted to spectrometers, allowing for the acquisition of spatially resolved compositional data. Integrating images from multiple different cameras is a challenge due to differences such as angle of acquisition, illumination, resolution, and pixel scale. However, combining these data enables a more comprehensive understanding of each target of interest by utilizing the complementary advantages of each imager or instrument. Here, we present two case studies that highlight the utility of computer vision to address science needs and improve science return of in situ analyses. In the first, we apply keypoint detection and matching to align images from two mapping spectrometers, thus allowing for complementary analysis with Raman spectroscopy and X-ray Fluorescence spectroscopy. We extend this case study by aligning images from two different cameras to yield a high-resolution blended composite that captures both texture and color information. In the second case study, we apply fast color transfer algorithms to adjust images taken by the same camera of many different rock targets, thus enabling a better comparison of textures and colors irrespective of lighting conditions during acquisition. We further apply this approach to match colors across images taken from different cameras to evaluate the similarity of the collected sample to the proxy abrasion patch on the parent rock.