Mars Science Laboratory R13 Mobility Regression Testing

Now in its 10th year of Operation on Mars, NASA's Curiosity Rover is preparing to undergo a software update, one that has been six years in the making. In order to qualify this update for use on Mars, the flight software first needs to be run through a regression test campaign here on Earth. This paper details the regression testing efforts of the Mobility team as they evaluated the performance of the rover's driving capabilities under this new Flight Software Upgrade - known as R13. This test campaign was conducted to ensure that all of the rover's mobility capabilities behave as expected, especially to check the parts that are "untouched" by R13 have not regressed in performance, hence the term regression testing. Of 75 mobility commands, Curiosity has 18 commands that initiate motion and each has a number of arguments and options that can completely change the rover's traverse behavior. This leads to a vast and complex problem space that is impossible to exhaustively explore, necessitating a regression test campaign that targets key subsets of commands and arguments in order to create the confidence needed to approve R13 for flight. To conduct this testing, the Mobility Team spent 10 days in the Jet Propulsion Laboratory's (JPL) Mars Yard commanding the Vehicle System Test Bed (VSTB) - Curiosity's Earthbound twin. The nominal performance for 11 of Curiosity's motion-inducing commands was evaluated in addition to the Fault Protection behavior for 29 off-nominal cases that could occur in relation to mobility activities. The off-nominal testing required commanding the rover in a way that the Operations Team never intentionally would on Mars, purposefully stalling motors, shutting down the rover mid-drive, and exceeding limits that are hopefully never reached in flight. The Mobility team's 10 shifts, the most of any subsystem, created two significant challenges consisting of staffing and data management. Unlike most other spacecraft testbeds that spend their days in clean rooms, Curiosity's drives around the Mars Yard and requires at least two testers to operate. One of which commands the rover, and the other who moves the umbilical cable that provides power and communication between the command station and vehicle a physically demanding task in the Southern California summer heat - leading to the often addition of a third tester to lighten the load. After testing concluded, data from the tests was analyzed to confirm that the rover behaved as expected, and if not, what further testing or documentation would be required to proceed with approval for R13. Finally, the Mobility Team presented their test results to the regression review board and the test campaign was approved, bringing R13 one step closer to running on Mars.