Visualizing planetary spectroscopy through immersive on-site rendering

Remote sensing is currently the primary method of obtaining knowledge about the composition and physical properties of the surface of other planets. In a commonly used technique, visible and near-infrared (VNIR) spectrometers onboard orbiting satellites capture reflectance data at different wavelengths, which in turn gives insight about the minerals present and the overall composition of the terrain. In select locations on Mars, rovers have also conducted up close in-situ investigation of the same terrains examined by orbiters, allowing direct comparisons at different spatial scales. In this work, we build Planetary Visor, a virtual reality tool to visualize orbital and ground data around NASA's Mars Science Laboratory Curiosity rover's ongoing traverse in Gale Crater. We have built a 3D terrain along Curiosity's traverse using rover images, and within it we visualize satellite data as polyhedrons, superimposed on that terrain. This system provides perspectives of VNIR spectroscopic data from a satellite aligned with ground images from the rover, allowing the user to explore both the physical aspects of the terrain and their relation to the mineral composition. The result is a system that provides seamless rendering of datasets at vastly different scales. We conduct a user study with subject matter experts to evaluate the success and potential of our tool. The results indicate that Visor assists with geometric understanding of spectral data, improved geological context, a better sense of scale while navigating terrain, and new insights into spectral data. The result is not only an immersive environment in a scientifically interesting area on Mars, but a robust tool for analysis and visualization of data that can yield improved scientific discovery. This technology is relevant to the ongoing operations of the Curiosity rover and will directly be able to represent the data collected in the upcoming Mars 2020 Perseverance rover mission.