Computational material design (CMD) aims to accelerate optimal design of complex material systems by integrating material science and design automation. For tractable CMD, it is required that (1) a feature space be identified to allow reconstruction of new designs, and (2) the reconstruction process be property-preserving. Existing solutions rely on the designer's understanding of specific material systems to identify geometric and statistical features, which could be insufficient for reconstructing physically meaningful microstructures of complex material systems. This paper develops a feature learning mechanism that automates a two-way conversion between microstructures and their lower-dimensional feature representations. The proposed model is applied to four material systems: Ti-6Al-4V alloy, Pb-Sn alloy, Fontainebleau sandstone, and spherical colloids, to produce random reconstructions that are visually similar to the samples. This capability is not achieved by existing synthesis methods relying on the Markovian assumption of material systems. For Ti-6Al-4V alloy, we also show that the reconstructions preserve the mean critical fracture force of the system for a fixed processing setting. Source code and datasets are available.