As more humans settle in dense urban areas, the effect of natural or anthropogenically induced shocks at these locations has an increased potential to impact larger numbers of individuals. In particular, a disruption to the delivery of goods and services can leave large portions of the population in a vulnerable state. Research suggests that resilience to shocks is a function of physical fortifications and social processes, such as levees and critical infrastructure, the strength of social networks, or community efficacy, and trust. While physical fortifications are relatively easy to identify and catalog, the measurement of social processes is more difficult due to data limitations and geographic constraints. Recent work has shown that certain types of infrastructure may correlate with social processes that enhance community resilience; however, the ability to assess where and to what extent that infrastructure exists depends on a complete representation of the built environment. OpenStreetMap (OSM) and Google Places are two sources of data commonly used to identify the location and type of infrastructure but can display varying degrees of completeness depending on geographic location. We address this limitation by applying a Convolution Neural Network (CNN) to remotely sensed data from Sentinel-2 to estimate the density and type of infrastructure. We compare the classification results to known infrastructure locations from OSM data. Our results show that the CNN classifier performs well and may be used to augment incomplete data sets for a deeper understanding of the prevalence of infrastructure associated with social processes that enhance community resilience.