When a newsworthy event occurs, media articles that report on the event can vary widely-a concept known as coverage diversity. To help investigate coverage diversity in event reporting, we de-velop a visual analytics system called News Kaleidoscope. News Kaleidoscope combines several backend language processing techniques with a coordinated visualization interface. Notably, News Kaleidoscope is tailored for visualization non-experts, and adopts an analytic workflow based around subselection analysis, whereby second-level features of articles are extracted to provide a more detailed and nuanced analysis of coverage diversity. To robustly evaluate News Kaleidoscope, we conduct a trio of user studies. (1) A study with news experts assesses the insights promoted for our targeted journalism-savvy users. (2) A follow-up study with news novices assesses the overall system and the specific insights pro-moted for journalism-agnostic users. (3) Based on identified system limitations in these two studies, we refine News Kaleidoscope's design and conduct a third study to validate these improvements. Results indicate that, for both news novice and experts, News Kalei-doscope supports an effective, task-driven workflow for analyzing the diversity of news coverage about events, though journalism expertise has a significant influence on the user's insights and take-aways. Our insights developing and evaluating News Kaleidoscope can aid future tools that combine visualization with natural language processing to analyze coverage diversity in news event reporting.