Blog

Of course! I began Newspaper Navigator in September 2019, while an Innovator in Residence at the Library of Congress. In partnership with LC Labs, the National Digital Newspaper Program, and IT Design & Development at the Library of Congress, as well as my Ph.D. advisor, Professor Daniel Weld, at the University of Washington, I began the project by extracting visual content across 16 million pages of digitized newspapers in the Chronicling America collection. In order to accomplish this, I leveraged annotations from the Beyond Words crowdsourcing initiative, which engaged the American public by asking volunteers to identify and classify visual content in Chronicling America. In particular, I finetuned an object detection model on these annotations in order to identify and categorize seven different classes of visual content on newspaper pages: photographs, illustrations, maps, comics, editorial cartoons, headlines, and advertisements. I then built an end-to-end pipeline to process all 100 terabytes of image and optical character recognition (OCR) data in Chronicling America using this finetuned model, yielding extracted visual content along with textual captions and other metadata. The Library of Congress and I released the resulting Newspaper Navigator dataset to the American public in May 2020, as the largest dataset of its kind ever produced. In pursuit of the Library’s mission of improving access, we placed the dataset and all code into the public domain for unrestricted re-use.

While caption-based keyword search for images provides much utility, the approach also has fundamental limitations: for example, how do people search for photographs with distinct visual motifs? In the second phase of Newspaper Navigator, I created and deployed a public search application for 1.5 million photographs in the dataset based on the real needs that historians and other users had articulated to us surrounding these limitations. Since launching with the Library of Congress in September 2020, the search application has reached tens of thousands of users and has been permanently adopted by the Library of Congress as part of their search infrastructure. In addition to providing keyword search functionality, the search application enables users to iteratively train machine learning algorithms in order to retrieve visually similar photos according to topics or concepts of interest, such as baseball players. From an exploratory search perspective, I call this search functionality open faceted search because it empowers users to create their own facets dynamically, facilitated by interactive machine learning algorithms that can train and predict over all 1.5 million photos in under a second. Unlike standard faceted search, open faceted search provides a path forward even when metadata is impoverished, making it extensible to a wide range of digitized collections. I presented open faceted search in a demo at UIST 2020.

To examine the ways in which a Chronicling America newspaper page is altered and decontextualized during its journey from a physical artifact to a series of probabilistic photographs in Newspaper Navigator, I wrote what I call a data archaeology. I released it with the Newspaper Navigator search application in order to provide scholars and the general public alike with a resource surrounding the ethical considerations and implications of this project (it is forthcoming in Digital Humanities Quarterly). In this data archaeology, I studied the digitization journeys of four different pages in Black newspapers in Chronicling America that reproduce the same photograph of W.E.B. Du Bois. In tracing the pages’ journeys, I unpacked how each step, from microfiliming to OCR to image embeddings, propagates bias, marginalization, and erasure via the machine learning algorithms employed.

Personally, I am very excited about the future of digital research with newspapers! The work being carried out by NewsEye, the National Digital Newspaper Program, and other initiatives across the world are re-imagining how we access digitized newspapers. As access to digitized newspapers continues to improve, I suspect that the already large communities surrounding them will only continue to grow. From a computational perspective, I am excited by the work being carried out to improve entity recognition, OCR correction, article segmentation, and beyond. From a humanistic perspective, I am looking forward to more text analysis across newspaper corpora. Lastly, given the ongoing digitization efforts across the world, I am excited to see scholars accessing new materials and doing scholarship across a diverse range of titles.