Modern Times 1936

Description

What is it that software sees, hears and perceives when technologies for pattern recognition are applied to media historical sources? All historical work requires interpretation, but what kind of algorithmic interpretations of modernity does software yield from historical archives? Modern Times 1936 is a research project funded by Sweden's second largest funding body (Riksbankens jubileumsfond) running between 2022 and 2025. The project involves four researchers, one developer and one master student (http://modernatider1936.se/en/). The project is empirically committed to everyday experiences and sets out to study how machines interpret symbols of modernity in media from the 1930s. By utilizing primarily photographic and audiovisual collections, the project seeks to analyze how modern Sweden was, while also exploring how computational methods can help us understand modernity in new ways.

We would like to propose a one-hour panel around some aspects of the research we have so far done within Modern Times 1936—with a focus on audiovisual media, foremost film and photography. Within our project we have collaborated with the heritage sector, assembling a dataset of some 80,000 photographs from the 1930s, scraped from the heritage portal DigitaltMuseum. We have also initiated a joint venture with the Nordic Museum in Stockholm. Regarding moving images we have used the media historical portal, filmarkivet.se—run by the National Library of Sweden and the Swedish Film Institute. We have used a number of speech-to-text-models on newsreels from the 1930s, but foremost upscaling algorithms for explorative film restoration. The latter work stems from previous film historical projects (Snickars 2015), as well as research done within, European History Reloaded: Curation and Appropriation of Digital Audiovisual Heritage, a EU-funded project that examined algorithmic ways of examining archival film reuse, and introducing a method for mapping video reuse with the help of AI and convolutional neural nets (Eriksson, Skotare & Snickars 2022). Hence, given the special theme of DHNB 2024 of collaboration with the heritage sector and involvement of ALM professionals alongside digital humanities research, we do think that a panel about Modern Times 1936 is a perfect match.

In our panel we will first make a short general presentation of Modern Times 1936—and yes, the Chaplin pun is intended. In short, our project explores how artificial intelligence and machine learning methods can foster new knowledge about the history of Swedish modernity—while at the same time critically scrutinizing algorithmic toolboxes for the study of the past. Within the panel we suggest to emphasize three particular strands that we have been working with: (1.) upscaling algorithms for film restoration, (2.) generative AI and pattern exploration within a major photographic dataset, and (3.) photographic super-resolution fantasies and the production of synthetic media.

(1.) Following a boom of user-friendly artificial intelligence tools in recent years, AI-enhanced (or manipulated) films have been framed as a serious threat to film archives. Film archivists are usually conservative; following their métier they are in the business of safeguarding film heritage. Today, however, the film archive—understood in a wide sense—is also elsewhere, most prominently online where each media asset becomes "at the instant of its release, an archive to be plundered, an original to be memorized, copied, and manipulated" (de Kosnik 2016). To explore these matters, trace and critically evaluate how algorithmic upscaling can modify older films, within our project we initiated a collaboration with Swedish AI artist ColorByCarl. He has been working with silent films from filmarkivet.se, and drawing on this collaboration we were able to study the procedures within the AI enhancement community, highlighting generative AI's potential to encourage reuse, remix and rediscovery of the filmic past.

(2.) Within our project we have been using pattern exploration within a major photographic dataset of some 80,000 photographs from the 1930s (taken from DigitaltMuseum). We have tagged each image with metadata categories using Doccano, and the general idea has been to train models to automatically find different types of images. Gender has been a case in point; two preliminary models can detect images with (or without) men and women with approximately 95 percent accuracy. We have also dealt with different kinds of object recognition models to track symbols of modernity such as factories, vehicles or cinemas. It has proven significantly more difficult, but we believe the work can be improved by using more older imagery as training data. The ambition of this work has been to develop models that can automatically annotate and sort larger visual heritage collections—and consequently we have initiated a collaboration with the Nordic Museum using their new collection 100,000 Bildminnen (image memories) as a case. On the one hand, we are developing models that can help the Nordic Museum to search and sort this collection in new ways, and on the other we are also interested in producing new images based on the same collection. Since Stable Diffusion is open source it can be trained on a specific dataset, such as 100,000 Bildminnen, and hence generate new historical photographs. Our aim is of course not to demonstrate that generative AI can picture the past in a better way. Rather, we believe that such a collaboration will open up new ways of understanding historical image collections.

(3.) Generative AI can indeed prove a useful tool to trace tropes and patterns in historical datasets (Offert & Bell 2020), and recent scholarship has also suggested that generative AI can offer new opportunities, not least in media history (Wilde 2023). Super-resolution technologies describe a set of computational methods for enhancing the resolution and/or sharpness of low-resolution digital visual content. Designed to fix what Hito Steyerl once referred to as poor images (Steyerl 2009), image upscaling is frequently promoted as a tool for improving visual content, as for example poorly digitized historical photographs. Yet what do super-resolution technologies actually do to visual and audiovisual content and make us see? Focusing on resolution technologies in real and imagined ways of enriching visual imagery, within our project we have explored how machine learning models are increasingly shaping ways of seeing, interpreting, and caring for historic photographs. Drawing from a series of experiments aimed at studying if/how super resolution technologies hallucinate and introduce new visual elements to historic photographs, we have explored how super resolution technologies unsettle boundaries between reality and fiction, hence both clarifying and occluding visions of the past.

Forthcoming publications:

Eriksson, Maria (2024) "On the Meaning of Scale in Image Upscaling", MontageAV (forthcoming)

Eriksson, Maria (2024), "Truthful Pixels: Synthetic Images the Measurement of Photorealism", Transbordeur (forthcoming)

Stjernholm, Emil & Snickars, Pelle (2024), "Upscaling Swedish Biograph", Journal of Scandinavian Cinema (forthcoming)

Period	2024 May 28
Event title	DHNB (Digital Humanities in the Nordic and Baltic Countries) 2024
Event type	Conference
Location	Reykjavic, IcelandShow on map