Capturing aesthetic complexity in art (and beyond) using compression ensembles

Author:

• Andres Karjus - Tallinn University, Tallinn, Estonia

Abstract

The quantification of visual aesthetics, including artistic expression, has a long history (cf. Birkhoff 1933; Rigau et al. 2007; Lee et al. 2020). Previous research drawing on information theory has shown that visual complexity can be estimated with some accuracy using compression algorithms such as GIF or JPEG, but results based on human complexity ratings diverge as to which single algorithm or approach is optimal (Forsythe et al. 2011; Palumbo et al. 2014). I will discuss our recent preprint which builds on these ideas (Karjus et al 2022, Compression ensembles quantify aesthetic complexity and the evolution of visual art). We propose a novel method — compression ensembles — which consists of a large number of compression lengths not only of the image itself but also its various transformations, embedded in a latent vector space.
The proposed approach allows for capturing the "algorithmic fingerprints" of artworks and by extension, artists, to compare artworks simultaneously in multiple dimensions of complexity, including hue, contrast, structure, composition, and more generally, capture polymorphic family resemblance. We show that our method is cognitively plausible by demonstrating strong correlations with visual complexity norms from a range of languages and cultures, but also that it performs reasonably well on downstream tasks like artist and style identification. We then apply this approach to tens of thousands of paintings spanning half a millennium, probing established narratives in the historiography of Western art. Finally, I will discuss ongoing work on extensions of the general methodology.

Andres Karjus

I am a research fellow at the CUDAN Cultural Data Analytics lab at Tallinn University, working on language and culture dynamics, using large corpora, machine learning, and human experiments. Recently I have been also working on, through various collaborations, on art history and creative industries and advising collaborative projects between involving academic and media industry partners.
I defended my PhD in 2020, in linguistics at the Centre for Language Evolution of University of Edinburgh, on lexical dynamics and communicative need in language. I also occasionally teach stats and data visualization for the humanities and social sciences (see more here). The photo on the right naturally slightly outdated, as is customary on academic home pages.