The images seen through the computer
Béatrice Joyeux-Prunel & Nicola Carboni
A digital image can be read and analyzed by a computer. Why not do it for millions?
A digital image is like a matrix of colored pixels, a grid filled with a multitude of colors.
Each pixel is characterized by its coordinates in the grid, as well as by its colors: a color is itself a mixture of a proportion R of red, a proportion G of green, a proportion B of blue. So a digital image can be translated into a series of numbers that describe the length and width of the image, the quantities of R, G, B for each pixel, etc...
All this can be analyzed with what is now called "computer vision".
The idea of using computer vision techniques in the human sciences is very recent. For our project, the simplest use of these techniques will be the most suitable. We do not need, for example, to recognize the content of the images we are studying. We simply ask the machine to tell us whether two images are similar or not. This is what allows us to work on the worldwide circulation of images in printed matter.
First step, a computer can automatically extract elements from a page in digital format.
In particular, the computer detects those quadrilaterals that distinguish an illustration on a printed page. This process, which is called "segmentation", makes it possible to automatically extract the images reproduced in thousands of pages of digitized printed matter (newspapers, magazines, illustrated magazines).
We are therefore able to build up a corpus of images published in the press at certain times.
Second step, the machine can, without much "effort", compare two images and determine that they are similar; or that they are the same.
--
A handful of illustrations from periodicals, gathered automatically by the machine. Algorithms used: ResNet18, HNSW, then IVPY.
So, determine in a larger batch of images, which images are the same or very similar.
How does the computer do this? It extracts from the pieces of the image their percentages of colors, the position of the lines, the proportion of white and black and many other formal elements that it quantifies... This extraction makes it possible to produce a vector, itself determined according to a given number of characters. The machine then projects all the vectors of all the images into a single space. It is then possible, algorithmically, to group all the points in this space according to groups of the closest points. By evaluating the mathematical proximity of the vectors that represent our images, computer vision helps us detect, from millions of images and without a human being having to "see" them individually, which images are the closest to each other. Copies, inspirations, images that look the same - they come to us in a matter of minutes. A chaos of images is suddenly ordered.
It would then be enough to know, for each image, when it was created, to order the images chronologically.
And to know, for each image, where it was created, to represent this chronology in space.