Researchers train AI to assign paintings based on detailed brushstroke analysis
Art historians could have a new tool to sort out attribution of disputed paintings using artificial intelligence (AI) thanks to research by an interdisciplinary team led by physicists from Case Western Reserve University in Cleveland, Ohio. The research, published in November in the review Heritage Sciences, shows how machine learning analysis of small sections of topographic scans of paintings, some as small as half a millimeter, was able to assign works to the right artist with up to 96% accuracy. The technology could potentially also help identify artists responsible for different areas of a painting made by multiple artists or produced by an artist’s studio, and help distinguish genuine works from counterfeits.
The project differs from others who have sought to harness AI to address attribution and authenticity issues in most of the previous ones. to research in this area has been based on machine analysis of high resolution images of paintings, and not on the painted surfaces of the canvases themselves.
“The idea was that the analysis of the brushstroke would create a fingerprint,” says Kenneth Singer, professor of physics at Case Western Reserve who led the research. “We found that even at the level of the bristles of the brush, the attribution sorting was quite successful. Frankly, we don’t really understand this, it’s a bit mind-boggling when you think about it, how the paint that comes off with just one hair is indicative of what we call the artist’s unintentional style.
The project focused on analyzing sets of paintings made especially by students at the Cleveland Institute of Arts, who were commissioned to paint copies of a photograph of a water lily. The research involved training convolutional neural networks (CNNs) with three-dimensional analyzes of the surfaces of paintings made with a profilometer. By dividing the canvases into tiny squares for analysis, CNNs identified the “unintentional style” or “fingerprint” of each artist. The AI was then able to correctly assign other paintings by matching the artists’ unintended styles in the textures of the brushstrokes.
The team behind the research are now looking for further tests of their AI’s abilities. He collaborated with the conservation firm Factum Arte to analyze a topographic analysis of the El Greco site. Portrait of Juan Pardo de Tavera (1609), which was badly damaged during the Spanish Civil War and extensively restored.
“This is a painting for which we have an answer, because we have photos of the destroyed painting and the current painting, so we are able to make a map of the areas that have been preserved, and [the AI] was able to identify these areas, ”says Singers. “But there was another part of the painting that he identified as preserved that wasn’t obvious, so we’re going to have a paint restorer in Spain to look at the painting to see what happens.”
Now the research team is turning its attention to paintings produced by several artists trying to reproduce the style of a painter in their studio or workshop. To discern between the hand of a Renaissance master, that of his star pupil and those of his lesser-known assistants has long been the subject of heated debate among art historians and scholars of the Old Masters, often with large sums of money at stake during the works. go to auction. The researchers hope to develop “unbiased and quantitative methods to provide insight into the contested attributions of studio paintings,” they write. To that end, they again work with artists from the Cleveland Institute of Art to create all-new paintings in a studio process, with multiple artists working on each canvas in a unified style.
In addition to student painters and members of the physics and biology departments at Case Western Reserve, research collaborators included the university’s chair of art history, Elizabeth Bolman, and contributions from chief curator from the Cleveland Museum of Art, Per Knutås. The company was a true marriage of art and science.
“The project was born from an idea of one of my students, who at the time had just started dating an art history student,” says Singer. “They went to an art and science conference and came up with the idea of using this profilometer that we have in one of our labs to do surface topography. I agreed to do it and then after a while all my students got involved and the collaboration developed. By the way, these two students are married now.
The next application of AI might be to test it on media that have less surface texture than paintings, Singer says, such as watercolors or drawings. “It seems to be more difficult,” he says, “but what I learned in this project is that I shouldn’t be as skeptical as usual, because this artificial intelligence is surprisingly good. “