FAME comes to an end! Initial results

4 Oct 2022

Adding content descriptions to more than 150,000 photos and videos in a management system is a huge challenge. With FAME, we’re therefore aiming to accelerate this process by using (semi-)automated facial recognition and simplifying how metadata is added, to make heritage collections more findable and searchable. The project is now coming to an end, so read on to learn about our initial results.

FAME in brief

With support from the Flemish Government, we’re testing innovative techniques for identifying people more easily on photos and videos. In collaboration with IDLab, we let an open source tool loose on the heritage collections of Kunstenpunt (Flanders Arts Institute) (link in Dutch), KOERS Museum of Cycle Racing, ADVN (Archive for National Movements) and the Flemish Parliament’s archive (link in Dutch). We focused on public figures, in particular: performing artists, professional cyclists, politicians and campaigners. This tool ensures not only that we know who is portrayed on which image, but also who is who on images with multiple people.

More about the project
Pictured: CVP premiers, photo by Michiel Hendryckx, CC BY-SA 3.0 via Wikimedia Commons.

Pictured: CVP premiers, photo by Michiel Hendryckx, CC BY-SA 3.0 via Wikimedia Commons.

What have we done so far?

In parallel with the FAME study days, last spring we started validating the faces automatically identified by the software. The four collection management organisations manually attached a validation label to each identification, to declare the recognition as accepted, rejected or unsuitable. The latter label refers to photos without people on them or with unclear images. We then also derived the number of unique individuals from the correctly recognised faces.

Results at a glance...

How many usable results did FAME deliver? The organisations accepted 78,440 of the 182,202 validation labels assigned in total. In other words, the facial recognition tool correctly identified more than 78,000 faces, and we can scale this number back to a remarkable 1,694 unique people!

One important nuance is that Kunstenpunt manually validated fewer results than the three other organisations. But we can extrapolate this if we accept a small margin of error. We therefore calculated this margin based on the validation labels that Kunstenpunt attached to the faces for which the software itself indicated a strong likeness with the reference set (a similarity score of at least 0.5). This showed that the tool reliability for recognising faces of performing artists was 93.6%. Applying this margin of error therefore gives us a maximum of 81,144 accepted recognised faces and 2,578 unique public figures.

Based on this percentage, we can therefore say that a maximum of 353 recognised faces and 65 unique people are incorrect from the Kunstenpunt collection. Finally, we can add to this the clusters of faces that the institutions compiled manually, accounting for another 2,003 faces of 96 unique individuals.

What do we get out of this?

The initial results look positive! We can therefore say that semi-automated facial recognition can have a positive impact on metadata creation, registration and enrichment. Including newly created metadata in the management systems will make it easier to search for and find content. There are also some other points for attention, e.g. inputting existing metadata, setting up good reference sets and (manual) validation all still take up a lot of time

What does the future hold?

We’re incorporating the conclusions from FAME in our search for an accessible and affordable tool that the majority of cultural heritage organisations can use on their own. ADVN and KOERS are also factoring the figures into their own photo digitisation project, Tegenlicht, on which they are collaborating with FOMU.

The insights are also important for the GIVE metadata project, where we are further exploring the potential of facial recognition and aiming to find a structural solution. The plan now is to set up a proof-of-concept for the large-scale application of facial recognition technology in the meemoo archive system.

Pictured: Christiane Goeminne, photo from Jelle Vermeersch, CC BY-SA 4.0 via Wikimedia Commons / KOERS.

Pictured: Christiane Goeminne, photo from Jelle Vermeersch, CC BY-SA 4.0 via Wikimedia Commons / KOERS.

This page is loading...