Computational and Historical Analysis of Hands and Gestures in Early Modern Art - Valentine Bernasconi - Ph.D. Fellow

I am currently a PhD student in the field of Digital Visual Studies at the University of Zurich (UZH). Prior to that, I graduated from the Ecole Polytechnique Fédérale de Lausanne (EPFL) with a M.Eng. in Digital Humanities. I also have a previous M.Sc. in Multimedia Design and 3D Technologies from Brunel University London and a B.Sc. in Computer Science from the University of Fribourg, where I received the JAACS award for best Bachelor thesis. The latter was part of an interdisciplinary collaboration with dancers from the University of the Art of Biel (HKB) and engineers from the Haute Ecole d’Ingénierie et d’Architecture of Fribourg (HEIA-FR).

My main research interest lies in between art and technology. Thanks to a minor in art history at the University of Fribourg, I was able to develop a critical approach to artistic content alongside an education in computer science. This interdisciplinary knowledge allowed me to work on a collaborative project called Nautilus. I there investigated the rise of digital media in performing arts from an art history point of view and contributed to the project with an interactive visualization. I further explored the world of augmented reality and 3D experiences with a master thesis called Interactive live 3D visualization.  Finally, I had the opportunity to delve into the world of design at the EPFL+ECAL Lab with a master thesis Valorization of visual heritage through A.I. algorithm trained on curated content. The curated content, the poster collection from the Museum für Gestaltung Zürich, allowed me to approach the field from both an art history and artistic perspective and to explore a diversity of computer vision tools. The research lead to the creation of a machine learning model to reproduce the creative thinking of the designer.

The core of my current research is at the intersection of computer vision and art history and is supervised by Prof. Dr. Tristan Weddigen and Dr. Leonardo Impett. I there wish to explore recurrent patterns in pictorial art and their evolution in time and place. This approach relies on the context of visual and cultural influence of artworks and intend to produce new exploring tools to the field of art history.

Project Summary

Reproductions, such as replicas of patterns, copies or similar representations of an iconography, have an important place in art history. European artists and their productions, since the early modern times, have been greatly influenced by growing artistic exchanges and the development of reproduction techniques. Furthermore, recent works have shown the possibility to create new browsing tools for art databases with precisely patterns search.
The present work, based on the assumption that we live in a context of visual influences, aims to explore in more details recurrent patterns in pictorial art, with a focus on hands and gestures. What are the most recurrent gestures and how do they participate in the narrative system of a painting? Who are the source painters of popular gestures and how to they evolve over time? What are their influences?
Placed at the intersection of two disciplines, art history and computer science, the project first aims to gain a better understanding of the evolution of hands and gestures in pictorial art and the way they take part in the narrative system and visual aesthetic of an artwork. A second phase consists of the elaboration of a tool to perform body recognition on a set of paintings and drawings from the digital collection of the Bibliotheca Hertziana. Because machine learning models already exist for objects detection on real images, the main challenge is to tailor them to the specific features of paintings with the help of data augmentation techniques. Further unsupervised clustering should then be performed on the bodies and hands collected in order to outline possible repetitive patterns and perform further analyses based on the available metadata from the collection. The project also does not exclude to enrich the metadata based on discoveries made with these gestures recognition. Finally, the project aims to provide a deeper exploration and analysis of a specific gesture or gestures engaged with a specific iconography.