The first research focus is set on computer vision’s potential contribution to the history of art and architecture. As the photographic collection of the BHMPI is currently being digitized a a fast rate, and a massive number of art-related research data is being pooled by several partnering institutions (see 3.3), new methodologies will be developed to read, order, and represent art historical visual material. Supervised and semi-supervised machine learning will be possible thanks to the high-quality metadata of existing visual records, automatically predicting metadata for the new unlabeled images – whilst unsupervised methods will also be useful to organize and search the image-datasets through elements not present in the metadata (style, iconography, gesture). These will combine a number of existing techniques, applied to different material (face recognition, style recognition, gesture recognition, sketch-based image retrieval) with new problems in computer vision (architectural pattern recognition, artistic attribute detection). Further annotations will be collected during users’ and scholars’ employment of these prototype systems, improving the quality of both the underlying data and – through online machine learning – future models: such a mechanism necessitates the rapid technology transfer of subsequent computer vision prototypes to the BHMPI’s online photo archive interface. As well as defining new problems in computer vision, the annotations created in such a process can be offered to the computer vision and pattern recognition community, providing new machine learning databases and baselines for relevant computer science research communities, such as ECCV-VISART and ACM Expressive.
Examples for possible research topics include:
- face recognition and social network analysis of people’s representation in photographs or other visual media;
- automatic cataloging and ordering of photographic collections based on individual hand-written metadata as to reconstruct the history of collections;
- developing machine learning for visual similarity as to test theories and study practices of visual comparison in scholarly publications;
- automatic movement categorization and correlation for performing art, video art and cinema as to extract and structure information from moving images, enabling new types of comparative analysis;
- embedding of textual, audio and visual information in digital representations, and the study of cross-correlations between textual and visual content;
- automatic semantic classification and /annotation of 2D and 3D elements using machine learning and textual resources;
- spatialization of textual and visual information within 2D and 3D representations;
- digital reconstructions of light, color, acoustics and other sensory phenomena, and computational analysis of these reconstructions
- critique of digital approaches for the classification and operationalization of visual information.