Project: FIN-CLARIAH
Grant agreement: Research Council of Finland no. 358720
Start date: 01-01-2024
Duration: 24 months
WP 3.3: Report on Reliable Enrichment of Visual Data
Date of reporting: 29-09-2025
Report authors: Matti Nelimarkka (University of Helsinki)
Contributors: Anton Berg (University of Helsinki), Leonardo Negri (University of Helsinki)
Deliverable location: https://github.com/uh-soco/coslab-core and https://github.com/uh-dcm/coslab-gui
Image recognition services, such as Amazon Rekognition, Google Vision and Azure AI Vision, allow anyone to label image content, however their outputs vary per service (ref to image as data book). Cross-service label agreement score (COSLAB) allows researchers to quantitatively compare labels across services and determine which of the output labels are reliable. This allows researchers to use these outputs in their research and addresses common critique for the scholarly use of such services (ref to image as data book).
The objective of this work was to (a) devise a method to assess the reliability of labels and (b) develop a graphical user interface allowing non-technical users to conduct this analysis. This objective aims to make image recognition tools available for humanities scholars and social scientists.
The underlying COSLAB was originally developed in Berg & Nelimarkka (2023), showing no systematic differences in the quality across different kinds of image datasets, thus suggesting that overall image recognition services can be used, particularly for explorative image analysis.
The graphical user interface provides non-technical frontend to image labelling services and COSLAB calculations. The drag & drop interface allows sending images for image recognition services and then calculates per-label scores, indicating if different image recognition services recognised similar things. The final output containing both the per-image labels and COSLAB scores can be exported e.g. to Microsoft Excel. This allows researchers to further use the results in their analysis tool of choice.
Berg, A., & Nelimarkka, M. (2023). Do you see what I see? Measuring the semantic differences in image‐recognition services’ outputs. In Journal of the Association for Information Science and Technology (Vol. 74, Issue 11, pp. 1307–1324). Wiley. https://doi.org/10.1002/asi.24827
FIN-CLARIAH project has received funding from the European Union – NextGenerationEU instrument and is funded by the Research Council of Finland under grant number 358720.
