<< List of all deliverables

D1.3.1: Tools and guidelines for video processing

Project: FIN-CLARIAH
Grant agreement: Research Council of Finland no. 358720
Start date: 01-01-2024
Duration: 24 months

Report Author:: Sam Hardwick (CSC)
WP 1.3: Report on Tools and guidelines for video processing
Date of reporting: 15-05-2025
Contributors: Anni Järvenpää (CSC)

The foreseen impact the work package was

to make it easier for researchers to use, manage, annotate and share collections of video recordings as research data

To that end, we have provided software tools via the computational infrastructure of CSC (a FIN-CLARIAH member) and documentation through our Language Bank service.

Software provided in the computational infrastructure

CSC’s computational infrastructure has the potential to host quite large and computationally intensive video workflows:

• The Allas object storage system provides high capacity networked storage
• The HPC (High Performance Computing) platforms Puhti and Mahti provide processing power

Transfer speeds between these services is high. However, use of these systems for social sciences and humanities research is still developing. On HPC, we have ensured the availability and functioning of the audiovisual processing tool ffmpeg, the audio editing tool SoX and the audio feature extraction and classification suite openSMILE.

Documentation and support for most relevant software

For AV-related software that is especially geared towards HSS users, our supported tools listing has articles for

• Elan, an audio and video annotation tool (including a video guide)
• Praat, a tool for phonetic analysis (including a video guide)
• openSMILE, an audio feature extraction and classification toolkit

To demonstrate combining multiple tools in a practical workflow, we wrote a hands-on tutorial for automatically adding subtitle tracks to video files containing speech. The tutorial combines ffmpeg, openSMILE, and LB’s own speech recognition API for Finnish, and it discusses video container formats and scripting a workflow with Python.

References

• • For the tools listings discussed, see https://www.kielipankki.fi/tools/
  • For the hands-on tutorial, see https://www.kielipankki.fi/support/subtitling-videos-with-ffmpeg-and-opensmile/ (also available in Finnish)
  • For the Elan support page, see https://www.kielipankki.fi/tuki/elan/ (Finnish only)
  • For the Praat support page, see https://www.kielipankki.fi/tuki/praat/ (Finnish only)

The FIN-CLARIAH project has received funding from the European Union – NextGenerationEU instrument and is funded by the Research Council of Finland under grant number 358720.