Opusparcus: Open Subtitles Paraphrase Corpus for Six Languages

Opusparcus is a paraphrase corpus for six European languages: German, English, Finnish, French, Russian, and Swedish. The paraphrases are extracted from the OpenSubtitles2016 corpus, which contains subtitles from movies and TV shows.

The data in Opusparcus has been extracted from OpenSubtitles2016 (http://opus.nlpl.eu/OpenSubtitles2016.php), which is in turn based on data from http://www.opensubtitles.org.

For each target language, the Opusparcus data have been partitioned into three types of data sets: training, development and test sets. The training sets are large, consisting of millions of sentence pairs, and have been compiled automatically, with the help of probabilistic ranking functions. The development and test sets consist of sentence pairs that have been annotated manually; each set contains approximately 1000 sentence pairs that have been verified to be acceptable paraphrases by two annotators.

Opusparcus is available for download at the Language Bank of Finland. The README file in the download folder contains detailed descriptions of the data sets.

Please cite the following paper in any work that utilizes any part of the Opusparcus corpus:
Mathias Creutz (2018). Open Subtitles Paraphrase Corpus for Six Languages. In Proceedings of the 11th edition of the Language Resources and Evaluation Conference (LREC 2018), 7-12 May, Miyazaki, Japan.

Of this language corpus different versions are (or might be in the future) published in the Language Bank of Finland. The versions are available through the Language Bank Download Service and/or through the Korp concordance tool. The links to the different versions can be found from the list above.

Detailed information on the content of each version, user rights and licenses can be found from it’s specific metadata record in META-SHARE.

This resource group page has a Persistent Identifier: http://urn.fi/urn:nbn:fi:lb-2021081203