Datasets

Filters:

OpenCSG

smoltalk-chinese

SmolTalk-Chinese: A multi-task Chinese conversational dataset covering 19 typical dialogue task scenarios.

License: Apache-2.0

Locale: zh

Task: LLM

Format: parquet

Size: 879.81 MB

Common Voice

Common Voice v24 English - en-AU subset for Everything Open 2026

Common Voice v24 English filtered on the `accent` field for Australian-related accents.

License: CC0-1.0

Locale: en-AU

Task: ASR

Format: CSV, MP3

Size: 1.92 GB

Institute of African Digital Humanities

Ewondo_Fong_ALCAM-MultimodalDataset

A multimodal linguistic resource comprising a curated datasheet of example sentences in Ewondo (Fong variety) and their French equivalents, along with their corresponding audio recordings and a sentence–audio alignment file. It is designed to support research, documentation and pedagogy in the field of speech and language technology for under-resourced African languages.

License: NOODL-1.0

Locale: ewo

Task: NLP

Format: MP3, TSV

Size: 16.80 MB

Amnesia

Informes de Actividades InfoCDMX (Ponencia Laura Enríquez)

El conjunto de datos se compone de los informes anuales de actividades y resultados del InfoCDMX, los cuales documentan el desempeño del organismo garante, el pleno y las ponencias de las personas comisionadas. Estos documentos son instrumentos fundamentales de rendición de cuentas que detallan la gestión institucional, la actividad cuasi-jurisdiccional (resolución de recursos de revisión y denuncias), y las

License: CC-BY-4.0

Locale: es-MX

Task: NLP

Format: PDF, XSLX

Size: 275.85 MB

Amnesia

Ficha de Documentación de Datos: Resoluciones InfoNL (Ponencia F. Guajardo)

Este conjunto de datos documenta la actividad resolutiva de la ponencia del Consejero Francisco Guajardo Martínez dentro del órgano garante de transparencia de Nuevo León. Cubre un periodo significativo de gestión (identificado preliminarmente entre 2018 y 2025), reflejando las disputas entre ciudadanos (solicitantes de información) y sujetos obligados (gobierno).

License: CC-BY-4.0

Locale: es-MX

Task: NLP

Format: PDF, XSLX

Size: 1.07 GB

ComparIA

Compar:IA conversations

French conversational AI conversation and preference dataset with 396K conversations from 50+ LLMs

License: Etalab 2.0

Locale: fr

Task: NLG

Format: PARQUET

Size: 1.81 GB

MEDIAMEN

English–Punjabi (Shahmukhi) Parallel Sentences Corpus (Mediamen Archives)

This parallel sentences corpus containing 30,405 aligned sentence pairs with a total of approximately 0.62 million tokens, curated from the archival materials of Mediamen (Advertising Agency). The sentences were professionally translated from English into Punjabi (Shahmukhi) and are intended to support machine translation, linguistic research, and Punjabi language technology development, particularly for real-world and contemporary language use.

License: CC-BY-NC-4.0

Locale: en-PK, pnb

Task: MT

Format: CSV

Size: 1.08 MB

Fundación Vía Libre

HESEIA Sentence Bias Dataset

This repository contains a dataset collected during the teacher training course HESEIA Sentence Bias (Tools for Exploring Biases and Artificial Intelligence). organized by Vía Libre, the Ministry of Education, and FAMAF-UNC. The course had an initial enrollment of 370 participating teachers, who also involved over 5,000 students in building a dataset that reflects stereotypes present in Argentina.

License: CC-BY-SA-4.0

Locale: es-AR

Task: OTH

Format: CSV

Size: 235.43 KB

RFERL

RFE/RL Tatar-Bashkir News Text Corpus

This dataset is longitudinal news corpus for the Tatar and Bashkir languages, sourced from Azatliq Radiosi, from 2001 to 2025, it contains over 105,000 articles (30M tokens).

License: CC-BY-NC-SA-4.0

Locale: tt,ba,ru

Task: NLP

Format: TXT

Size: 102.44 MB

Digital Divide Data

ddd-kenya-luhya-70hrs-asr

A collection of high-quality, crowdsourced audio recordings and verified transcriptions for the Luhya language, produced by Digital Divide Data.

License: CC-BY-4.0

Locale: luy

Task: ASR

Format: WAV, XLSX, TSV

Size: 13.90 GB

Effect AI

Effect AI Scripted Speech 1.0 - English

A collection of scripted spoken phrases in English.

License: CC0-1.0

Locale: en

Task: TTS

Format: CSV, MP3

Size: 663.45 MB

Amara Hub

DataTrust Africa: Speech Corpus of Public Radio Recordings from Northern Uganda

This is an open-access corpus of short clips of public radio content from Mega 100 FM, Q FM, Radio Pacis and Radio Rupiny in Northern Uganda. As of now, the online corpus has over 350 clips of recordings in English. We also hope to add finely-annotated transcripts to them. The dataset is for use in NLP research and non-commercial use. Upcoming datasets to look out for from Amara Hub are public radio recordings in other languages spoken in the region like Acholi, Lango, Lugbara and Akaramajong.

License: NOODL-1.0

Locale: en-US

Task: NLP

Format: MP3

Size: 179.82 MB

Digital Divide Data

Khmer ASR Cultural Dataset

37.62 hours manually curated speech-text pairs by native speakers in Khmer language about Cambodian cultural topics. On average, each recording is 8.29 seconds with the standard deviation of 3.87. Speaker metadata (gender, age group, and origin city) is provided.

License: CC-BY-SA-4.0

Locale: khm

Task: ASR

Format: WAV

Size: 12.59 GB

PT Pancaran Semangat Jaya

Corpus of Panjebar Semangat Javanese-Language Magazine

This dataset is a TXT-format collection compiled from three years of popular articles published in the Javanese-language weekly magazine Panjebar Semangat. It compiles widely read, non-academic Javanese texts reflecting contemporary themes and language use.

License: CC-BY-SA-4.0

Locale: Jav

Task: OTH

Format: TXT

Size: 4.31 MB

Center za jezikovne vire in tehnologije Univerze v Ljubljani

SI-NLI

SI-NLI (Slovene Natural Language Inference Dataset) contains 5,937 human-created Slovene sentence pairs (premise and hypothesis) that are manually labeled with the labels "entailment", "contradiction", and "neutral". We created the dataset using sentences that appear in the Slovenian reference corpus ccKres (http://hdl.handle.net/11356/1034). Annotators were tasked to modify the hypothesis in a candidate pair in a way that reflects one of the labels. The dataset is balanced since the annotators created three modifications (entailment, contradiction, neutral) for each candidate sentence pair. The dataset is split into train, validation, and test sets, with sizes of 4,392, 547, and 998. We used Slovenian pre-trained language models to create splits, thereby ensuring that difficult and easy instances are evenly distributed in all three subsets. The dataset is released in a tabular TSV format. The README.txt file contains a description of the attributes. Only the hypothesis and premise are given in the test set (i.e. no annotations) since SI-NLI is integrated into the Slovene evaluation framework SloBENCH (https://slobench.cjvt.si/). If you use the dataset to train your models, please consider submitting the test set predictions to SloBENCH to get the evaluation score and see how it compares to others.

License: CC-BY-NC-SA-4.0

Locale: sl

Task: NLU

Format: TSV

Size: 392.44 KB

Pro Svizra Rumantscha

Vallader Newspaper Corpus

6.2 million tokens in the Vallader variety of Romansh from the daily newspaper ”La Quotidiana”.

License: CC0-1.0

Locale: rm-vallader

Task: OTH

Format: TSV

Size: 18.71 MB

Kaleem Art Press

Multilingual Religious Parallel Corpus (Kaleem Art Press)

This dataset is a multilingual parallel sentences corpus containing 6,465 aligned sentence units with approximately 0.98 million words, curated from Kaleem Art Press archives. It includes parallel religious text data in Arabic, Urdu, Saraiki (standard and dialectal), Punjabi (Shahmukhi), and English, supporting research in machine translation, comparative linguistics, digital humanities, and low-resource language studies.

License: CC-BY-SA-4.0

Locale: mul

Task: MT

Format: CSV

Size: 2.27 MB

Sindh Line Publishers

The corpus contains 1.029 million tokens from the Sindh Line a Sindhi Newspaper published from the year 2024-2025. The text consists of the complete newspaper content including headlines, editorials, finance news and advertisements. The newspaper published in Karachi, Pakistan on daily basis

License: CC-BY-SA-4.0

Locale: snd

Task: NLP

Format: TXT

Size: 2.22 MB

Institute of African Digital Humanities

Spoken-Congolese-French-Dataset

The dataset consists of paired audio and text resources on spoken French from the Republic of the Congo. The audio files were extracted from longer recordings of semi-guided interviews conducted in Brazzaville, and orthographic transcriptions were added. The long audio recordings and their corresponding TRJS transcription files were automatically clipped alongside their respective transcriptions. The dataset comprises ten folders containing audio files and ten audio/text mapping files.

License: NOODL-1.0

Locale: fr-CG

Task: NLP

Format: MP3, WAV, TSV

Size: 3.44 GB

Institute of African Digital Humanities

Ewondo_Mbida-Mbani_ALCAM-MultimodalDataset

This dataset comprises a datasheet of Ewondo (Ewo) lexical entries collected in the speech area known as Mbida Mbani. Each entry is accompanied by illustrative sentences, word-by-word glosses and French translations. The resource is enriched with aligned audio recordings, making it suitable for linguistic analysis and speech technology development.

License: NOODL-1.0

Locale: ewo

Task: NLP

Format: MP3, TSV

Size: 19.25 MB

Balochi Academy

Balochi Academy Text Corpus

This corpus contains approximately 500k tokens of text from novels, poetry, articles, riddles, and proverbs, covering both literary and traditional genres. It is intended for linguistic research, NLP tasks (e.g., language modeling and text analysis), and cultural documentation.

License: CC-BY-NC-SA-4.0

Locale: bgn

Task: NLP

Format: TXT

Size: 1.88 MB

Institute of African Digital Humanities

Mada Narratives

This dataset contains 17 transcribed oral narratives in Mada (mxu), a language belonging to the Afro-Asiatic family that is spoken in Cameroon. The texts, derived from audio recordings of oral literature, reflect natural spoken discourse. This dataset can be used for language modelling, text analysis and other natural language processing (NLP) tasks.

License: NOODL-1.0

Locale: mxu

Task: NLP

Format: TXT

Size: 65.04 KB

Pro Svizra Rumantscha

Surmiran Newspaper Corpus

2.9 million tokens in the Surmiran variety of Romansh from the daily newspaper “La Quotidiana”.

License: CC0-1.0

Locale: rm-surmiran

Task: OTH

Format: TSV

Size: 11.89 MB

Maseno Centre for Applied Artificial Intelligence (MCAAI)

DhoNam: Dholuo Speech dataset

DhoNam: Dholuo Speech dataset is a speech corpus designed to supercharge Automatic Speech Recognition (ASR) and other speech technologies for Dholuo, one of Kenya’s major indigenous languages. This dataset contains native-speaker audio recordings collected through a platform where users read aloud a displayed sentence. The dataset includes the audio recordings and the corresponding prompt/sentence that was read.

License: NOODL-1.0

Locale: Luo

Task: ASR

Format: WEBM

Size: 2.49 GB