Skip to the main content.
MTQE Lite DAAIT
Facebook Instagram Linkedin X YouTube Medium GitHub
AI LANGUAGE CAPABILITIES

AI language capabilities across 250+ languages and dialects

Explore Pangeanic's structured language capability matrix covering language identification, text and speech collection, transcription, annotation, fine-tuning, evaluation, cultural relevance and red-teaming capabilities.

Explore the language matrix Talk to our AI data team
261
Language and variety capability units

Each row represents a structured language or language/locale capability unit that can be used for multilingual AI data sourcing, training, evaluation and localization workflows.

DIRECT ANSWER

What does the language capability matrix show?

Pangeanic's language capability matrix maps language and locale-specific AI data capabilities across text collection, speech collection, transcription, annotation, supervised fine-tuning, RLHF, STEM evaluation, cultural evaluation, translation evaluation and red-teaming.

The matrix is designed to show the difference between broad language support and specific, language-level AI data capabilities. A language or regional variety can therefore appear as its own capability unit where its data, cultural or evaluation requirements differ.

HOW TO READ THE MATRIX

Structured language data for multilingual AI

The matrix combines language identifiers with data collection, annotation, training and evaluation capabilities. Capability statuses can differ by language, market, modality and project scope.

01

Language identity

ISO 639 identifiers, BCP 47 locale information, region, country and language or variety type.

02

Data capabilities

Text, speech, transcription and annotation capabilities relevant to AI training and production workflows.

03

Training & evaluation

SFT, RLHF, STEM, cultural, translation and red-teaming capabilities at language level.

LANGUAGE CAPABILITY MATRIX

Explore language and locale capabilities

Search by language, ISO code, BCP 47 locale, country or region. Use the filters to narrow the matrix to the capabilities you need.

⌕
261 of 261 rows
Capability status P Project-ready S Specialist-scoped — Not currently listed
Language / variety ISO 639-1 ISO 639-3 BCP 47 Region Country / market Type Tier Annotation Text collection Speech collection Transcription SFT RLHF STEM evaluation Cultural evaluation Translation evaluation Red teaming
Abkhazian ab abk ab-GE Central Asia & Caucasus Georgia Language Project-ready P P P P P S — S P —
Afar aa aar aa-DJ Sub-Saharan Africa Djibouti Language Specialist-scoped P P S S S S — S S —
Afrikaans af afr af-ZA Sub-Saharan Africa South Africa Language Established ✓ ✓ P P P P — ✓ ✓ —
Akan ak aka ak-GH Sub-Saharan Africa Ghana Language Project-ready P P P P P S — S P —
Albanian sq sqi sq-AL Europe Albania Language Established ✓ ✓ P S P P — ✓ ✓ —
Algerian Arabic arq arq-DZ Middle East & North Africa Algeria Language / variety Project-ready P P P P P S — S P —
Amharic am amh am-ET Sub-Saharan Africa Ethiopia Language Project-ready P P P P P S — S P —
Aragonese an arg an-ES Europe Spain Regional / minority language Project-ready P P P P P S — S P —
Armenian (Armenia) hy hye hy-AM Central Asia & Caucasus Armenia Regional variety Established ✓ ✓ P S P P — ✓ ✓ —
Assamese as asm as-IN South Asia India Language Established ✓ ✓ P S P P — ✓ ✓ —
Asturian ast ast-ES Europe Spain Language Specialist-scoped P P S S S S — S S —
Azerbaijani az aze az-AZ Central Asia & Caucasus Azerbaijan Language Established ✓ ✓ P S P P — ✓ ✓ —
Bambara bm bam bm-ML Sub-Saharan Africa Mali Language Project-ready P P P P P S — S P —
Bangla (Bangladesh) bn ben bn-BD South Asia Bangladesh Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Baoulé bci bci-CI Sub-Saharan Africa Côte d’Ivoire Language Specialist-scoped P P S S S S — S S —
Basque eu eus eu-ES Europe Spain Language Established ✓ ✓ P P P P — ✓ ✓ S
Bassari bsc bsc-SN Sub-Saharan Africa Senegal Language Specialist-scoped P P S S S S — S S —
Bavarian bar bar-AT Europe Austria Language Specialist-scoped P P S S S S — S S —
Beja bej bej-SD Sub-Saharan Africa Sudan Language Specialist-scoped P P S S S S — S S —
Belarusian be bel be-BY Europe Belarus Language Specialist-scoped P P S S S S — S S —
Bemba bem bem-ZM Sub-Saharan Africa Zambia Language Specialist-scoped P P S S S S — S S —
Bhojpuri (India) bho bho-IN South Asia India Language / regional variety Project-ready P P P P P S — S P —
Bosnian bs bos bs-BA Europe Bosnia & Herzegovina Language Established ✓ ✓ P S P P — ✓ ✓ —
Bube bvb bvb-GQ Sub-Saharan Africa Equatorial Guinea Language Specialist-scoped P P S S S S — S S —
Bulgarian bg bul bg-BG Europe Bulgaria Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Burmese my mya my-MM Southeast Asia Myanmar (Burma) Language Specialist-scoped P P S S S S — S S —
Cantonese (China) yue yue-CN East Asia China Regional variety Project-ready P P P P P S — S P —
Cantonese (Hong Kong SAR China) yue yue-HK East Asia Hong Kong SAR China Regional variety Project-ready P P P P P S — S P —
Catalan (Andorra) ca cat ca-AD Europe Andorra Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Catalan (Spain) ca cat ca-ES Europe Spain Language / regional variety Established ✓ ✓ P P P P S ✓ ✓ S
Cebuano ceb ceb-PH Southeast Asia Philippines Language Project-ready P P P P P S — S P —
Central Kurdish ckb ckb-IQ Middle East & North Africa Iraq Language Project-ready P P P P P S — S P —
Chechen ce che ce-RU Europe Russia Language Specialist-scoped P P S S S S — S S —
Chiga cgg cgg-UG Sub-Saharan Africa Uganda Language Specialist-scoped P P S S S S — S S —
Croatian (Bosnia & Herzegovina) hr hrv hr-BA Europe Bosnia & Herzegovina Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Croatian (Croatia) hr hrv hr-HR Europe Croatia Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Czech cs ces cs-CZ Europe Czechia Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Danish (Denmark) da dan da-DK Europe Denmark Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Dari (Afghanistan) fa prs fa-AF South Asia Afghanistan Language / variety Established ✓ ✓ ✓ ✓ P P — ✓ ✓ S
Divehi dv div dv-MV South Asia Maldives Language Project-ready P P P P P S — S P —
Dutch (Belgium) nl nld nl-BE Europe Belgium Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Dutch (Netherlands) nl nld nl-NL Europe Netherlands Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Dutch (Suriname) nl nld nl-SR Americas Suriname Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Dyula dyu dyu-BF Sub-Saharan Africa Burkina Faso Language Project-ready P P P P P S — S P —
Dzongkha dz dzo dz-BT South Asia Bhutan Language Project-ready P P P P P S — S P —
Egyptian Arabic arz arz-EG Middle East & North Africa Egypt Language / variety Project-ready P P P P P S — S P —
Emirati Gulf Arabic afb afb-AE Middle East & North Africa United Arab Emirates Dialect / regional variety Project-ready P P P P P S — S P —
English (Australia) en eng en-AU Oceania & Pacific Australia Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (Canada) en eng en-CA Americas Canada Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (India) en eng en-IN South Asia India Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (Ireland) en eng en-IE Europe Ireland Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (Jamaica) en eng en-JM Americas Jamaica Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (New Zealand) en eng en-NZ Oceania & Pacific New Zealand Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (Nigeria) en eng en-NG Sub-Saharan Africa Nigeria Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (Philippines) en eng en-PH Southeast Asia Philippines Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (Singapore) en eng en-SG Southeast Asia Singapore Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (South Africa) en eng en-ZA Sub-Saharan Africa South Africa Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (Trinidad & Tobago) en eng en-TT Americas Trinidad & Tobago Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (United Kingdom) en eng en-GB Europe United Kingdom Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
English (United States) en eng en-US Americas United States Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Estonian et est et-EE Europe Estonia Language Established ✓ ✓ P S P P — ✓ ✓ —
Ewe ee ewe ee-GH Sub-Saharan Africa Ghana Language Project-ready P P P P P S — S P —
Fang fan fan-GQ Sub-Saharan Africa Equatorial Guinea Language Specialist-scoped P P S S S S — S S —
Faroese fo fao fo-FO Europe Faroe Islands Language Specialist-scoped P P S S S S — S S —
Filipino fil fil-PH Southeast Asia Philippines Language Project-ready P P P P P S — S P —
Finnish fi fin fi-FI Europe Finland Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Fon fon fon-BJ Sub-Saharan Africa Benin Language Project-ready P P P P P S — S P —
French (Algeria) fr fra fr-DZ Middle East & North Africa Algeria Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
French (Belgium) fr fra fr-BE Europe Belgium Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
French (Cameroon) fr fra fr-CM Sub-Saharan Africa Cameroon Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
French (Canada) fr fra fr-CA Americas Canada Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
French (Congo - Kinshasa) fr fra fr-CD Sub-Saharan Africa Congo - Kinshasa Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
French (France) fr fra fr-FR Europe France Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
French (Lebanon) fr fra fr-LB Middle East & North Africa Lebanon Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
French (Morocco) fr fra fr-MA Middle East & North Africa Morocco Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
French (Switzerland) fr fra fr-CH Europe Switzerland Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
French (Tunisia) fr fra fr-TN Middle East & North Africa Tunisia Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Fula ff ful ff-SN Sub-Saharan Africa Senegal Language Project-ready P P P P P S — S P —
Galician gl glg gl-ES Europe Spain Language Established ✓ ✓ P P P P — ✓ ✓ S
Ganda lg lug lg-UG Sub-Saharan Africa Uganda Language Project-ready P P P P P S — S P —
Georgian ka kat ka-GE Central Asia & Caucasus Georgia Language Established ✓ ✓ P S P P — ✓ ✓ —
German (Austria) de deu de-AT Europe Austria Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
German (Germany) de deu de-DE Europe Germany Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
German (Switzerland) de deu de-CH Europe Switzerland Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Greek (Cyprus) el ell el-CY Europe Cyprus Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Greek (Greece) el ell el-GR Europe Greece Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Guarani gn grn gn-PY Americas Paraguay Language Specialist-scoped P P S S S S — S S —
Guianese Creole French gcr gcr-GF Americas French Guiana Language / variety Specialist-scoped P P S S S S — S S —
Gujarati gu guj gu-IN South Asia India Language Established ✓ ✓ P P P P — ✓ ✓ —
Hadrami Arabic ayh ayh-YE Middle East & North Africa Yemen Dialect / regional variety Project-ready P P P P P S — S P —
Haitian Creole ht hat ht-HT Americas Haiti Language / variety Specialist-scoped P P S S S S — S S —
Hakka Chinese (Taiwan) hak hak-TW East Asia Taiwan Language / regional variety Project-ready P P P P P S — S P —
Hausa ha hau ha-NE Sub-Saharan Africa Niger Language Project-ready P P P P P S — S P —
Hawaiian haw haw-US Americas United States Language Specialist-scoped P P S S S S — S S —
Hebrew he heb he-IL Middle East & North Africa Israel Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Herero hz her hz-NA Sub-Saharan Africa Namibia Language Specialist-scoped P P S S S S — S S —
Hijazi Arabic acw acw-SA Middle East & North Africa Saudi Arabia Dialect / regional variety Project-ready P P P P P S — S P —
Hindi hi hin hi-IN South Asia India Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Hungarian hu hun hu-HU Europe Hungary Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Icelandic is isl is-IS Europe Iceland Language Established ✓ ✓ P S P P — ✓ ✓ —
Igbo ig ibo ig-NG Sub-Saharan Africa Nigeria Language Project-ready P P P P P S — S P —
Indonesian id ind id-ID Southeast Asia Indonesia Language Established ✓ ✓ ✓ ✓ P P — ✓ ✓ S
Iraqi Arabic (Mesopotamian) acm acm-IQ Middle East & North Africa Iraq Dialect / regional variety Project-ready P P P P P S — S P —
Irish ga gle ga-IE Europe Ireland Language Established ✓ ✓ P S P P — ✓ ✓ —
Italian (Italy) it ita it-IT Europe Italy Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Italian (Switzerland) it ita it-CH Europe Switzerland Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Jamaican Creole English jam jam-JM Americas Jamaica Language / variety Specialist-scoped P P S S S S — S S —
Japanese ja jpn ja-JP East Asia Japan Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Javanese jv jav jv-ID Southeast Asia Indonesia Language Project-ready P P P P P S — S P —
Kabuverdianu kea kea-CV Sub-Saharan Africa Cape Verde Language Specialist-scoped P P S S S S — S S —
Kalanga kck kck-ZW Sub-Saharan Africa Zimbabwe Language Specialist-scoped P P S S S S — S S —
Kannada kn kan kn-IN South Asia India Language Established ✓ ✓ P S P P — ✓ ✓ —
Kashmiri ks kas ks-IN South Asia India Language Project-ready P P P P P S — S P —
Kazakh kk kaz kk-KZ Central Asia & Caucasus Kazakhstan Language Established ✓ ✓ P S P P — ✓ ✓ —
Khmer km khm km-KH Southeast Asia Cambodia Language Specialist-scoped P P S S S S — S S —
Kimbundu kmb kmb-AO Sub-Saharan Africa Angola Language Specialist-scoped P P S S S S — S S —
Kinyarwanda rw kin rw-RW Sub-Saharan Africa Rwanda Language Project-ready P P P P P S — S P —
Konkani kok kok-IN South Asia India Language Project-ready P P P P P S — S P —
Korean (China) ko kor ko-CN East Asia China Regional variety Established ✓ ✓ P P P P S ✓ ✓ S
Korean (South Korea) ko kor ko-KR East Asia South Korea Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Kuwaiti Gulf Arabic afb afb-KW Middle East & North Africa Kuwait Dialect / regional variety Project-ready P P P P P S — S P —
Kyrgyz ky kir ky-KG Central Asia & Caucasus Kyrgyzstan Language Established ✓ ✓ P S P P — ✓ ✓ —
Lao lo lao lo-LA Southeast Asia Laos Language Specialist-scoped P P S S S S — S S —
Latin la lat la-VA Europe Vatican City Language Specialist-scoped P P S S S S — S S —
Latvian lv lav lv-LV Europe Latvia Language Established ✓ ✓ P S P P — ✓ ✓ —
Libyan Arabic ayl ayl-LY Middle East & North Africa Libya Dialect / regional variety Project-ready P P P P P S — S P —
Lingala ln lin ln-CD Sub-Saharan Africa Congo - Kinshasa Language Project-ready P P P P P S — S P —
Lithuanian lt lit lt-LT Europe Lithuania Language Established ✓ ✓ P S P P — ✓ ✓ —
Lomwe ngl ngl-MZ Sub-Saharan Africa Mozambique Language Specialist-scoped P P S S S S — S S —
Lule Sami smj smj-SE Europe Sweden Minority language Project-ready P P P P P S — S P —
Luxembourgish lb ltz lb-LU Europe Luxembourg Language Project-ready P P P P P S — S P —
Macedonian mk mkd mk-MK Europe North Macedonia Language Established ✓ ✓ P S P P — ✓ ✓ —
Maithili (India) mai mai-IN South Asia India Regional variety Project-ready P P P P P S — S P —
Makhuwa vmw vmw-MZ Sub-Saharan Africa Mozambique Language Specialist-scoped P P S S S S — S S —
Malagasy mg mlg mg-MG Sub-Saharan Africa Madagascar Language Project-ready P P P P P S — S P —
Malay (Malaysia) ms msa ms-MY Southeast Asia Malaysia Regional variety Established ✓ ✓ ✓ ✓ P P — ✓ ✓ S
Malay (Singapore) ms msa ms-SG Southeast Asia Singapore Regional variety Established ✓ ✓ ✓ ✓ P P — ✓ ✓ S
Malayalam ml mal ml-IN South Asia India Language Established ✓ ✓ P S P P — ✓ ✓ —
Maltese mt mlt mt-MT Europe Malta Language Established ✓ ✓ P S P P — ✓ ✓ —
Mandarin Chinese (Simplified) zh cmn zh-Hans-CN East Asia China Language / regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Mandarin Chinese (Traditional) zh cmn zh-Hant-TW East Asia Taiwan Language / regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Manx gv glv gv-IM Europe Isle of Man Language Specialist-scoped P P S S S S — S S —
Marathi mr mar mr-IN South Asia India Language Established ✓ ✓ P P P P — ✓ ✓ —
Marshallese mh mah mh-MH Oceania & Pacific Marshall Islands Language Specialist-scoped P P S S S S — S S —
Modern Standard Arabic ar ara ar Middle East & North Africa Multiple Standard / written variety Project-ready P P P P P S — S P —
Moksha mdf mdf-RU Europe Russia Language Specialist-scoped P P S S S S — S S —
Mongolian mn mon mn-MN East Asia Mongolia Language Project-ready P P P P P S — S P —
Moroccan Arabic ary ary-MA Middle East & North Africa Morocco Language / variety Project-ready P P P P P S — S P —
Najdi Arabic ars ars-SA Middle East & North Africa Saudi Arabia Dialect / regional variety Project-ready P P P P P S — S P —
Nauru na nau na-NR Oceania & Pacific Nauru Language Specialist-scoped P P S S S S — S S —
Nepali (Nepal) ne nep ne-NP South Asia Nepal Regional variety Established ✓ ✓ ✓ ✓ P P — ✓ ✓ —
North Levantine Arabic (Lebanon) apc apc-LB Middle East & North Africa Lebanon Language / variety Project-ready P P P P P S — S P —
North Levantine Arabic (Syria) apc apc-SY Middle East & North Africa Syria Language / variety Project-ready P P P P P S — S P —
Norwegian no nor no-NO Europe Norway Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Norwegian Bokmål nb nob nb-NO Europe Norway Language Specialist-scoped P P S S S S — S S —
Norwegian Nynorsk nn nno nn-NO Europe Norway Language Specialist-scoped P P S S S S — S S —
Odia or ori or-IN South Asia India Language Established ✓ ✓ P S P P — ✓ ✓ —
Omani Arabic acx acx-OM Middle East & North Africa Oman Dialect / regional variety Project-ready P P P P P S — S P —
Palauan pau pau-PW Oceania & Pacific Palau Language Specialist-scoped P P S S S S — S S —
Pashto (Afghanistan) ps pus ps-AF South Asia Afghanistan Regional variety Project-ready P P P P P S — S P —
Pashto (Pakistan) ps pus ps-PK South Asia Pakistan Regional variety Project-ready P P P P P S — S P —
Persian (Iran) fa fas fa-IR Middle East & North Africa Iran Regional variety Established ✓ ✓ ✓ ✓ P P — ✓ ✓ S
Polish pl pol pl-PL Europe Poland Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Portuguese (Angola) pt por pt-AO Sub-Saharan Africa Angola Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Portuguese (Brazil) pt por pt-BR Americas Brazil Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Portuguese (Cape Verde) pt por pt-CV Sub-Saharan Africa Cape Verde Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Portuguese (Guinea-Bissau) pt por pt-GW Sub-Saharan Africa Guinea-Bissau Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Portuguese (Mozambique) pt por pt-MZ Sub-Saharan Africa Mozambique Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Portuguese (Portugal) pt por pt-PT Europe Portugal Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Punjabi pa pan pa-IN South Asia India Language Established ✓ ✓ P P P P — ✓ ✓ —
Qatari Gulf Arabic afb afb-QA Middle East & North Africa Qatar Dialect / regional variety Project-ready P P P P P S — S P —
Quechua (Bolivia) qu que qu-BO Americas Bolivia Regional variety Specialist-scoped P P S S S S — S S —
Quechua (Ecuador) qu que qu-EC Americas Ecuador Regional variety Specialist-scoped P P S S S S — S S —
Quechua (Peru) qu que qu-PE Americas Peru Regional variety Specialist-scoped P P S S S S — S S —
Rangpuri rkt rkt-BD South Asia Bangladesh Language Project-ready P P P P P S — S P —
Romanian (Moldova) ro ron ro-MD Europe Moldova Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Romanian (Romania) ro ron ro-RO Europe Romania Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Russian (Belarus) ru rus ru-BY Europe Belarus Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Russian (Kazakhstan) ru rus ru-KZ Central Asia & Caucasus Kazakhstan Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Russian (Russia) ru rus ru-RU Europe Russia Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Russian (Ukraine) ru rus ru-UA Europe Ukraine Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Samoan sm smo sm-WS Oceania & Pacific Samoa Language Specialist-scoped P P S S S S — S S —
Sanskrit sa san sa-IN South Asia India Language Project-ready P P P P P S — S P —
Santali sat sat-IN South Asia India Language Project-ready P P P P P S — S P —
Saraiki skr skr-PK South Asia Pakistan Language Project-ready P P P P P S — S P —
Sardinian sc srd sc-IT Europe Italy Language Project-ready P P P P P S — S P —
Scottish Gaelic gd gla gd-GB Europe United Kingdom Language Specialist-scoped P P S S S S — S S —
Serbian (Bosnia & Herzegovina) sr srp sr-BA Europe Bosnia & Herzegovina Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Serbian (Serbia) sr srp sr-RS Europe Serbia Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Sindhi (India) sd snd sd-IN South Asia India Regional variety Project-ready P P P P P S — S P —
Sindhi (Pakistan) sd snd sd-PK South Asia Pakistan Regional variety Project-ready P P P P P S — S P —
Sinhala si sin si-LK South Asia Sri Lanka Language Established ✓ ✓ ✓ ✓ P P — ✓ ✓ —
Slave den den-CA Americas Canada Language Specialist-scoped P P S S S S — S S —
Slovak sk slk sk-SK Europe Slovakia Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Slovenian sl slv sl-SI Europe Slovenia Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Somali so som so-SO Sub-Saharan Africa Somalia Language Project-ready P P P P P S — S P —
South Levantine Arabic (Jordan) ajp ajp-JO Middle East & North Africa Jordan Dialect / regional variety Project-ready P P P P P S — S P —
South Levantine Arabic (Palestine) ajp ajp-PS Middle East & North Africa Palestinian Territories Dialect / regional variety Project-ready P P P P P S — S P —
Spanish (Argentina) es spa es-AR Americas Argentina Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Belize) es spa es-BZ Americas Belize Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Bolivia) es spa es-BO Americas Bolivia Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Chile) es spa es-CL Americas Chile Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Colombia) es spa es-CO Americas Colombia Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Costa Rica) es spa es-CR Americas Costa Rica Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Cuba) es spa es-CU Americas Cuba Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Dominican Republic) es spa es-DO Americas Dominican Republic Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Ecuador) es spa es-EC Americas Ecuador Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (El Salvador) es spa es-SV Americas El Salvador Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Guatemala) es spa es-GT Americas Guatemala Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Honduras) es spa es-HN Americas Honduras Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Mexico) es spa es-MX Americas Mexico Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Nicaragua) es spa es-NI Americas Nicaragua Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Panama) es spa es-PA Americas Panama Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Paraguay) es spa es-PY Americas Paraguay Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Peru) es spa es-PE Americas Peru Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Philippines) es spa es-PH Southeast Asia Philippines Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Puerto Rico) es spa es-PR Americas Puerto Rico Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Spain) es spa es-ES Europe Spain Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (United States) es spa es-US Americas United States Regional variety Established ✓ ✓ P P P P S ✓ ✓ S
Spanish (Uruguay) es spa es-UY Americas Uruguay Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Spanish (Venezuela) es spa es-VE Americas Venezuela Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Sudanese Arabic apd apd-SD Sub-Saharan Africa Sudan Dialect / regional variety Project-ready P P P P P S — S P —
Sundanese su sun su-ID Southeast Asia Indonesia Language Project-ready P P P P P S — S P —
Swahili (Kenya) sw swa sw-KE Sub-Saharan Africa Kenya Regional variety Established ✓ ✓ ✓ ✓ P P — ✓ ✓ —
Swahili (Tanzania) sw swa sw-TZ Sub-Saharan Africa Tanzania Regional variety Established ✓ ✓ ✓ ✓ P P — ✓ ✓ —
Swahili (Uganda) sw swa sw-UG Sub-Saharan Africa Uganda Regional variety Established ✓ ✓ ✓ ✓ P P — ✓ ✓ —
Swati ss ssw ss-SZ Sub-Saharan Africa Eswatini Language Specialist-scoped P P S S S S — S S —
Swedish (Sweden) sv swe sv-SE Europe Sweden Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Sylheti syl syl-BD South Asia Bangladesh Language Project-ready P P P P P S — S P —
Tajik tg tgk tg-TJ Central Asia & Caucasus Tajikistan Language Established ✓ ✓ P S P P — ✓ ✓ —
Tamashek tmh tmh-NE Sub-Saharan Africa Niger Language Specialist-scoped P P S S S S — S S —
Tamil (India) ta tam ta-IN South Asia India Regional variety Established ✓ ✓ P P P P — ✓ ✓ —
Tamil (Sri Lanka) ta tam ta-LK South Asia Sri Lanka Regional variety Established ✓ ✓ ✓ ✓ P P — ✓ ✓ —
Tatar tt tat tt-RU Europe Russia Language Specialist-scoped P P S S S S — S S —
Telugu te tel te-IN South Asia India Language Established ✓ ✓ P P P P — ✓ ✓ —
Tetum tet tet-TL Southeast Asia Timor-Leste Language Project-ready P P P P P S — S P —
Thai th tha th-TH Southeast Asia Thailand Language Established ✓ ✓ ✓ ✓ P P — ✓ ✓ S
Tibetan bo bod bo-CN East Asia China Language Project-ready P P P P P S — S P —
Tonga (Zambia) toi toi-ZM Sub-Saharan Africa Zambia Language Specialist-scoped P P S S S S — S S —
Tongan to ton to-TO Oceania & Pacific Tonga Language Specialist-scoped P P S S S S — S S —
Tsonga ts tso ts-ZA Sub-Saharan Africa South Africa Language Project-ready P P P P P S — S P —
Tswana tn tsn tn-BW Sub-Saharan Africa Botswana Language Project-ready P P P P P S — S P —
Tunisian Arabic aeb aeb-TN Middle East & North Africa Tunisia Language / variety Project-ready P P P P P S — S P —
Turkish (Cyprus) tr tur tr-CY Europe Cyprus Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Turkish (Türkiye) tr tur tr-TR Middle East & North Africa Türkiye Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Turkmen (Turkmenistan) tk tuk tk-TM Central Asia & Caucasus Turkmenistan Regional variety Established ✓ ✓ P S P P — ✓ ✓ —
Tuvalu tvl tvl-TV Oceania & Pacific Tuvalu Language Specialist-scoped P P S S S S — S S —
Ukrainian uk ukr uk-UA Europe Ukraine Language Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Urdu (India) ur urd ur-IN South Asia India Regional variety Established ✓ ✓ P P P P S ✓ ✓ S
Urdu (Pakistan) ur urd ur-PK South Asia Pakistan Regional variety Established ✓ ✓ ✓ ✓ P P S ✓ ✓ S
Uyghur ug uig ug-CN East Asia China Language Project-ready P P P P P S — S P —
Uzbek (Uzbekistan) uz uzb uz-UZ Central Asia & Caucasus Uzbekistan Regional variety Established ✓ ✓ P S P P — ✓ ✓ —
Valencian ca cat ca-ES-valencia Europe Spain Regional variety Established ✓ ✓ P P P P S ✓ ✓ S
Venetian vec vec-IT Europe Italy Language / regional variety Specialist-scoped P P S S S S — S S —
Vietnamese vi vie vi-VN Southeast Asia Vietnam Language Established ✓ ✓ ✓ ✓ P P — ✓ ✓ S
Welsh cy cym cy-GB Europe United Kingdom Language Established ✓ ✓ P S P P — ✓ ✓ —
Western Panjabi lah lah-PK South Asia Pakistan Language Project-ready P P P P P S — S P —
Wolof wo wol wo-SN Sub-Saharan Africa Senegal Language Project-ready P P P P P S — S P —
Xhosa xh xho xh-ZA Sub-Saharan Africa South Africa Language Project-ready P P P P P S — S P —
Yoruba yo yor yo-NG Sub-Saharan Africa Nigeria Language Project-ready P P P P P S — S P —
Zulu zu zul zu-ZA Sub-Saharan Africa South Africa Language Project-ready P P P P P S — S P —
?

No language capabilities found

Try changing your search or removing one of the filters.

LANGUAGE VARIETY MATTERS

One language does not always mean one AI capability.

AI systems can require different data collection, annotation, evaluation and cultural-context workflows depending on the language variety, locale, market and intended use case.

This is why the matrix can distinguish languages, regional dialects and locale-specific capability units rather than reducing multilingual AI support to a single list of language names.

LANGUAGE

Arabic

Multiple regional varieties can require different speech, cultural and evaluation workflows.

LOCALE

Omani Arabic

A locale-specific capability unit can identify the exact speech, market and cultural context relevant to an AI project.

BCP 47

ar-OM

Locale identifiers make language capabilities more useful for structured data, AI systems and future machine-readable workflows.

DATA COLLECTION CAPABILITIES

Explore Pangeanic's data capabilities by data type

Pangeanic collects, curates and prepares data across multiple modalities and AI use cases. Explore our capabilities by data type, from multilingual parallel corpora and speech to image, video, multimodal and instruction-tuning data.

01 Multilingual

Parallel Corpora

Multilingual aligned datasets for machine translation, cross-lingual AI, LLM training, terminology management and domain adaptation across languages and language pairs.

  • Translation pairs
  • Multilingual aligned segments
  • Domain-specific corpora
  • Terminology and linguistic metadata
Explore parallel corpora →
02 Audio

Speech & Audio

Speech data for ASR, TTS, conversational AI and multilingual voice systems, including diverse speakers, accents, languages, domains and recording environments.

  • ASR datasets
  • TTS and voice data
  • Conversational speech
  • Transcription and metadata
Explore speech data →
03 Multimodal

Video & Multimodal

Video and multimodal datasets combining visual, audio and textual information for computer vision, multimodal LLMs, retrieval, reasoning and AI evaluation.

  • Video annotation
  • Scene and event detection
  • Temporal segmentation
  • Text, image and video alignment
Explore video & multimodal data →
04 Vision

Image Data

Structured and annotated image datasets for computer vision, OCR, classification, object detection, visual search and vision-language model development.

  • Image classification
  • Object detection
  • OCR and document imagery
  • Visual metadata and annotation
Explore image data →
05 Text

Monolingual Text

Large-scale monolingual corpora for language model training, domain adaptation, retrieval, semantic search and language-specific AI systems.

  • Domain-specific text
  • LLM training corpora
  • Language adaptation
  • Metadata-enriched text
Explore monolingual data →
06 Alignment

Instruction Tuning Data

Question-answer, prompt-response and dialogue datasets for supervised fine-tuning, instruction tuning, conversational AI, assistants and agentic workflows.

  • Question-answer datasets
  • Instruction-following data
  • Dialogue and conversational data
  • Human-reviewed alignment data
Explore instruction data →
FAQ

Frequently asked questions about language capabilities

What is a language capability unit? +
A language capability unit represents a specific language, language variety or locale together with its relevant AI data, training and evaluation capabilities.
Why are languages and dialects listed separately? +
Different language dialects and markets can require different speech collection, annotation, cultural relevance and evaluation workflows. The matrix therefore makes these distinctions explicit.
What do P and S mean in the capability fields? +
P indicates a project-ready capability, while S indicates a specialist-scoped capability. A dash indicates that the capability is not currently listed for that language in the matrix.
Can Pangeanic collect data for languages not listed? +
Yes. This language capability matrix is not exhaustive. If a language or language variety is not listed, Pangeanic can review the specific requirements of the project and assess whether the required data collection, annotation, speech, transcription, evaluation or other AI data services can be supported.
How can I discuss a specific language requirement? +
Contact the Pangeanic AI data team with the language, locale, modality, volume, quality requirements and intended AI use case.
BUILD MULTILINGUAL AI

Need a language or capability not shown here?

From language-specific speech collection and annotation to multilingual LLM training, evaluation and cultural relevance, Pangeanic can support custom AI data requirements at scale.

Contact Pangeanic Explore AI datasets
Datasets
AI Data Services
Sovereign AI
Language Technologies
Use Cases

© 2026 Pangeanic S.L.

Facebook X Instagram Linkedin YouTube GitHub

Pangeanic engineers production-grade multilingual AI data and secure language technologies. We supply licensed datasets, execute bespoke annotation, evaluation, and alignment workflows, and deploy sovereign AI infrastructure for enterprises, AI labs, and public administrations that require control over data, models, and governance.

Return to top