cis-lmu
/

glotlid

Text Classification

fastText

language-identification

Model card Files Files and versions

xet

Community

kargaranamir commited on Apr 18, 2024

Commit

85cd671

1 Parent(s): 6877193

update list of languages and License

Browse files

Files changed (2) hide show

LICENSE +253 -0
README.md +364 -37

LICENSE ADDED Viewed

	@@ -0,0 +1,253 @@

+Apache License 2.0 plus notices
+---
+### Additional Notice on Training Data and Model Behavior
+This software includes a machine learning model for language identification
+based on statistical representations (e.g., character n-grams and linear classification).
+The model was trained on a mixture of publicly available data sources, which may
+include materials subject to copyright or other rights.
+The model does not store or encode expressive text content. Instead, it represents
+aggregated statistical patterns (such as n-gram frequency weights) that are not
+intended to reproduce original source material.
+No training data is included or distributed with this software.
+Due to its design, this model performs classification only and is not capable of
+generating or reconstructing source texts.
+---
+### Good-Faith Notice
+If you are a rights holder and believe that your content has been included in the
+training data and wish to request its exclusion from future versions, please contact
+the authors. Reasonable efforts will be made to review and address such requests.
+---
+### Limitation and Responsibility
+No representation or warranty is made regarding the licensing status of the
+underlying training data.
+Users are solely responsible for ensuring that their use of this software complies
+with applicable laws and licensing requirements.
+---
+### Disclaimer
+Unless required by applicable law or agreed to in writing, this software is
+provided "AS IS", WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+or implied, including, without limitation, any warranties of TITLE,
+NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE.
+---
+### Full Apache License Text
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+   1. Definitions.
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+   END OF TERMS AND CONDITIONS
+   APPENDIX: How to apply the Apache License to your work.
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+   Copyright [2023] [AUTHORS]
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+       http://www.apache.org/licenses/LICENSE-2.0
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md CHANGED Viewed

@@ -1,13 +1,20 @@
 ---
-license: apache-2.0
 language:
 - aak
 - aau
 - aaz
 - aba
 - abk
 - abn
 - abq
 - abt
 - abx
 - aby
@@ -28,12 +35,14 @@ language:
 - adi
 - adj
 - adl
 - ady
 - adz
 - aeb
 - aer
 - aeu
 - aey
 - afb
 - afh
 - afr
@@ -56,6 +65,7 @@ language:
 - aji
 - ajp
 - ajz
 - aka
 - akb
 - ake
@@ -71,6 +81,7 @@ language:
 - alt
 - aly
 - alz
 - ame
 - amf
 - amh
@@ -82,9 +93,11 @@ language:
 - amr
 - amu
 - amx
 - ang
 - anm
 - ann
 - anv
 - any
 - aoc
@@ -103,6 +116,8 @@ language:
 - apw
 - apy
 - apz
 - ara
 - arb
 - are
@@ -112,9 +127,11 @@ language:
 - arn
 - arp
 - arq
 - ars
 - ary
 - arz
 - asg
 - asm
 - aso
@@ -124,26 +141,34 @@ language:
 - atd
 - atg
 - ati
 - att
 - auc
 - aui
 - auy
 - ava
 - avk
 - avt
 - avu
 - awa
 - awb
 - awi
 - awx
 - aym
 - ayo
 - ayr
 - azb
 - aze
 - azg
 - azj
 - azz
 - bak
 - bal
 - bam
@@ -156,6 +181,8 @@ language:
 - bbb
 - bbc
 - bbj
 - bbr
 - bcc
 - bch
@@ -165,6 +192,8 @@ language:
 - bcw
 - bdd
 - bdh
 - bea
 - bef
 - bel
@@ -172,18 +201,23 @@ language:
 - ben
 - beq
 - ber
 - bex
 - bfd
 - bfo
 - bfz
 - bgr
 - bgs
 - bgz
 - bhg
 - bhl
 - bho
 - bhp
 - bhw
 - bib
 - big
 - bih
@@ -198,26 +232,35 @@ language:
 - bjr
 - bjv
 - bkd
 - bkq
 - bku
 - bkv
 - bla
 - blh
 - blw
 - blz
 - bmb
 - bmh
 - bmk
 - bmq
 - bmr
 - bmu
 - bnj
 - bnp
 - boa
 - bod
 - boj
 - bom
 - bon
 - bos
 - bov
 - box
@@ -227,9 +270,12 @@ language:
 - bqc
 - bqj
 - bqp
 - bre
 - bru
 - brx
 - bsc
 - bsn
 - bsp
@@ -248,6 +294,8 @@ language:
 - bul
 - bum
 - bus
 - bvr
 - bvy
 - bvz
@@ -265,6 +313,7 @@ language:
 - bzi
 - bzj
 - bzt
 - caa
 - cab
 - cac
@@ -279,7 +328,6 @@ language:
 - cat
 - cav
 - cax
-- cay
 - cbc
 - cbi
 - cbk
@@ -290,6 +338,9 @@ language:
 - cbv
 - cce
 - cco
 - ceb
 - ceg
 - cek
@@ -297,11 +348,11 @@ language:
 - cfm
 - cgc
 - cgg
 - cha
 - chd
 - che
 - chf
-- chg
 - chj
 - chk
 - chn
@@ -317,8 +368,8 @@ language:
 - cjp
 - cjs
 - cjv
-- cjy
 - ckb
 - cko
 - ckt
 - cle
@@ -328,11 +379,15 @@ language:
 - cmi
 - cmn
 - cmo
 - cnh
 - cni
 - cnl
 - cnt
 - cnw
 - coe
 - cof
 - cok
@@ -341,21 +396,24 @@ language:
 - cor
 - cos
 - cot
 - cpa
 - cpb
 - cpc
-- cpi
 - cpu
 - cpy
 - crh
 - cri
 - crk
 - crm
 - crn
 - crq
 - crs
 - crt
 - crx
 - csb
 - csk
 - cso
@@ -366,6 +424,7 @@ language:
 - cto
 - ctp
 - ctu
 - cub
 - cuc
 - cui
@@ -373,21 +432,27 @@ language:
 - cul
 - cut
 - cux
 - cwd
 - cwe
 - cwt
 - cya
 - cym
 - czt
 - daa
 - dad
 - daf
 - dah
 - dak
 - dan
 - dar
 - ddg
-- ddo
 - ded
 - des
 - deu
@@ -399,6 +464,7 @@ language:
 - dhg
 - dhm
 - dhv
 - dig
 - dik
 - din
@@ -411,55 +477,73 @@ language:
 - djk
 - djr
 - dks
 - dng
 - dnj
 - dob
 - dop
 - dow
 - drg
 - drt
 - dru
 - dsb
 - dtp
 - dts
 - dua
 - due
 - dug
 - duo
 - dur
 - dwr
 - dws
 - dww
 - dyi
 - dyo
 - dyu
 - dzo
 - ebk
 - efi
 - egl
 - eka
 - ekk
 - eko
 - ell
 - emi
 - eml
 - emp
-- emx
 - enb
 - eng
 - enl
 - enm
 - enx
 - epo
 - eri
 - ese
 - esi
 - esk
 - est
 - esu
 - eto
 - etr
 - etu
 - eus
 - eve
 - evn
@@ -467,29 +551,38 @@ language:
 - ewo
 - ext
 - eza
 - faa
 - fad
 - fai
 - fal
 - fan
 - fao
 - fas
 - fat
 - ffm
 - fij
 - fil
 - fin
 - fkv
 - fmp
 - fon
 - for
 - fra
 - frm
 - fro
 - frr
 - fry
 - fub
-- fuc
 - fud
 - fue
 - fuf
@@ -497,6 +590,8 @@ language:
 - fuq
 - fur
 - fuv
 - gaa
 - gag
 - gah
@@ -506,11 +601,11 @@ language:
 - gaz
 - gba
 - gbi
-- gbm
 - gbo
 - gbr
 - gcf
 - gcr
 - gde
 - gdg
 - gdn
@@ -526,12 +621,16 @@ language:
 - gjn
 - gkn
 - gkp
 - gla
 - gle
 - glg
 - glk
 - glv
 - gmv
 - gnb
 - gnd
 - gng
@@ -540,6 +639,7 @@ language:
 - goa
 - gof
 - gog
 - gom
 - gor
 - gos
@@ -550,6 +650,7 @@ language:
 - grt
 - gso
 - gsw
 - gub
 - guc
 - gud
@@ -564,18 +665,23 @@ language:
 - guo
 - guq
 - gur
 - guw
 - gux
 - guz
 - gvc
 - gvf
 - gvl
 - gvn
 - gwi
 - gxx
 - gya
 - gym
 - gyr
 - hae
 - hag
 - hak
@@ -587,11 +693,12 @@ language:
 - hbo
 - hbs
 - hch
-- hdn
 - heb
 - heg
 - heh
 - her
 - hif
 - hig
 - hil
@@ -606,15 +713,18 @@ language:
 - hnj
 - hnn
 - hns
 - hoc
 - hop
 - hot
 - hra
 - hrv
 - hrx
 - hsb
-- hsn
 - hto
 - hub
 - hui
 - hun
@@ -623,28 +733,36 @@ language:
 - huv
 - hvn
 - hwc
 - hye
 - hyw
 - ian
 - iba
 - ibg
 - ibo
 - icr
 - ido
 - idu
 - ifa
 - ifb
 - ife
 - ifk
 - ifu
 - ify
 - ige
 - ign
 - igs
 - iii
 - ijc
 - ike
 - ikk
 - ikw
 - ilb
 - ile
@@ -653,18 +771,23 @@ language:
 - ina
 - inb
 - ind
 - ino
 - iou
 - ipi
 - iqw
 - iri
 - irk
 - iry
 - isd
 - ish
 - isl
 - iso
 - ita
 - its
 - itv
 - ium
@@ -675,6 +798,8 @@ language:
 - izh
 - izr
 - izz
 - jac
 - jae
 - jam
@@ -689,10 +814,14 @@ language:
 - jpa
 - jpn
 - jra
 - jvn
 - kaa
 - kab
 - kac
 - kal
 - kam
 - kan
@@ -706,9 +835,13 @@ language:
 - kbd
 - kbh
 - kbm
 - kbp
 - kbq
 - kbr
 - kck
 - kdc
 - kde
@@ -716,44 +849,58 @@ language:
 - kdi
 - kdj
 - kdl
 - kea
 - kei
 - kek
 - ken
-- ket
 - kew
 - kex
 - kez
 - kff
 - kgf
 - kgk
 - kgp
 - kha
 - khk
 - khm
 - khs
 - khy
 - khz
 - kia
 - kik
 - kin
 - kir
 - kiu
 - kix
 - kjb
 - kje
 - kjh
 - kjs
 - kkc
 - kki
 - kkj
 - kkl
-- klj
 - kln
 - klt
 - klv
 - kma
 - kmb
 - kmg
 - kmh
 - kmk
@@ -762,6 +909,8 @@ language:
 - kmr
 - kms
 - kmu
 - knc
 - kne
 - knf
@@ -772,6 +921,7 @@ language:
 - knv
 - knx
 - kny
 - kog
 - koi
 - kom
@@ -779,14 +929,17 @@ language:
 - koo
 - kor
 - kos
 - kpf
 - kpg
 - kpj
 - kpr
 - kpv
 - kpw
 - kpx
 - kpz
 - kqc
 - kqe
 - kqf
@@ -802,6 +955,8 @@ language:
 - krj
 - krl
 - kru
 - ksb
 - ksc
 - ksd
@@ -817,6 +972,7 @@ language:
 - ktm
 - kto
 - ktu
 - kua
 - kub
 - kud
@@ -825,8 +981,11 @@ language:
 - kum
 - kup
 - kus
 - kvj
 - kvn
 - kwd
 - kwf
 - kwi
@@ -836,6 +995,7 @@ language:
 - kxc
 - kxm
 - kxw
 - kyc
 - kyf
 - kyg
@@ -846,22 +1006,26 @@ language:
 - kzf
 - kzj
 - kzn
-- laa
 - lac
 - lad
 - lai
 - laj
 - lam
 - lao
 - las
 - lat
 - lav
 - lbb
 - lbe
 - lbj
 - lbk
 - lch
 - lcm
 - ldi
 - ldn
 - lea
@@ -875,10 +1039,14 @@ language:
 - lex
 - lez
 - lfn
 - lgm
 - lhi
 - lhm
 - lhu
 - lia
 - lid
 - lif
@@ -887,37 +1055,49 @@ language:
 - lin
 - lip
 - lir
 - lit
 - liv
 - ljp
-- lkt
 - llb
 - lld
 - lln
 - lmk
 - lmo
 - lmp
 - lob
 - loe
 - log
 - lol
 - lom
 - loq
-- lou
 - loz
 - lsi
 - lsm
 - ltg
 - ltz
 - lua
 - lub
 - lue
 - lug
 - lun
 - luo
 - lus
 - lut
 - lvs
 - lwo
 - lww
 - lzh
@@ -971,28 +1151,33 @@ language:
 - men
 - meq
 - mer
 - meu
 - mev
 - mfa
 - mfe
 - mfh
 - mfi
 - mfk
 - mfq
 - mfy
 - mfz
 - mgc
 - mgh
 - mgm
 - mgo
 - mgr
 - mgv
 - mhi
 - mhl
 - mhr
 - mhw
 - mhx
 - mhy
 - mib
 - mic
 - mie
@@ -1001,8 +1186,10 @@ language:
 - mih
 - mik
 - mil
 - min
 - mio
 - miq
 - mir
 - mit
@@ -1010,11 +1197,13 @@ language:
 - miz
 - mjc
 - mjw
 - mkd
 - mkl
 - mkn
 - mks
 - mkz
 - mlg
 - mlh
 - mlp
@@ -1023,13 +1212,13 @@ language:
 - mmn
 - mmo
 - mmx
 - mna
 - mnb
-- mnc
 - mnf
 - mni
 - mnk
-- mnr
 - mnw
 - mnx
 - mny
@@ -1052,12 +1241,14 @@ language:
 - mqb
 - mqj
 - mqy
 - mrg
 - mri
 - mrj
 - mrq
 - mrv
 - mrw
 - msa
 - msb
 - msc
@@ -1065,6 +1256,7 @@ language:
 - msk
 - msm
 - msy
 - mta
 - mtg
 - mti
@@ -1074,6 +1266,8 @@ language:
 - mua
 - mug
 - muh
 - mur
 - mus
 - mux
@@ -1081,7 +1275,6 @@ language:
 - mva
 - mvn
 - mvp
-- mvv
 - mwc
 - mwf
 - mwl
@@ -1096,6 +1289,7 @@ language:
 - mxq
 - mxt
 - mxv
 - mya
 - myb
 - myk
@@ -1117,13 +1311,12 @@ language:
 - nah
 - nak
 - nan
-- nan
 - nap
 - naq
 - nas
-- nau
 - nav
 - naw
 - nba
 - nbc
 - nbe
@@ -1134,9 +1327,11 @@ language:
 - nch
 - ncj
 - ncl
 - nct
 - ncu
 - ncx
 - ndc
 - nde
 - ndh
@@ -1145,17 +1340,19 @@ language:
 - ndo
 - ndp
 - nds
 - ndz
 - neb
 - nep
 - new
 - nfa
 - nfr
 - ngb
 - ngc
 - ngl
 - ngp
-- ngt
 - ngu
 - nhd
 - nhe
@@ -1174,9 +1371,9 @@ language:
 - nij
 - nim
 - nin
 - niq
 - niu
-- niv
 - niy
 - njb
 - njm
@@ -1184,18 +1381,21 @@ language:
 - njo
 - njz
 - nka
 - nki
 - nko
 - nla
 - nlc
 - nld
-- nlv
 - nma
 - nmf
 - nmh
 - nmo
 - nmw
 - nmz
 - nnb
 - nng
 - nnh
@@ -1204,8 +1404,10 @@ language:
 - nnp
 - nnq
 - nnw
 - noa
 - nob
 - nog
 - non
 - nop
@@ -1218,9 +1420,12 @@ language:
 - npl
 - npo
 - npy
 - nre
 - nrf
 - nri
 - nsa
 - nse
 - nsm
@@ -1231,13 +1436,18 @@ language:
 - nsu
 - ntp
 - ntr
 - nus
 - nuy
 - nvm
 - nwb
 - nwi
 - nwx
 - nxd
 - nya
 - nyf
 - nyk
@@ -1245,40 +1455,49 @@ language:
 - nyo
 - nyu
 - nyy
 - nzb
 - nzi
 - nzm
-- oar
 - obo
 - oci
-- ofs
 - ogo
 - ojb
 - oji
 - ojs
 - oke
 - okv
 - old
 - omw
 - ong
 - ons
 - ood
 - opm
 - ori
 - orm
 - orv
 - ory
-- osp
 - oss
 - ota
 - ote
 - otm
 - otn
 - otq
 - ots
 - otw
 - oym
 - ozm
 - pab
 - pad
 - pag
@@ -1300,35 +1519,40 @@ language:
 - pdt
 - pem
 - pes
 - pfe
 - pfl
 - phm
-- phn
 - pib
 - pid
 - pio
 - pir
 - pis
 - pjt
 - pkb
 - plg
-- pli
 - pls
 - plt
 - plu
 - plw
 - pma
 - pmf
 - pms
 - pmx
 - pnb
 - pne
 - poe
 - poh
 - poi
 - pol
 - pon
 - por
 - pot
 - pov
 - poy
@@ -1340,14 +1564,22 @@ language:
 - prg
 - pri
 - prk
 - prs
 - pse
 - ptp
 - ptu
 - pua
 - pus
 - pwg
 - pww
 - qub
 - quc
 - que
@@ -1376,19 +1608,25 @@ language:
 - qxl
 - qxn
 - qxo
-- qxq
 - qxr
 - qya
 - rad
 - rai
 - rap
 - rar
 - rcf
 - rhg
 - ria
 - rif
 - rim
 - rkb
 - rmc
 - rme
 - rml
@@ -1396,51 +1634,68 @@ language:
 - rmo
 - rmq
 - rmy
 - rnd
 - rng
 - rnl
 - roh
 - rom
 - ron
 - roo
 - rop
 - rro
 - rtm
 - rub
 - rue
 - ruf
 - run
 - rup
 - rus
 - rwo
-- ryu
 - sab
 - sag
 - sah
 - san
 - sas
 - sat
 - sba
 - sbd
 - sbe
 - sbl
 - sbs
 - sby
 - sck
 - scn
 - sco
 - sda
 - sdh
 - sdo
 - seh
 - ses
 - sey
 - sfw
 - sgb
 - sgh
 - sgs
 - sgw
 - sgz
 - shi
 - shk
 - shn
@@ -1449,6 +1704,7 @@ language:
 - shs
 - shu
 - shy
 - sid
 - sig
 - sil
@@ -1456,18 +1712,27 @@ language:
 - sin
 - sja
 - sjn
 - skg
 - skr
 - sld
 - slk
 - sll
 - slv
 - sma
 - sme
 - smk
 - sml
 - smo
 - smt
 - sna
 - snc
 - snd
@@ -1476,6 +1741,7 @@ language:
 - snp
 - snw
 - sny
 - soe
 - som
 - sop
@@ -1488,7 +1754,9 @@ language:
 - spp
 - sps
 - spy
 - sqi
 - srd
 - sri
 - srm
@@ -1496,13 +1764,16 @@ language:
 - srp
 - srq
 - srr
 - ssd
 - ssg
 - ssw
 - ssx
 - stn
 - stp
 - stq
 - sua
 - suc
 - sue
@@ -1512,6 +1783,8 @@ language:
 - sus
 - sux
 - suz
 - swa
 - swb
 - swc
@@ -1527,6 +1800,8 @@ language:
 - syl
 - szb
 - szl
 - tab
 - tac
 - tah
@@ -1538,11 +1813,13 @@ language:
 - tat
 - tav
 - taw
 - tbc
 - tbg
 - tbk
 - tbl
 - tbo
 - tby
 - tbz
 - tca
@@ -1553,6 +1830,7 @@ language:
 - tcz
 - tdt
 - tdx
 - ted
 - tee
 - tel
@@ -1562,13 +1840,17 @@ language:
 - tet
 - tew
 - tfr
 - tgk
 - tgl
 - tgo
 - tgp
 - tha
 - thk
 - thv
 - tif
 - tig
 - tih
@@ -1577,24 +1859,29 @@ language:
 - tir
 - tiv
 - tiy
 - tke
 - tkl
 - tkr
 - tku
 - tlb
 - tlf
 - tlh
 - tlj
 - tll
 - tly
 - tmd
 - tmr
-- tmw
 - tna
 - tnc
 - tnk
 - tnn
 - tnp
 - tob
 - toc
 - tod
@@ -1610,22 +1897,29 @@ language:
 - tpa
 - tpi
 - tpm
 - tpp
 - tpt
 - tpw
 - tpz
 - tqb
 - trc
 - trn
 - tro
 - trp
 - trq
 - tsc
 - tsg
 - tsn
 - tso
 - tsw
 - tsz
 - ttc
 - tte
 - ttj
@@ -1636,17 +1930,21 @@ language:
 - tuf
 - tui
 - tuk
 - tum
 - tuo
 - tur
 - tuv
 - tvk
 - tvl
 - twi
 - twu
 - twx
 - txq
 - txu
 - tyv
 - tzh
 - tzj
@@ -1657,10 +1955,14 @@ language:
 - ubu
 - udm
 - udu
 - uig
 - ukr
 - umb
 - upv
 - ura
 - urb
 - urd
@@ -1668,19 +1970,26 @@ language:
 - uri
 - urk
 - urt
 - usa
 - usp
 - uvh
 - uvl
 - uzb
 - uzn
 - vag
 - vap
 - var
 - vec
 - ven
 - vep
 - vgt
 - vid
 - vie
 - viv
@@ -1688,10 +1997,13 @@ language:
 - vmk
 - vmw
 - vmy
 - vol
 - vro
 - vun
 - vut
 - waj
 - wal
 - wap
@@ -1714,15 +2026,18 @@ language:
 - wln
 - wls
 - wlv
 - wmt
 - wmw
 - wnc
 - wnu
 - wob
 - wol
 - wos
 - wrk
 - wrs
 - wsk
 - wuu
 - wuv
@@ -1732,15 +2047,16 @@ language:
 - xbi
 - xbr
 - xed
 - xho
 - xla
 - xmf
 - xmv
 - xnn
 - xog
 - xon
 - xpe
-- xqa
 - xrb
 - xsb
 - xsi
@@ -1750,6 +2066,7 @@ language:
 - xtd
 - xtm
 - xtn
 - xuo
 - yaa
 - yad
@@ -1759,17 +2076,21 @@ language:
 - yao
 - yap
 - yaq
 - ybb
 - yby
 - ycn
 - ydd
 - yid
 - yim
 - yka
-- ykg
 - yle
 - yli
 - yml
 - yom
 - yon
 - yor
@@ -1803,11 +2124,13 @@ language:
 - zdj
 - zea
 - zgh
 - zho
 - zia
 - ziw
 - zlm
 - zne
 - zom
 - zos
 - zpa
@@ -1829,10 +2152,13 @@ language:
 - zsr
 - ztq
 - zty
 - zul
 - zyb
 - zyp
 - zza
 tags:
 - text-classification
 - language-identification
@@ -1840,6 +2166,7 @@ library_name: fasttext
 datasets:
 - cis-lmu/GlotSparse
 - cis-lmu/GlotStoryBook
 metrics:
 - f1
 ---
@@ -1850,10 +2177,10 @@ metrics:
 ## Description
-**GlotLID** is a Fasttext language identification (LID) model that supports more than **1600 languages**.
-**Latest:** GlotLID is now updated to **V3**. V3 supports **2114 labels** (three-letter ISO codes with script). For more details on the supported languages and performance, as well as significant changes from previous versions, please refer to [https://github.com/cisnlp/GlotLID/blob/main/languages-v3.md](https://github.com/cisnlp/GlotLID/blob/main/languages-v3.md).
 - **Demo:** [huggingface](https://huggingface.co/spaces/cis-lmu/glotlid-space)
@@ -1893,7 +2220,7 @@ If you are not a fan of huggingface_hub, then download the model directyly:
 ## License
-The model is distributed under the Apache License, Version 2.0.
 ## Version

 ---
+license: other
+license_name: apache-2.0-plus-notices
+license_link: LICENSE
 language:
+- aah
+- aai
 - aak
 - aau
 - aaz
+- ab
 - aba
+- abi
 - abk
 - abn
 - abq
+- abs
 - abt
 - abx
 - aby
 - adi
 - adj
 - adl
+- adx
 - ady
 - adz
 - aeb
 - aer
 - aeu
 - aey
+- af
 - afb
 - afh
 - afr
 - aji
 - ajp
 - ajz
+- ak
 - aka
 - akb
 - ake
 - alt
 - aly
 - alz
+- am
 - ame
 - amf
 - amh
 - amr
 - amu
 - amx
+- an
 - ang
 - anm
 - ann
+- anp
 - anv
 - any
 - aoc
 - apw
 - apy
 - apz
+- aqz
+- ar
 - ara
 - arb
 - are
 - arn
 - arp
 - arq
+- arr
 - ars
 - ary
 - arz
+- as
 - asg
 - asm
 - aso
 - atd
 - atg
 - ati
+- atj
+- atq
 - att
 - auc
 - aui
 - auy
+- av
 - ava
 - avk
+- avn
 - avt
 - avu
 - awa
 - awb
 - awi
 - awx
+- ay
 - aym
 - ayo
+- ayp
 - ayr
+- az
 - azb
 - aze
 - azg
 - azj
 - azz
+- ba
 - bak
 - bal
 - bam
 - bbb
 - bbc
 - bbj
+- bbk
+- bbo
 - bbr
 - bcc
 - bch
 - bcw
 - bdd
 - bdh
+- bdq
+- be
 - bea
 - bef
 - bel
 - ben
 - beq
 - ber
+- bew
 - bex
 - bfd
 - bfo
 - bfz
+- bg
 - bgr
 - bgs
+- bgt
 - bgz
 - bhg
 - bhl
 - bho
 - bhp
 - bhw
+- bhz
+- bi
 - bib
 - big
 - bih
 - bjr
 - bjv
 - bkd
+- bkl
 - bkq
 - bku
 - bkv
 - bla
 - blh
+- blk
+- blt
 - blw
 - blz
+- bm
 - bmb
 - bmh
 - bmk
 - bmq
 - bmr
 - bmu
+- bmv
+- bn
 - bnj
+- bno
 - bnp
+- bo
 - boa
 - bod
 - boj
 - bom
 - bon
+- bor
 - bos
 - bov
 - box
 - bqc
 - bqj
 - bqp
+- br
 - bre
+- brh
 - bru
 - brx
+- bs
 - bsc
 - bsn
 - bsp
 - bul
 - bum
 - bus
+- bvc
+- bvd
 - bvr
 - bvy
 - bvz
 - bzi
 - bzj
 - bzt
+- ca
 - caa
 - cab
 - cac
 - cat
 - cav
 - cax
 - cbc
 - cbi
 - cbk
 - cbv
 - cce
 - cco
+- ccp
+- cdf
+- ce
 - ceb
 - ceg
 - cek
 - cfm
 - cgc
 - cgg
+- ch
 - cha
 - chd
 - che
 - chf
 - chj
 - chk
 - chn
 - cjp
 - cjs
 - cjv
 - ckb
+- ckm
 - cko
 - ckt
 - cle
 - cmi
 - cmn
 - cmo
+- cmr
 - cnh
 - cni
+- cnk
 - cnl
+- cnr
 - cnt
 - cnw
+- co
 - coe
 - cof
 - cok
 - cor
 - cos
 - cot
+- cou
 - cpa
 - cpb
 - cpc
 - cpu
 - cpy
 - crh
 - cri
+- crj
 - crk
+- crl
 - crm
 - crn
 - crq
 - crs
 - crt
 - crx
+- cs
 - csb
 - csk
 - cso
 - cto
 - ctp
 - ctu
+- cu
 - cub
 - cuc
 - cui
 - cul
 - cut
 - cux
+- cv
 - cwd
 - cwe
 - cwt
+- cy
 - cya
 - cym
 - czt
+- da
 - daa
 - dad
 - daf
+- dag
 - dah
 - dak
 - dan
 - dar
+- dbq
 - ddg
+- ddn
+- de
 - ded
 - des
 - deu
 - dhg
 - dhm
 - dhv
+- did
 - dig
 - dik
 - din
 - djk
 - djr
 - dks
+- dln
 - dng
 - dnj
+- dnw
 - dob
+- doi
 - dop
+- dos
 - dow
 - drg
 - drt
 - dru
 - dsb
+- dsh
+- dtb
 - dtp
 - dts
+- dty
 - dua
 - due
 - dug
 - duo
 - dur
+- dv
 - dwr
 - dws
 - dww
 - dyi
 - dyo
 - dyu
+- dz
 - dzo
 - ebk
+- ee
 - efi
 - egl
 - eka
 - ekk
 - eko
+- el
 - ell
+- eme
 - emi
 - eml
 - emp
+- en
 - enb
 - eng
 - enl
 - enm
+- enq
 - enx
+- eo
 - epo
 - eri
+- es
 - ese
 - esi
 - esk
+- ess
 - est
 - esu
+- et
 - eto
 - etr
 - etu
+- eu
 - eus
 - eve
 - evn
 - ewo
 - ext
 - eza
+- fa
 - faa
 - fad
 - fai
 - fal
 - fan
 - fao
+- far
 - fas
 - fat
 - ffm
+- fi
 - fij
 - fil
 - fin
+- fit
+- fj
 - fkv
 - fmp
+- fmu
+- fo
 - fon
 - for
+- fr
 - fra
+- frd
 - frm
 - fro
+- frp
 - frr
 - fry
 - fub
 - fud
 - fue
 - fuf
 - fuq
 - fur
 - fuv
+- fy
+- ga
 - gaa
 - gag
 - gah
 - gaz
 - gba
 - gbi
 - gbo
 - gbr
 - gcf
 - gcr
+- gd
 - gde
 - gdg
 - gdn
 - gjn
 - gkn
 - gkp
+- gl
 - gla
 - gle
 - glg
 - glk
 - glv
+- gmh
 - gmv
+- gn
+- gna
 - gnb
 - gnd
 - gng
 - goa
 - gof
 - gog
+- goh
 - gom
 - gor
 - gos
 - grt
 - gso
 - gsw
+- gu
 - gub
 - guc
 - gud
 - guo
 - guq
 - gur
+- guu
 - guw
 - gux
 - guz
+- gv
 - gvc
 - gvf
 - gvl
 - gvn
 - gwi
+- gwr
 - gxx
 - gya
 - gym
 - gyr
+- ha
+- hac
 - hae
 - hag
 - hak
 - hbo
 - hbs
 - hch
+- he
 - heb
 - heg
 - heh
 - her
+- hi
 - hif
 - hig
 - hil
 - hnj
 - hnn
 - hns
+- ho
 - hoc
 - hop
 - hot
+- hr
 - hra
 - hrv
 - hrx
 - hsb
+- ht
 - hto
+- hu
 - hub
 - hui
 - hun
 - huv
 - hvn
 - hwc
+- hy
 - hye
 - hyw
+- hz
+- ia
 - ian
 - iba
 - ibg
 - ibo
 - icr
+- id
 - ido
 - idu
+- ie
 - ifa
 - ifb
 - ife
 - ifk
 - ifu
 - ify
+- ig
 - ige
 - ign
 - igs
+- ii
 - iii
 - ijc
 - ike
 - ikk
+- ikt
 - ikw
 - ilb
 - ile
 - ina
 - inb
 - ind
+- inh
 - ino
+- io
 - iou
 - ipi
 - iqw
 - iri
 - irk
 - iry
+- is
 - isd
 - ish
 - isl
 - iso
+- it
 - ita
+- itl
 - its
 - itv
 - ium
 - izh
 - izr
 - izz
+- ja
+- jaa
 - jac
 - jae
 - jam
 - jpa
 - jpn
 - jra
+- jun
+- jv
 - jvn
+- ka
 - kaa
 - kab
 - kac
+- kak
 - kal
 - kam
 - kan
 - kbd
 - kbh
 - kbm
+- kbo
 - kbp
 - kbq
 - kbr
+- kby
+- kca
+- kcg
 - kck
 - kdc
 - kde
 - kdi
 - kdj
 - kdl
+- kdp
+- kdr
 - kea
 - kei
 - kek
 - ken
+- keo
+- ker
 - kew
 - kex
 - kez
 - kff
+- kg
 - kgf
 - kgk
 - kgp
+- kgr
 - kha
+- khg
 - khk
 - khm
+- khq
 - khs
 - khy
 - khz
+- ki
 - kia
+- kij
 - kik
 - kin
 - kir
 - kiu
 - kix
+- kj
 - kjb
 - kje
 - kjh
 - kjs
+- kk
 - kkc
 - kki
 - kkj
 - kkl
+- kl
+- kle
 - kln
 - klt
 - klv
+- km
 - kma
 - kmb
+- kmd
 - kmg
 - kmh
 - kmk
 - kmr
 - kms
 - kmu
+- kmy
+- kn
 - knc
 - kne
 - knf
 - knv
 - knx
 - kny
+- ko
 - kog
 - koi
 - kom
 - koo
 - kor
 - kos
+- kpe
 - kpf
 - kpg
 - kpj
+- kpq
 - kpr
 - kpv
 - kpw
 - kpx
 - kpz
+- kqa
 - kqc
 - kqe
 - kqf
 - krj
 - krl
 - kru
+- krx
+- ks
 - ksb
 - ksc
 - ksd
 - ktm
 - kto
 - ktu
+- ktz
 - kua
 - kub
 - kud
 - kum
 - kup
 - kus
+- kv
+- kvg
 - kvj
 - kvn
+- kw
 - kwd
 - kwf
 - kwi
 - kxc
 - kxm
 - kxw
+- ky
 - kyc
 - kyf
 - kyg
 - kzf
 - kzj
 - kzn
+- la
 - lac
 - lad
 - lai
 - laj
 - lam
 - lao
+- lap
 - las
 - lat
 - lav
+- law
+- lb
 - lbb
 - lbe
 - lbj
 - lbk
 - lch
 - lcm
+- lcp
 - ldi
 - ldn
 - lea
 - lex
 - lez
 - lfn
+- lg
+- lgg
+- lgl
 - lgm
 - lhi
 - lhm
 - lhu
+- li
 - lia
 - lid
 - lif
 - lin
 - lip
 - lir
+- lis
 - lit
 - liv
 - ljp
+- lki
 - llb
 - lld
+- llg
 - lln
 - lmk
 - lmo
 - lmp
+- ln
+- lnd
+- lo
 - lob
 - loe
 - log
+- lok
 - lol
 - lom
 - loq
 - loz
+- lrc
 - lsi
 - lsm
+- lt
 - ltg
 - ltz
+- lu
 - lua
 - lub
+- luc
+- lud
 - lue
 - lug
 - lun
 - luo
 - lus
 - lut
+- lv
 - lvs
+- lwg
 - lwo
 - lww
 - lzh
 - men
 - meq
 - mer
+- met
 - meu
 - mev
 - mfa
 - mfe
+- mfg
 - mfh
 - mfi
 - mfk
 - mfq
 - mfy
 - mfz
+- mg
 - mgc
 - mgh
 - mgm
 - mgo
 - mgr
 - mgv
+- mh
 - mhi
 - mhl
 - mhr
 - mhw
 - mhx
 - mhy
+- mi
 - mib
 - mic
 - mie
 - mih
 - mik
 - mil
+- mim
 - min
 - mio
+- mip
 - miq
 - mir
 - mit
 - miz
 - mjc
 - mjw
+- mk
 - mkd
 - mkl
 - mkn
 - mks
 - mkz
+- ml
 - mlg
 - mlh
 - mlp
 - mmn
 - mmo
 - mmx
+- mn
 - mna
 - mnb
 - mnf
 - mni
 - mnk
+- mns
 - mnw
 - mnx
 - mny
 - mqb
 - mqj
 - mqy
+- mr
 - mrg
 - mri
 - mrj
 - mrq
 - mrv
 - mrw
+- ms
 - msa
 - msb
 - msc
 - msk
 - msm
 - msy
+- mt
 - mta
 - mtg
 - mti
 - mua
 - mug
 - muh
+- mui
+- mup
 - mur
 - mus
 - mux
 - mva
 - mvn
 - mvp
 - mwc
 - mwf
 - mwl
 - mxq
 - mxt
 - mxv
+- my
 - mya
 - myb
 - myk
 - nah
 - nak
 - nan
 - nap
 - naq
 - nas
 - nav
 - naw
+- nb
 - nba
 - nbc
 - nbe
 - nch
 - ncj
 - ncl
+- ncq
 - nct
 - ncu
 - ncx
+- nd
 - ndc
 - nde
 - ndh
 - ndo
 - ndp
 - nds
+- ndy
 - ndz
+- ne
 - neb
 - nep
 - new
 - nfa
 - nfr
+- ng
 - ngb
 - ngc
 - ngl
 - ngp
 - ngu
 - nhd
 - nhe
 - nij
 - nim
 - nin
+- nio
 - niq
 - niu
 - niy
 - njb
 - njm
 - njo
 - njz
 - nka
+- nkf
 - nki
 - nko
+- nl
 - nla
 - nlc
 - nld
+- nlg
 - nma
 - nmf
 - nmh
 - nmo
 - nmw
 - nmz
+- nn
 - nnb
 - nng
 - nnh
 - nnp
 - nnq
 - nnw
+- no
 - noa
 - nob
+- nod
 - nog
 - non
 - nop
 - npl
 - npo
 - npy
+- nqo
+- nr
 - nre
 - nrf
 - nri
+- nrm
 - nsa
 - nse
 - nsm
 - nsu
 - ntp
 - ntr
+- ntu
+- nuj
 - nus
 - nuy
+- nuz
+- nv
 - nvm
 - nwb
 - nwi
 - nwx
 - nxd
+- ny
 - nya
 - nyf
 - nyk
 - nyo
 - nyu
 - nyy
+- nza
 - nzb
 - nzi
 - nzm
 - obo
+- oc
 - oci
 - ogo
+- oj
 - ojb
 - oji
 - ojs
 - oke
+- oku
 - okv
 - old
+- olo
+- om
+- omb
 - omw
 - ong
 - ons
 - ood
 - opm
+- or
 - ori
 - orm
 - orv
 - ory
+- os
 - oss
 - ota
+- otd
 - ote
 - otm
 - otn
+- oto
 - otq
 - ots
 - otw
 - oym
 - ozm
+- pa
 - pab
 - pad
 - pag
 - pdt
 - pem
 - pes
+- pez
 - pfe
 - pfl
 - phm
 - pib
 - pid
+- pih
 - pio
 - pir
 - pis
 - pjt
 - pkb
+- pl
 - plg
 - pls
 - plt
 - plu
 - plw
 - pma
 - pmf
+- pmq
 - pms
 - pmx
 - pnb
 - pne
+- pnt
+- pny
 - poe
 - poh
 - poi
 - pol
 - pon
 - por
+- pos
 - pot
 - pov
 - poy
 - prg
 - pri
 - prk
+- prq
 - prs
+- ps
 - pse
+- pss
+- pt
 - ptp
 - ptu
 - pua
+- pui
 - pus
 - pwg
+- pwn
 - pww
+- pxm
+- qu
 - qub
 - quc
 - que
 - qxl
 - qxn
 - qxo
 - qxr
 - qya
 - rad
 - rai
 - rap
 - rar
+- rav
+- raw
 - rcf
+- rej
+- rel
+- rgu
 - rhg
 - ria
 - rif
 - rim
+- rjs
 - rkb
+- rm
 - rmc
 - rme
 - rml
 - rmo
 - rmq
 - rmy
+- rn
 - rnd
 - rng
 - rnl
+- ro
 - roh
 - rom
 - ron
 - roo
 - rop
+- row
 - rro
 - rtm
+- ru
 - rub
 - rue
 - ruf
+- rug
 - run
 - rup
 - rus
+- rw
 - rwo
+- sa
 - sab
 - sag
 - sah
+- saj
 - san
 - sas
 - sat
+- say
 - sba
 - sbd
 - sbe
 - sbl
 - sbs
 - sby
+- sc
 - sck
 - scn
 - sco
+- sd
 - sda
+- sdc
 - sdh
 - sdo
+- sdq
+- se
 - seh
+- sel
 - ses
 - sey
 - sfw
+- sg
 - sgb
+- sgc
 - sgh
 - sgs
 - sgw
 - sgz
+- sh
 - shi
 - shk
 - shn
 - shs
 - shu
 - shy
+- si
 - sid
 - sig
 - sil
 - sin
 - sja
 - sjn
+- sjo
+- sju
+- sk
 - skg
 - skr
+- sl
 - sld
 - slk
 - sll
 - slv
+- sm
 - sma
 - sme
+- smj
 - smk
 - sml
+- smn
 - smo
+- sms
 - smt
+- sn
 - sna
 - snc
 - snd
 - snp
 - snw
 - sny
+- so
 - soe
 - som
 - sop
 - spp
 - sps
 - spy
+- sq
 - sqi
+- sr
 - srd
 - sri
 - srm
 - srp
 - srq
 - srr
+- ss
 - ssd
 - ssg
 - ssw
 - ssx
+- st
 - stn
 - stp
 - stq
+- su
 - sua
 - suc
 - sue
 - sus
 - sux
 - suz
+- sv
+- sw
 - swa
 - swb
 - swc
 - syl
 - szb
 - szl
+- szy
+- ta
 - tab
 - tac
 - tah
 - tat
 - tav
 - taw
+- tay
 - tbc
 - tbg
 - tbk
 - tbl
 - tbo
+- tbw
 - tby
 - tbz
 - tca
 - tcz
 - tdt
 - tdx
+- te
 - ted
 - tee
 - tel
 - tet
 - tew
 - tfr
+- tg
 - tgk
 - tgl
 - tgo
 - tgp
+- th
 - tha
 - thk
+- thl
 - thv
+- ti
 - tif
 - tig
 - tih
 - tir
 - tiv
 - tiy
+- tk
 - tke
 - tkl
 - tkr
 - tku
+- tl
 - tlb
 - tlf
 - tlh
 - tlj
 - tll
 - tly
+- tmc
 - tmd
 - tmr
+- tn
 - tna
 - tnc
 - tnk
 - tnn
 - tnp
+- tnr
+- to
 - tob
 - toc
 - tod
 - tpa
 - tpi
 - tpm
+- tpn
 - tpp
 - tpt
 - tpw
 - tpz
 - tqb
+- tqo
+- tr
 - trc
 - trn
 - tro
 - trp
 - trq
+- trs
+- trv
+- ts
 - tsc
 - tsg
 - tsn
 - tso
 - tsw
 - tsz
+- tt
 - ttc
 - tte
 - ttj
 - tuf
 - tui
 - tuk
+- tul
 - tum
 - tuo
 - tur
 - tuv
 - tvk
 - tvl
+- tw
+- twb
 - twi
 - twu
 - twx
 - txq
 - txu
+- ty
 - tyv
 - tzh
 - tzj
 - ubu
 - udm
 - udu
+- ug
 - uig
+- uk
 - ukr
 - umb
+- und
 - upv
+- ur
 - ura
 - urb
 - urd
 - uri
 - urk
 - urt
+- urw
+- ury
 - usa
 - usp
+- uth
 - uvh
 - uvl
+- uz
 - uzb
 - uzn
+- uzs
 - vag
 - vap
 - var
+- ve
 - vec
 - ven
 - vep
 - vgt
+- vi
 - vid
 - vie
 - viv
 - vmk
 - vmw
 - vmy
+- vo
 - vol
+- vot
 - vro
 - vun
 - vut
+- wa
 - waj
 - wal
 - wap
 - wln
 - wls
 - wlv
+- wlx
 - wmt
 - wmw
 - wnc
 - wnu
+- wo
 - wob
 - wol
 - wos
 - wrk
 - wrs
+- wsg
 - wsk
 - wuu
 - wuv
 - xbi
 - xbr
 - xed
+- xh
 - xho
 - xla
 - xmf
+- xmm
 - xmv
 - xnn
 - xog
 - xon
 - xpe
 - xrb
 - xsb
 - xsi
 - xtd
 - xtm
 - xtn
+- xum
 - xuo
 - yaa
 - yad
 - yao
 - yap
 - yaq
+- yas
+- yat
+- yaz
 - ybb
 - yby
 - ycn
 - ydd
+- yi
 - yid
 - yim
 - yka
 - yle
 - yli
 - yml
+- yo
 - yom
 - yon
 - yor
 - zdj
 - zea
 - zgh
+- zh
 - zho
 - zia
 - ziw
 - zlm
 - zne
+- zoc
 - zom
 - zos
 - zpa
 - zsr
 - ztq
 - zty
+- zu
 - zul
+- zxx
 - zyb
 - zyp
 - zza
 tags:
 - text-classification
 - language-identification
 datasets:
 - cis-lmu/GlotSparse
 - cis-lmu/GlotStoryBook
+- cis-lmu/glotlid-corpus
 metrics:
 - f1
 ---
 ## Description
+**GlotLID** is a Fasttext language identification (LID) model that supports more than **2000 labels**.
+**Latest:** GlotLID is now updated to **V3**. V3 supports **2102 labels** (three-letter ISO codes with script). For more details on the supported languages and performance, as well as significant changes from previous versions, please refer to [https://github.com/cisnlp/GlotLID/blob/main/languages-v3.md](https://github.com/cisnlp/GlotLID/blob/main/languages-v3.md).
 - **Demo:** [huggingface](https://huggingface.co/spaces/cis-lmu/glotlid-space)
 ## License
+The model is distributed under the Apache License, Version 2.0 plus notices (see LICENSE file for full terms).
 ## Version