rust/language.rs: add full BCP 47 compliant pipeline

- Removes custom, incomplete language mapping tables
- Adds validation for the BCP 47-defined format
- Compares all components separately instead of treating them as the
  same entity
- Adds canonicalization support for ISO 639-{1,2t,2b,3}

It is not overly strict in validation to preserve compatibility with
"custom" tags. If the language can be parsed and normalized, this will
be used; otherwise, it falls back to direct string comparison.

For BCP 47 components, all mismatches are penalized equally, with 1000
points deducted per mismatch. This value can be adjusted if needed.

This commit should greatly improve support for matching language tags
consisting of more than just the primary language and region.

Author: kasper93

.pre-commit-config.yaml

@@ -19,4 +19,3 @@ repos:
   hooks:
     - id: codespell
       args: ["--ignore-words-list", "datas,DNE,enew,HDA,numer,ontop,optionA,Paeth,pathc,inout"]
-      exclude: ^misc/language.c$

meson.build

@@ -11,6 +11,8 @@ project('mpv',
             'cpp_std=c++20',
             'cpp_eh=default',
             'warning_level=2',
+            'rust_std=2021',
+            'build.rust_std=2021',
         ]
 )
 
@@ -140,7 +142,6 @@ sources = files(
     'misc/dispatch.c',
     'misc/io_utils.c',
     'misc/json.c',
-    'misc/language.c',
     'misc/natural_sort.c',
     'misc/node.c',
     'misc/path_utils.c',
@@ -1764,6 +1765,8 @@ major = client_h_define.split('|')[0].split('<<')[0].strip('() ')
 minor = client_h_define.split('|')[1].strip('() ')
 client_api_version = major + '.' + minor + '.0'
 
+subdir('rust')
+
 libmpv = library('mpv', sources, dependencies: dependencies, gnu_symbol_visibility: 'hidden',
                  include_directories: incdir_public,
                  link_args: cc.get_supported_link_arguments(['-Wl,-Bsymbolic']),