How to make Django support IETF language tag (xx-YY format)?

Question

We have a Django site that supports many languages. Trying to add opensearch plug-ins support for multi-language.

OpenSearch.org spec uses IETF language tag (xx-YY format). Default Django setup uses exact matching.

• The current state, the site support only xx form anything else return E404. Example:

  http://website.domain/en/...
  

Depending on user configuration the browser insert language parameter as xx or xx-YY. It needs to work for both

• If xx-YY is not available, site should provide xx (mother country language) results.
• if xx is not available, site should provide en (English) results.

Example of URL to be supported:

http://website.domain/fr-YY/...

  fall-back to: http://website.domain/fr/...

http://website.domain/xx/...

  fall-back to: http://website.domain/en/...

Example URL from Mozilla site: https://support.mozilla.org/en-US/questions/949545

How to make Django support IETF language tag (xx-YY format)? I'm looking even for hints to implement this without modifying the django upstream code.

Update:

Well, official documentation says clearly it should fall-back (ex: en-us to en) but my case raises 404 error.

Source: https://django.readthedocs.io/en/1.5.x/topics/i18n/translation.html

If a base language is available but the sublanguage specified is not, Django uses the base language. For example, if a user specifies de-at (Austrian German) but Django only has de available, Django uses de.

...

LANGUAGES = ( ('de', _('German')), ('en', _('English')), )

This example restricts languages that are available for automatic selection to German and English (and any sublanguage, like de-ch or en-us).

Here are related code portions:

settings.py

# Language code for this installation. All choices can be found here:
# http://www.i18nguy.com/unicode/language-identifiers.html
LANGUAGE_CODE = 'en'

LANGUAGES = (
  ( 'ar', "Arabic" ),
  ( 'en', "English" ),
  ( 'fr', "French" ),
  ( 'id', "Indonesian" ),
  ( 'ja', "Japanese"),
  ( 'ku', "Kurdish" ),
  #( 'ur', "Urdu" ),
  ( 'ms', "Malay" ),
  ( 'ml', "Malayalam" ),
  #( 'tr', "Turkish" ),
  ( 'es', "Spanish" ),
  ( 'pt', "Portuguese"),
  #( 'sv', "swedish" )
)

# These are languages not supported by Django core. We have to provide
# their info here so we can use them in our templates. This is mainly
# used in `wui.templatetags.languages`.
EXTRA_LANGUAGES = {
  'ku': {
    'code': 'ku',
    'name': 'Kurdish',
    'bidi': True,
    'name_local': 'Kurdish'
  },
  'ms': {
    'code': 'ms',
    'name': 'Malay',
    'bidi': False,
    'name_local': 'Malay'
  },
}


SITE_ID = 1

# If you set this to False, Django will make some optimizations so as not
# to load the internationalization machinery.
USE_I18N = True

# If you set this to False, Django will not format dates, numbers and
# calendars according to the current locale
USE_L10N = True

urls.py

from django.conf.urls import patterns, include, url
from django.conf.urls.i18n import i18n_patterns

# Uncomment the next two lines to enable the admin:
# from django.contrib import admin
# admin.autodiscover()

urlpatterns = patterns( '',
  url( r'^jos2', 'wui.views.jos2' ),
  url(r'^r', 'wui.views.one_aya_page'),
  url(r'^$', 'wui.views.results'),
  # url( r'^admin/', include( admin.site.urls ) ),
)

# These URLs accept the language prefix.
urlpatterns += i18n_patterns('',
  url(r'^$', 'wui.views.results'),
  url(r'^(?P<unit>\w{3,15})/', 'wui.views.results'),
)

# 404 not found handler

handler404 = 'wui.views.custom_404'

Answer 1

Well as far I can test, Django i18n does supports fall-back xx-YY to xx then to default (en in my case) but only for Accept-Language user agent header. It does not do same for URL language switch.

Here is the solution I could come up with:

from django.views.generic import RedirectView
from django.conf import settings
...
urlpatterns += patterns('',
  url(r'^(?P<lang>[a-z]{2})-[A-Za-z]{2}/(?P<path>.*)$', RedirectView.as_view(url='/%(lang)s/%(path)s',query_string=True)),
  url(r'^[a-z]{2}/(?P<path>.*)$', RedirectView.as_view(url='/{}/%(path)s'.format(settings.LANGUAGE_CODE),query_string=True)),
)

• Any xx-YY not handled by i18n pattern redirected to xx
• Any xx not handled by i18n pattern redirected to default language set using LANGUAGE_CODE.