Skip to content

Correct invalid universal locale names#122877

Closed
stefanor wants to merge 1 commit into
python:mainfrom
stefanor:invalid-locales
Closed

Correct invalid universal locale names#122877
stefanor wants to merge 1 commit into
python:mainfrom
stefanor:invalid-locales

Conversation

@stefanor

Copy link
Copy Markdown
Contributor

These probably don't matter for anything. According to the comments in locale.alias, univ and universal apply to HPUX 9.x.

But the "en_US.utf" values are obviously incorrect, and invalid in glibc. Debian has been carrying this patch since around Python 2.4. Either it should be upstream, or it should be dropped from Debian.

Looking into this leads to bpo-20087, which temporarily fixed this before, as part of a larger cleanup in the migration to glibc 2.24 locales. That was reverted.

The invalid value itself comes from xfree86, as far as I can tell. It appeared in 1999 0 and were corrected in 2000 1. locale.py was written in-between and cribbed the broken values 2.

@vstinner

Copy link
Copy Markdown
Member

cc @serhiy-storchaka

These probably don't matter for anything. According to the comments in
locale.alias, univ and universal apply to HPUX 9.x.

But the "en_US.utf" values are obviously incorrect, and invalid in glibc.
Debian has been carrying this patch since around Python 2.4. Either it
should be upstream, or it should be dropped from Debian.

Looking into this leads to bpo-20087, which temporarily fixed this
before, as part of a larger cleanup in the migration to glibc 2.24
locales. That was reverted.

The invalid value itself comes from xfree86, as far as I can tell.
It appeared in 1999 [0] and were corrected in 2000 [1]. locale.py was
written in-between and cribbed the broken values [2].

[0]: https://gitlab.freedesktop.org/ajax/xfree86/-/commit/16664e079de9938a4354e94c5c5afe5476bbaa98#8c0c2f24be5c75f99e8d6e55aa310736636d2584_10_480
[1]: https://gitlab.freedesktop.org/ajax/xfree86/-/commit/40f478e907f33fc56633bb16f7a6756314d0c10d?page=3#8c0c2f24be5c75f99e8d6e55aa310736636d2584_496_496
[2]: 5431bc3

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@malemburg malemburg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is fine. Thank you.

@malemburg

Copy link
Copy Markdown
Member

Do we need an issue # and NEWS entry for this ?

@serhiy-storchaka

Copy link
Copy Markdown
Member

This data is generated from the locale.alias file from X.org distribution. Some old aliases are preserved from old times. Currently there are no "univ" and "universal" aliases in the mainstream (https://gitlab.freedesktop.org/xorg/lib/libx11/-/blob/master/nls/locale.alias.pre?ref_type=heads) -- there are "univ.utf8" and "universal.utf8@ucs4" instead. The same is on my version of Ubuntu.

I think that we can replace "univ" and "universal" with "univ.utf8" and "universal.utf8@ucs4". We can also use opportunity to update the rest of the mapping. I have created issue #64286.

@serhiy-storchaka

Copy link
Copy Markdown
Member

#129647 includes also this change.

@stefanor

stefanor commented Feb 4, 2025

Copy link
Copy Markdown
Contributor Author

Thanks @serhiy-storchaka! We can close this, then.

@stefanor stefanor closed this Feb 4, 2025
@stefanor stefanor deleted the invalid-locales branch February 4, 2025 13:23
@malemburg

Copy link
Copy Markdown
Member

Regeneration of the mappings from a more up-to-date X11 file would, of course, also be good, but since the current X11 file does not include those old lowercase names, they would not get updated via the script, AFAIR.

@malemburg

Copy link
Copy Markdown
Member

Looking at @serhiy-storchaka new ticket, the old entries would get removed, which is probably fine as well.

@serhiy-storchaka

Copy link
Copy Markdown
Member

I removed them manually before using makelocalealias.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants