Skip to content

gh-131510: Deprecate PyUnicodeObject structs, revert memory layout#130697

Closed
encukou wants to merge 17 commits into
python:mainfrom
encukou:pyasciiobject-alignas
Closed

gh-131510: Deprecate PyUnicodeObject structs, revert memory layout#130697
encukou wants to merge 17 commits into
python:mainfrom
encukou:pyasciiobject-alignas

Conversation

@encukou

@encukou encukou commented Feb 28, 2025

Copy link
Copy Markdown
Member

(Draft for discussion with the C API WG.)


Deprecate PyASCIIObject, PyCompactUnicodeObject, PyUnicodeObject structs, and the PyUnicode_IS_COMPACT macro.

Schedule removal for 3.16, the shortest non-exceptional deprecation period per PEP-387. The structs are blocking development of the free-threaded build (bitfields can't be used atomically) and of possible UTF-8-focused string representations.
Add an example of what I'd like to see in porting notes for a deprecation.

In non-free-threaded builds, revert PyASCIIObject to be ABI-compatible with in 3.13.

For free-threaded builds, use alignas instead of padding to ensure alignment.
All relevant compilers support alignas, as do the C11 & C++11 standards, but they may need different spelling or compiler options for it.
Introduce _Py_ALIGN_AS for this.


📚 Documentation preview 📚: https://cpython-previews--130697.org.readthedocs.build/

Deprecate `PyASCIIObject`, `PyCompactUnicodeObject`, `PyUnicodeObject`
structs, and the `PyUnicode_IS_COMPACT` macro.

Schedule removal for 3.16, the shortest non-exceptional
deprecation period per PEP-387. The structs are blocking development
of the free-threaded build (bitfields can't be used atomically)
and of possible UTF-8-focused string representations.

In non-free-threaded builds, revert `PyASCIIObject` to the binary
layout used in 3.13.

For free-threaded builds, use `alignas` instead of padding to ensure
alignment.
All relevant compilers support `alignas`, as do the C11 & C++11 standards,
but they may need different spelling or compiler options for it.
@encukou

encukou commented Feb 28, 2025

Copy link
Copy Markdown
Member Author

Discussion: https://discuss.python.org/t/deprecating-the-pyasciiobject-struct/82543

I'll look at the compile errors later. (Thank you GHA; I can't easily do a matrix build locally.)

@encukou encukou added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 5, 2025
@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @encukou for commit 500b483 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F130697%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 5, 2025
@encukou encukou changed the title Deprecate PyUnicodeObject structs, revert memory layout gh-131510: Deprecate PyUnicodeObject structs, revert memory layout Mar 20, 2025
Comment thread Include/pymacro.h
&& (defined(__GNUC__) || defined(__clang__))
# define _Py_ALIGN_AS(V) __attribute__((aligned(V)))
# else
# define _Py_ALIGN_AS(V) alignas(V)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be used if __cplusplus < 201103L and neither __GNUC__ nor __clang__ are defined.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's unknown compilers that aren't in C++11+-standard mode.
IMO, the best we can do is hope that they do support this part of C++11.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if it is an old Microsoft compiler? Then we could use __declspec(align(V)).

@encukou

encukou commented Apr 28, 2025

Copy link
Copy Markdown
Member Author

It's not yet time to deprecate these. I'll send a new PR for the memory layout revert.

@encukou encukou closed this Apr 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants