GH-140061: Use _PyObject_IsUniquelyReferenced to check if object is uniquely referenced

[sergey-miryanov]

There are a lot of places where Py_REFCNT(..)==1 used. IIUC it is not correct for no-gil version.

• Issue: Use _PyObject_IsUniquelyReferenced instead of Py_REFCNT(x) == 1 for free-threaded compatibility #140061

[sergey-miryanov]

I think it should skip news, but not sure about backporting.

[ZeroIntensity]

skip news isn't right -- Py_REFCNT(op) == 1 is an actual bug on the free-threaded build (and sometimes the GILful build), at least when using it for optimizations.

[ZeroIntensity]

When the reference is owned by the eval loop, we need to use PyUnstable_Object_IsUniqueReferencedTemporary instead. I think there are a few cases of that here.

cc @colesbury

[colesbury]

• You need to be careful about changing assert(), especially when the code paths can be called by extensions.
• There are a number of conditions like Py_REFCNT(result) > 1 that need to be changed, such as in itertooslmodule.c

[sergey-miryanov]

@ZeroIntensity @colesbury Thanks for review! I have fixed your suggestions, but it fails in TSAN(free-threading). Can you take a look?

[ZeroIntensity]

It seems that me and Kumar broke that earlier (see #140138), sorry! You can ignore it for now and we'll rebase this once we get it fixed on main.

[sergey-miryanov]

@ZeroIntensity Thanks!

[vstinner]

Is this PR ready for a review?

[sergey-miryanov]

@vstinner Yes, it is ready to review. If @colesbury don't have any comments, of course.

[vstinner]

Ok, I click on [Ready for review] button in this case :-)

[sergey-miryanov]

Ough, I forgot that I turned it to draft :(

[vstinner]

Remaining code using Py_REFCNT() with 1:

$ git grep 'Py_REFCNT.*\(==\|<\|<=\|>\|>=\) *1'
Doc/c-api/object.rst:   :c:expr:`Py_REFCNT(op) == 1`.
Doc/c-api/object.rst:   is only used by this thread. :c:expr:`Py_REFCNT(op) == 1` is **not**
Doc/whatsnew/3.14.rst:  a replacement for ``Py_REFCNT(op) == 1`` on :term:`free threaded
Include/internal/pycore_object.h:    return Py_REFCNT(ob) == 1;
Modules/_ctypes/_ctypes.c:   assert (Py_REFCNT(obj) == 1);
Modules/_functoolsmodule.c:            assert(Py_REFCNT(args) == 1);
Modules/_io/bytesio.c:  * Py_REFCNT(buf) == 1, exports == 0.
Modules/_io/bytesio.c:  * Py_REFCNT(buf) > 1.  exports == 0,
Modules/_io/bytesio.c:  * exports > 0.  Py_REFCNT(buf) == 1, any modifications are forbidden.
Modules/_testcapi/object.c:    assert(Py_REFCNT(obj) == 1);
Modules/_testcapi/object.c:    assert(Py_REFCNT(obj) == 1);
Modules/_testcapi/object.c:        assert(Py_REFCNT(obj) == 1); \
Modules/_testcapimodule.c:    assert(Py_REFCNT(obj) == 1);
Modules/itertoolsmodule.c:        assert (npools==0 || Py_REFCNT(result) == 1);
Modules/itertoolsmodule.c:        assert(r == 0 || Py_REFCNT(result) == 1);
Modules/itertoolsmodule.c:        assert(r == 0 || Py_REFCNT(result) == 1);
Modules/itertoolsmodule.c:        assert(r == 0 || Py_REFCNT(result) == 1);
Objects/listobject.c:    assert(Py_REFCNT(self) == 1 || PyMutex_IsLocked(&_PyObject_CAST(self)->ob_mutex));
Objects/longobject.c:                assert(Py_REFCNT(z) == 1);
Objects/longobject.c:                assert(_PyLong_IsZero(z) || Py_REFCNT(z) == 1);
Objects/longobject.c:        assert(Py_REFCNT(z) == 1);
Objects/longobject.c:    assert(Py_REFCNT(obj) == 1);
Objects/longobject.c:    assert(Py_REFCNT(obj) == 1);
Objects/object.c:    CHECK(Py_REFCNT(op) >= 1);
Objects/object.c:    _PyObject_ASSERT(name, Py_REFCNT(name) >= 1);
Objects/setobject.c:    assert(Py_REFCNT(so) == 1);
Objects/typeobject.c:    CHECK(Py_REFCNT(type) >= 1);
Objects/unicodeobject.c:    assert(Py_REFCNT(unicode) == 1);
Objects/unicodeobject.c:            assert(Py_REFCNT(unicode) == 1);
Python/ceval.c:    assert(Py_REFCNT(locals) > 1);
Python/gc_free_threading.c:                _PyObject_ASSERT(op, Py_REFCNT(op) > 1);

There are documentation and assertions. Assertions should continue to use Py_REFCNT(), right?

[vstinner]

LGTM

[sergey-miryanov]

Assertions should continue to use Py_REFCNT(), right?

As I understand @colesbury - yes.

[colesbury]

Thanks @sergey-miryanov!

[miss-islington-app]

Thanks @sergey-miryanov for the PR, and @colesbury for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

[bedevere-app]

GH-140157 is a backport of this pull request to the 3.14 branch.

[sergey-miryanov]

Thanks everyone for review!