Support openSUSE specific behaviors

[hswong3i]

setuptools version

60.0.3

Python version

Python 3.8

OS

openSUSE Tumbleweed

Additional environment information

During RPM packaging.

Description

I am packaging setuptools 60.0.3 with OBS (https://build.opensuse.org/package/show/home:alvistack/pypa-setuptools-60.0.3) for multiple OS deb/rpm.

It is only failed for openSUSE Tumbleweed during rpm packaging (https://build.opensuse.org/package/live_build_log/home:alvistack/pypa-setuptools-60.0.3/openSUSE_Tumbleweed/x86_64).

platsubdir is coming from an additional patch for openSUSE Factory Python 3.8: https://build.opensuse.org/package/view_file/openSUSE:Factory/python38/SUSE-FEDORA-multilib.patch

Also see https://bugzilla.opensuse.org/show_bug.cgi?id=1193977

P.S. my changes for deb/rpm packaging: alvistack/pypa-setuptools@f73b5bc

Expected behavior

Following OS build are successful:

• Fedora Rawhide (https://build.opensuse.org/package/live_build_log/home:alvistack/pypa-setuptools-60.0.3/Fedora_Rawhide/x86_64)
• Debian Testing (https://build.opensuse.org/package/live_build_log/home:alvistack/pypa-setuptools-60.0.3/Debian_Testing/x86_64)

How to Reproduce

With python3 setup.py install --skip-build we FAILED:

sudo podman run -ti --rm opensuse/tumbleweed bash
zypper update
zypper install -y git curl wget python3 python3-devel python3-pip python3-setuptools python3-wheel
git clone https://github.com/pypa/setuptools.git
cd setuptools
git checkout v60.0.3
python3 setup.py build
python3 setup.py install --skip-build

With pip install . we PASS:

sudo podman run -ti --rm opensuse/tumbleweed bash
zypper update
zypper install -y git curl wget python3 python3-devel python3-pip python3-setuptools python3-wheel
git clone https://github.com/pypa/setuptools.git
cd setuptools
git checkout v60.0.3
python3 setup.py build
pip install .

Output

Full log (https://build.opensuse.org/build/home:alvistack/openSUSE_Tumbleweed/x86_64/pypa-setuptools-60.0.3/_log):

[   22s] + /usr/bin/python3 setup.py install -O1 --skip-build --force --root /home/abuild/rpmbuild/BUILDROOT/python-setuptools-60.0.3-3.1.x86_64 --prefix /usr
[   23s] running install
[   23s] /home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
[   23s]   warnings.warn(
[   23s] Traceback (most recent call last):
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/_distutils/util.py", line 258, in subst_vars
[   23s]     return _subst_compat(s).format_map(lookup)
[   23s] KeyError: 'platsubdir'
[   23s] 
[   23s] During handling of the above exception, another exception occurred:
[   23s] 
[   23s] Traceback (most recent call last):
[   23s]   File "setup.py", line 87, in <module>
[   23s]     dist = setuptools.setup(**setup_params)
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/__init__.py", line 153, in setup
[   23s]     return distutils.core.setup(**attrs)
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/_distutils/core.py", line 148, in setup
[   23s]     return run_commands(dist)
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/_distutils/core.py", line 163, in run_commands
[   23s]     dist.run_commands()
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/_distutils/dist.py", line 967, in run_commands
[   23s]     self.run_command(cmd)
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/_distutils/dist.py", line 985, in run_command
[   23s]     cmd_obj.ensure_finalized()
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/_distutils/cmd.py", line 107, in ensure_finalized
[   23s]     self.finalize_options()
[   23s]   File "setup.py", line 66, in finalize_options
[   23s]     install.finalize_options(self)
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/command/install.py", line 45, in finalize_options
[   23s]     orig.install.finalize_options(self)
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/_distutils/command/install.py", line 422, in finalize_options
[   23s]     self.expand_dirs()
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/_distutils/command/install.py", line 582, in expand_dirs
[   23s]     self._expand_attrs(['install_purelib', 'install_platlib',
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/_distutils/command/install.py", line 572, in _expand_attrs
[   23s]     val = subst_vars(val, self.config_vars)
[   23s]   File "/home/abuild/rpmbuild/BUILD/python-setuptools_60.0.3-3.1/setuptools/_distutils/util.py", line 260, in subst_vars
[   23s]     raise ValueError(f"invalid variable {var}")
[   23s] ValueError: invalid variable 'platsubdir'
[   23s] error: Bad exit status from /var/tmp/rpm-tmp.qnWuVC (%install)

Code of Conduct

• I agree to follow the PSF Code of Conduct

[jaraco]

I notice that platsubdir doesn't exist in the setuptools/distutils code. That tells me that the environment is supplying custom install schemes that rely on that variable.

[jaraco]

I started looking into this issue, but I don't have any muscle memory around setting up opensuse environments. I figured out that the docker image is opensuse/tumbleweed, but that still doesn't give me git or python, so clearly there are more steps. Could you perhaps put together a docker script that replicates the issue?

[hswong3i]

I started looking into this issue, but I don't have any muscle memory around setting up opensuse environments. I figured out that the docker image is opensuse/tumbleweed, but that still doesn't give me git or python, so clearly there are more steps. Could you perhaps put together a docker script that replicates the issue?

With python3 setup.py install --skip-build we FAILED:

sudo podman run -ti --rm opensuse/tumbleweed bash
zypper update
zypper install -y git curl wget python3 python3-devel python3-pip python3-setuptools python3-wheel
git clone https://github.com/pypa/setuptools.git
cd setuptools
git checkout v60.0.3
python3 setup.py build
python3 setup.py install --skip-build

With pip install . we PASS:

sudo podman run -ti --rm opensuse/tumbleweed bash
zypper update
zypper install -y git curl wget python3 python3-devel python3-pip python3-setuptools python3-wheel
git clone https://github.com/pypa/setuptools.git
cd setuptools
git checkout v60.0.3
python3 setup.py build
pip install .

[hswong3i]

@jaraco oh no this platsubdir is coming from an additional patch for openSUSE Factory Python 3.8:

• https://build.opensuse.org/package/view_file/openSUSE:Factory/python38/SUSE-FEDORA-multilib.patch

Bug reported for openSUSE:

• https://bugzilla.opensuse.org/show_bug.cgi?id=1193977

[hswong3i]

Looks like pypa/setuptools@5807a33 between v59.8.0 to v60.0.3 break the python3 setup.py install for openSUSE Tumbleweed patched python38 :-(

[jaraco]

In https://github.com/pypa/distutils/milestone/1, I've been tracking the various platforms' patches of distutils and supporting them. This issue appears to be another such patch. I'll move this issue to distutils as we continue investigation.

Thanks for the repro, I can confirm that the recommended opt-out for distutils is to set SETUPTOOLS_USE_DISTUTILS=stdlib, which does work around the reported issue.

[hswong3i]

In https://github.com/pypa/distutils/milestone/1, I've been tracking the various platforms' patches of distutils and supporting them. This issue appears to be another such patch. I'll move this issue to distutils as we continue investigation.

Thanks for the repro, I can confirm that the recommended opt-out for distutils is to set SETUPTOOLS_USE_DISTUTILS=stdlib, which does work around the reported issue.

With SETUPTOOLS_USE_DISTUTILS=stdlib python3 setup.py install --skip-build we PASS:

sudo podman run -ti --rm opensuse/tumbleweed bash
zypper update
zypper install -y git curl wget python3 python3-devel python3-pip python3-setuptools python3-wheel
git clone https://github.com/pypa/setuptools.git
cd setuptools
git checkout v60.0.3
python3 setup.py build
SETUPTOOLS_USE_DISTUTILS=stdlib python3 setup.py install --skip-build

P.S. this is so annoying for deb/rpm packaging, which 60.0.0+ enforcing me to add SETUPTOOLS_USE_DISTUTILS=stdlib for all build script manually...

[jaraco]

Running with this dockerfile proves the above patch is working:

from opensuse/tumbleweed
RUN zypper update
RUN zypper install -y git curl wget python3 python3-devel python3-pip python3-setuptools python3-wheel
RUN git clone https://github.com/pypa/setuptools.git
WORKDIR setuptools
RUN git config --global user.email "nobody@example.com"
RUN git config --global user.name "Nobody"
RUN git pull --no-rebase https://github.com/pypa/distutils.git
RUN python3 setup.py build
RUN python3 setup.py install --skip-build

[jaraco]

I note that although the install succeeds, there may still be other aspects of the patch that need to be addressed. For example, on Fedora, a patch to sysconfig needs to be added to set _prefix_addition. See some of the other issues in this milestone for various efforts to provide hooks to customize behavior.

[jaraco]

Oh, I should mention too that the install locations need to be customized in sysconfig (if they're not already).