[BUG] Permission denied error to install pkg editable and user mode

[jinserk]

setuptools version

60.5.0

Python version

3.10.1

OS

Archlinux

Additional environment information

No response

Description

When I install a custom package with pip in user mode and editable, it produces permission denied error:

$ pip install --user --no-deps --no-use-pep517 -e .
Obtaining file:///home/jinserk/public/c2tk
  Preparing metadata (setup.py) ... done
Installing collected packages: c2tk
  Running setup.py develop for c2tk
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/jinserk/public/c2tk/setup.py'"'"'; __file__='"'"'/home/jinserk/public/c2tk/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix=
         cwd: /home/jinserk/public/c2tk/
    Complete output (7 lines):
    running develop
    /home/jinserk/.local/lib/python3.10/site-packages/setuptools/command/easy_install.py:156: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    /home/jinserk/.local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running egg_info
    error: [Errno 13] Permission denied
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/jinserk/public/c2tk/setup.py'"'"'; __file__='"'"'/home/jinserk/public/c2tk/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.

Expected behavior

Install the package into $HOME/.local/lib/python3.10/site-packages

How to Reproduce

1. Check you're using python 3.10.1
2. upgrade all the build env

pip install -U pip setuptools wheel

3. clone the repo:

git clone https://github.com/jinserk/c2tk.git

4. Install it with pip, editable and user mode

cd c2tk
pip install --user --no-deps --no-use-pep517 -e .

Output

$ pip install --user --no-deps --no-use-pep517 -e .
Obtaining file:///home/jinserk/public/c2tk
  Preparing metadata (setup.py) ... done
Installing collected packages: c2tk
  Running setup.py develop for c2tk
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/jinserk/public/c2tk/setup.py'"'"'; __file__='"'"'/home/jinserk/public/c2tk/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix=
         cwd: /home/jinserk/public/c2tk/
    Complete output (7 lines):
    running develop
    /home/jinserk/.local/lib/python3.10/site-packages/setuptools/command/easy_install.py:156: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    /home/jinserk/.local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running egg_info
    error: [Errno 13] Permission denied
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/jinserk/public/c2tk/setup.py'"'"'; __file__='"'"'/home/jinserk/public/c2tk/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.

Code of Conduct

• I agree to follow the PSF Code of Conduct

[abravalheri]

Hi @jinserk, thank you very much for submitting this issue.
Could you please use the verbose mode in pip (-vvv) and also send us the log (not only the output)?
(you can specify a file to collect the logs with --log).

[abravalheri]

Also, what would be your output for the command python3 -c 'import site; print(site.USER_SITE)'?

[jinserk]

Thanks @abravalheri . Here is the log file using verbose option:

2022-01-09T18:12:57,862 Using pip 21.3.1 from /home/mpiuser/.local/lib/python3.10/site-packages/pip (python 3.10)
2022-01-09T18:12:57,863 User install by explicit request
2022-01-09T18:12:57,870 Created temporary directory: /tmp/pip-ephem-wheel-cache-g0zddrcv
2022-01-09T18:12:57,870 Created temporary directory: /tmp/pip-req-tracker-7ihl3bkf
2022-01-09T18:12:57,870 Initialized build tracking at /tmp/pip-req-tracker-7ihl3bkf
2022-01-09T18:12:57,871 Created build tracker: /tmp/pip-req-tracker-7ihl3bkf
2022-01-09T18:12:57,871 Entered build tracker: /tmp/pip-req-tracker-7ihl3bkf
2022-01-09T18:12:57,871 Created temporary directory: /tmp/pip-install-1w6d8y9d
2022-01-09T18:12:57,879 Obtaining file:///home/mpiuser/c2tk
2022-01-09T18:12:57,880   Added file:///home/mpiuser/c2tk to build tracker '/tmp/pip-req-tracker-7ihl3bkf'
2022-01-09T18:12:57,880   Running setup.py (path:/home/mpiuser/c2tk/setup.py) egg_info for package from file:///home/mpiuser/c2tk
2022-01-09T18:12:57,880   Created temporary directory: /tmp/pip-pip-egg-info-qouig4uy
2022-01-09T18:12:57,880   Running command python setup.py egg_info
2022-01-09T18:12:58,140   running egg_info
2022-01-09T18:12:58,141   creating /tmp/pip-pip-egg-info-qouig4uy/c2tk.egg-info
2022-01-09T18:12:58,142   writing manifest file '/tmp/pip-pip-egg-info-qouig4uy/c2tk.egg-info/SOURCES.txt'
2022-01-09T18:12:58,144   writing manifest file '/tmp/pip-pip-egg-info-qouig4uy/c2tk.egg-info/SOURCES.txt'
2022-01-09T18:12:58,162   running egg_info
2022-01-09T18:12:58,165   writing manifest file '/tmp/pip-pip-egg-info-qouig4uy/c2tk.egg-info/SOURCES.txt'
2022-01-09T18:12:58,196   Source in /home/mpiuser/c2tk has version 0.0.1, which satisfies requirement c2tk==0.0.1 from file:///home/mpiuser/c2tk
2022-01-09T18:12:58,196   Removed c2tk==0.0.1 from file:///home/mpiuser/c2tk from build tracker '/tmp/pip-req-tracker-7ihl3bkf'
2022-01-09T18:12:58,226 Created temporary directory: /tmp/pip-unpack-0txv5pep
2022-01-09T18:12:58,226 Installing collected packages: c2tk
2022-01-09T18:12:58,227   Running setup.py develop for c2tk
2022-01-09T18:12:58,227     Running command /usr/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/mpiuser/c2tk/setup.py'"'"'; __file__='"'"'/home/mpiuser/c2tk/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix=
2022-01-09T18:12:58,510     running develop
2022-01-09T18:12:58,510     /home/mpiuser/.local/lib/python3.10/site-packages/setuptools/command/easy_install.py:156: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
2022-01-09T18:12:58,510       warnings.warn(
2022-01-09T18:12:58,513     /home/mpiuser/.local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
2022-01-09T18:12:58,513       warnings.warn(
2022-01-09T18:12:58,604     running egg_info
2022-01-09T18:12:58,604     error: [Errno 13] Permission denied
2022-01-09T18:12:58,640 ERROR: Command errored out with exit status 1: /usr/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/mpiuser/c2tk/setup.py'"'"'; __file__='"'"'/home/mpiuser/c2tk/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.
2022-01-09T18:12:58,640 Exception information:
2022-01-09T18:12:58,640 Traceback (most recent call last):
2022-01-09T18:12:58,640   File "/home/mpiuser/.local/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper
2022-01-09T18:12:58,640     status = run_func(*args)
2022-01-09T18:12:58,640   File "/home/mpiuser/.local/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
2022-01-09T18:12:58,640     return func(self, options, args)
2022-01-09T18:12:58,640   File "/home/mpiuser/.local/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 404, in run
2022-01-09T18:12:58,640     installed = install_given_reqs(
2022-01-09T18:12:58,640   File "/home/mpiuser/.local/lib/python3.10/site-packages/pip/_internal/req/__init__.py", line 73, in install_given_reqs
2022-01-09T18:12:58,640     requirement.install(
2022-01-09T18:12:58,640   File "/home/mpiuser/.local/lib/python3.10/site-packages/pip/_internal/req/req_install.py", line 776, in install
2022-01-09T18:12:58,640     install_editable_legacy(
2022-01-09T18:12:58,640   File "/home/mpiuser/.local/lib/python3.10/site-packages/pip/_internal/operations/install/editable_legacy.py", line 43, in install_editable
2022-01-09T18:12:58,640     call_subprocess(
2022-01-09T18:12:58,640   File "/home/mpiuser/.local/lib/python3.10/site-packages/pip/_internal/utils/subprocess.py", line 254, in call_subprocess
2022-01-09T18:12:58,640     raise InstallationSubprocessError(proc.returncode, command_desc)
2022-01-09T18:12:58,640 pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status 1: /usr/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/mpiuser/c2tk/setup.py'"'"'; __file__='"'"'/home/mpiuser/c2tk/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.
2022-01-09T18:12:58,646 Removed build tracker: '/tmp/pip-req-tracker-7ihl3bkf

and the result of site.USER_SITE:

$ python
Python 3.10.1 (main, Dec 18 2021, 23:53:45) [GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import site
>>> print(site.USER_SITE)
/home/mpiuser/.local/lib/python3.10/site-packages

[abravalheri]

Hi @jinserk, thank you very much for providing the logs.

Unfortunately pip does not help a lot by hiding the stack trace of the error 😞
So I will have to investigate it deeper. I am afraid it might take some time.

Meanwhile if you need to carry on with your work, please consider using an editable install inside a virtual environment.

[hexagonrecursion]

Ref: https://github.com/pypa/pip/blob/6d1c027ec877f9a52bd7e7d3ff53e3a4ca7e9cea/src/pip/_internal/operations/install/editable_legacy.py

[abravalheri]

Thank you very much for the reference @hexagonrecursion, I still haven't had the time to dive into the issue, but this link will definitely be useful!

[hexagonrecursion]

I have found a workaround:

import site
site.ENABLE_USER_SITE=True

from setuptools import setup
setup()

I do not know why ENABLE_USER_SITE is false when setup.py develop is executed via pip. Gotta run. Will look into this later.

[hexagonrecursion]

setuptools/setuptools/command/easy_install.py

Lines 272 to 277 in 00fbad0

	if site.ENABLE_USER_SITE:
	self.config_vars['userbase'] = self.install_userbase
	self.config_vars['usersite'] = self.install_usersite
	elif self.user:
	log.warn("WARNING: The user site-packages directory is disabled.")

[abravalheri]

@hexagonrecursion thank you very much for investigating that and solving it!
Would you like to open a PR?

[hexagonrecursion]

Would you like to open a PR?

What PR? I'll open one once I have figured out how to fix it. For now I only know how to hack around the issue and even then I am not sure that my hack works properly. I am, havever, making progress towards uncovering the root cause.