Unable to install setuptools from GitHub main

[jaraco]

It would be nice if pip were able to install setuptools from GitHub main branch. Instead, if one attempts to pip install, an error about the bootstrap script is emitted:

~ $ pip install git+https://github.com/pypa/setuptools
Collecting git+https://github.com/pypa/setuptools
  Cloning https://github.com/pypa/setuptools to /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-req-build-j6hkzir0
  Installing build dependencies ... done
  Missing build requirements in pyproject.toml for git+https://github.com/pypa/setuptools.
  The project does not specify a build backend, and pip cannot fall back to setuptools without 'setuptools>=38.2.5'.
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-req-build-j6hkzir0/setup.py", line 193, in <module>
        require_metadata()
      File "/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-req-build-j6hkzir0/setup.py", line 24, in require_metadata
        raise RuntimeError(msg)
    RuntimeError: Cannot build setuptools without metadata. Run `bootstrap.py`.

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-req-build-j6hkzir0/

I'm wondering if there's a solution to this limitation. I can imagine a few:

• setuptools could wait for and rely on PEP 517 support and provide its own "installer" (itself) that does the bootstrapping.
• pip could special-case the installation of anything that looks like setuptools and run bootstrap.py before installing.
• setuptools could materialize the metadata (egg-info) to the repo, avoiding the need to bootstrap (but requiring special handling of that metadata which is derived from other files in the repo).
• setuptools could inject a hack to bootstrap and relaunch (this hack might interfere or interact badly with pip's injection of setuptools).

There are probably others. Only the first option seems (somewhat) clean.

[LucidOne]

This is causing me issues. On Ubuntu 20.04, I'm trying to install from a local mirror via

python3 -m pip install --no-index --user git+file:/mirror/pypa/setuptools.git@50.0.1 --no-build-isolation

I think having require_metadata() run bootstrap.py and relaunch makes the most sense, as setuptools is probably the only thing that needs this and it keeps the extra tooling where it is needed. Perhaps it would be less hacky to use importlib to reload setup.py, maybe.

[jaraco]

I've tried relying on PEP-517 which has landed, but ran into issues. See also #2001 where I expended some energy on this effort.

[JulienPalard]

According to bootstrap.py, it's only usefull if setuptools is not installed yet, so I though we could call require_metadata only if setuptools is not installed yet, but looks like there's already an import setuptools before the require_metadata() call, so I don't see what bootstrap.py fixes.

Can someone shed a light on what I missed? (I know nothing about egg_info, I may miss something big).

[jaraco]

The underlying problem is this:

• Setuptools repo doesn't have metadata (most repos don't).
• Setuptools relies on itself (including its metadata) to generate metadata (egg_info command) for a project.
• Attempting to run egg_info on setuptools has no effect until metadata is present.

The bootstrap.py function works around this issue by generating some minimal metadata outside of Setuptools, enough that Setuptools can then generate its own metadata.

The require_metadata function in setup.py is a stopgap to provide a clear indication when metadata isn't present and to direct the user to run bootstrap.py to create the minimal metadata before attempting to run setup.py again.

Does that clarify?

[JulienPalard]

Does that clarify?

A bit, thanks. But I won't spend too much time on it as it looks like it's no longer usefull thanks to pip / PEP 517.