Support for local URLs in install_requires

[ozars]

Version Info

OS: Debian 10 (Buster)

(.venv) omer@omer:~/src/pip-subdir-example$ python --version
Python 3.7.3
(.venv) omer@omer:~/src/pip-subdir-example$ pip --version
pip 20.1.1 from /home/omer/src/pip-subdir-example/.venv/lib/python3.7/site-packages/pip (python 3.7)
(.venv) omer@omer:~/src/pip-subdir-example$ pip list
Package    Version
---------- -------
pip        20.1.1
setuptools 47.1.1
wheel      0.34.2

Issue

Using local URLs (e.g. somepackage @ git+file:///somepath) in install_requires seems like not supported. Currently running python setup.py install or pip install . causes error.

Here is a minimal example:

from setuptools import setup
from pathlib import Path

subpackage_path = (Path(__file__).parent / "deps" / "subpackage").resolve()

setup(
    name="mainpackage",
    version="0.1",
    packages="mainpackage",
    install_requires=[
        f"subpackage @ git+file://{subpackage_path}#subpackage-0.1",
    ],
)

See https://github.com/ozars/pip-subdir-example for complete code. (Run git init in deps/subpackage to treat it like a git submodule.)

Above code gives this error:

(.venv) omer@omer:~/src/pip-subdir-example$ python setup.py install
error in mainpackage setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid URL given
(.venv) omer@omer:~/src/pip-subdir-example$ pip install .
Processing /home/omer/src/pip-subdir-example
    ERROR: Command errored out with exit status 1:
     command: /home/omer/src/pip-subdir-example/.venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-_h1z9t2a/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-_h1z9t2a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-4452gi_r
         cwd: /tmp/pip-req-build-_h1z9t2a/
    Complete output (1 lines):
    error in mainpackage setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid URL given
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I figured out that it is caused by these lines:

setuptools/pkg_resources/_vendor/packaging/requirements.py

Lines 97 to 102 in 6214d4d

	if req.url:
	parsed_url = urlparse.urlparse(req.url)
	if not (parsed_url.scheme and parsed_url.netloc) or (
	not parsed_url.scheme and not parsed_url.netloc):
	raise InvalidRequirement("Invalid URL given")
	self.url = req.url

Above code is under pkg_resources and it looks like the root cause. There is a similar code piece under setuptools, but it is handled differently:

setuptools/setuptools/_vendor/packaging/requirements.py

Lines 102 to 106 in 6214d4d

	if req.url:
	parsed_url = urlparse.urlparse(req.url)
	if parsed_url.scheme == "file":
	if urlparse.urlunparse(parsed_url) != req.url:
	raise InvalidRequirement("Invalid URL given")

This part is updated by bf069fe in #1829.

I was wondering if it's possible to support this behavior. Thanks.