[Forward compatibility][BUG] pkg_resources.extern.VendorImporter lacks find_spec() method

[hroncok]

setuptools version

54.1.2 and main

Python version

3.10.0a7

OS

All

Additional environment information

When running tests for setuptools_scm.

Description

We get:

<frozen importlib._bootstrap>:933: in _find_spec
    ???
E   AttributeError: 'VendorImporter' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:
/usr/lib/python3.10/site-packages/pluggy/hooks.py:286: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/lib/python3.10/site-packages/pluggy/manager.py:93: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
/usr/lib/python3.10/site-packages/pluggy/manager.py:84: in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
/usr/lib/python3.10/site-packages/_pytest/doctest.py:129: in pytest_collect_file
    elif _is_doctest(config, path, parent):
/usr/lib/python3.10/site-packages/_pytest/doctest.py:147: in _is_doctest
    if path.check(fnmatch=glob):
/usr/lib/python3.10/site-packages/py/_path/local.py:387: in check
    return super(LocalPath, self).check(**kw)
/usr/lib/python3.10/site-packages/py/_path/common.py:241: in check
    return self.Checkers(self)._evaluate(kw)
/usr/lib/python3.10/site-packages/py/_path/common.py:108: in _evaluate
    if py.code.getrawcode(meth).co_argcount > 1:
/usr/lib/python3.10/site-packages/py/_vendored_packages/apipkg/__init__.py:152: in __makeattr
    result = importobj(modpath, attrname)
/usr/lib/python3.10/site-packages/py/_vendored_packages/apipkg/__init__.py:72: in importobj
    module = __import__(modpath, None, None, ['__doc__'])
/usr/lib/python3.10/site-packages/py/_code/code.py:7: in <module>
    reprlib = py.builtin._tryimport('repr', 'reprlib')
/usr/lib/python3.10/site-packages/py/_builtin.py:144: in _tryimport
    __import__(name)
<frozen importlib._bootstrap>:1021: in _find_and_load
    ???
<frozen importlib._bootstrap>:996: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:935: in _find_spec
    ???
<frozen importlib._bootstrap>:908: in _find_spec_legacy
    ???
E   ImportWarning: VendorImporter.find_spec() not found; falling back to find_module()

The traceback is not very helpful, but the VendorImporter is from pkg_resourecs.

Python added a warning: https://bugs.python.org/issue42134 but other packages treat it as error.

Expected behavior

No ImportWarning.

How to Reproduce

1. Get Python 3.10.0a7 and tox (e.g. on Fedora via $ sudo dnf --enablerepo=updates-testing install python3.10 tox)
2. git clone https://github.com/pypa/setuptools_scm.git and cd in
3. Run tox -e py310-test

Output

py310-test develop-inst-nodeps: .../setuptools_scm
py310-test installed: attrs==20.3.0,iniconfig==1.1.1,packaging==20.9,pluggy==0.13.1,py==1.10.0,pyparsing==2.4.7,pytest==6.2.3,-e git+https://github.com/pypa/setuptools_scm.git@abb67b15985f380d8cf4451b9f2ef3dd11cb8a91#egg=setuptools_scm,toml==0.10.2
py310-test run-test-pre: PYTHONHASHSEED='1426720794'
py310-test run-test: commands[0] | pytest
============================= test session starts ==============================
platform linux -- Python 3.10.0a7, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
cachedir: .tox/py310-test/.pytest_cache
setuptools version 53.0.0 from '.../setuptools_scm/.tox/py310-test/lib/python3.10/site-packages/setuptools/__init__.py'
setuptools_scm version 6.0.1 from '.../setuptools_scm/src/setuptools_scm/__init__.py'
rootdir: .../setuptools_scm, configfile: tox.ini, testpaths: testing
collected 0 items / 1 error

==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
<frozen importlib._bootstrap>:933: in _find_spec
    ???
E   AttributeError: 'VendorImporter' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:
.tox/py310-test/lib/python3.10/site-packages/pluggy/hooks.py:286: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
.tox/py310-test/lib/python3.10/site-packages/pluggy/manager.py:93: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
.tox/py310-test/lib/python3.10/site-packages/pluggy/manager.py:84: in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
.tox/py310-test/lib/python3.10/site-packages/_pytest/doctest.py:129: in pytest_collect_file
    elif _is_doctest(config, path, parent):
.tox/py310-test/lib/python3.10/site-packages/_pytest/doctest.py:147: in _is_doctest
    if path.check(fnmatch=glob):
.tox/py310-test/lib/python3.10/site-packages/py/_path/local.py:387: in check
    return super(LocalPath, self).check(**kw)
.tox/py310-test/lib/python3.10/site-packages/py/_path/common.py:241: in check
    return self.Checkers(self)._evaluate(kw)
.tox/py310-test/lib/python3.10/site-packages/py/_path/common.py:108: in _evaluate
    if py.code.getrawcode(meth).co_argcount > 1:
.tox/py310-test/lib/python3.10/site-packages/py/_vendored_packages/apipkg/__init__.py:152: in __makeattr
    result = importobj(modpath, attrname)
.tox/py310-test/lib/python3.10/site-packages/py/_vendored_packages/apipkg/__init__.py:72: in importobj
    module = __import__(modpath, None, None, ['__doc__'])
.tox/py310-test/lib/python3.10/site-packages/py/_code/code.py:7: in <module>
    reprlib = py.builtin._tryimport('repr', 'reprlib')
.tox/py310-test/lib/python3.10/site-packages/py/_builtin.py:144: in _tryimport
    __import__(name)
<frozen importlib._bootstrap>:1021: in _find_and_load
    ???
<frozen importlib._bootstrap>:996: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:935: in _find_spec
    ???
<frozen importlib._bootstrap>:908: in _find_spec_legacy
    ???
E   ImportWarning: VendorImporter.find_spec() not found; falling back to find_module()
=========================== short test summary info ============================
ERROR  - ImportWarning: VendorImporter.find_spec() not found; falling back to...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.16s ===============================
ERROR: InvocationError for command .../setuptools_scm/.tox/py310-test/bin/pytest (exited with code 2)
___________________________________ summary ____________________________________
ERROR:   py310-test: commands failed

Code of Conduct

I agree to follow the PSF Code of Conduct

[hroncok]

setuptools/pkg_resources/extern/__init__.py

Lines 23 to 33 in ee6a5ff

	def find_module(self, fullname, path=None):
	"""
	Return self when fullname starts with root_name and the
	target module is one vendored through this importer.
	"""
	root, base, target = fullname.partition(self.root_name + '.')
	if root:
	return
	if not any(map(target.startswith, self.vendored_names)):
	return
	return self

setuptools/setuptools/extern/__init__.py

Lines 23 to 33 in ee6a5ff

	def find_module(self, fullname, path=None):
	"""
	Return self when fullname starts with root_name and the
	target module is one vendored through this importer.
	"""
	root, base, target = fullname.partition(self.root_name + '.')
	if root:
	return
	if not any(map(target.startswith, self.vendored_names)):
	return
	return self

[webknjaz]

@hroncok pro tip: you can mark code blocks that have tracebacks with python-traceback and they'll be more colorful.

[webknjaz]

Ref: https://bugs.python.org/issue43540

[webknjaz]

In that bug, @vstinner asks @brettcannon how to fix this issue — I think the answer would be more useful right here on GitHub rather than on bpo so I tag you both in case you want to comment here :)

[webknjaz]

@hroncok @jaraco a more minimalist reproducer is

$ python3.10 -W error -c 'import pkg_resources'                 
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 939, in _find_spec
AttributeError: 'VendorImporter' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "~/.pyenv/versions/3.10-dev/lib/python3.10/site-packages/pkg_resources/__init__.py", line 74, in <module>
    from pkg_resources.extern import appdirs
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 941, in _find_spec
  File "<frozen importlib._bootstrap>", line 914, in _find_spec_legacy
ImportWarning: VendorImporter.find_spec() not found; falling back to find_module()

$ python3.10 -m pip show setuptools                                     
Name: setuptools
Version: 52.0.0
Summary: Easily download, build, install, upgrade, and uninstall Python packages
Home-page: https://github.com/pypa/setuptools
Author: Python Packaging Authority
Author-email: distutils-sig@python.org
License: UNKNOWN
Location: ~/.pyenv/versions/3.10-dev/lib/python3.10/site-packages
Requires: 
Required-by:

(I used pyenv install 3.10-dev which just got the bleeding edge CPython from the main repository, I didn't even need to change the setuptools version — the one bundled with Python 3.10 causes this same problem; And yeah... it's under Gentoo Linux but the bug is clearly unrelated to the OS)

[webknjaz]

@hroncok this should address the problem #2633