Reject packages without a Requires-Python

[pganssle]

A problem I've come across many times now is that a package will drop support for an old version and they'll even add a python_requires directive to their setup.py, only to cut a release using an old version of setuptools that doesn't actually understand the python_requires directive and thus silently fails. See, e.g. sphinx. This causes a bunch of problems because the erroneously uploaded packages now pollute pip installs of that package unless they are removed from PyPI or a post release is made.

Since there is no good reason to upload a package without explicitly specifying a Requires-Python directive, I think we can assume that this is an error and reject packages that don't have a Requires-Python.

My recommendation for migrating to the "always throw an error" version:

1. Start throwing a warning when Requires-Python is missing, with a link to the documentation on how to add Requires-Python
2. Update twine and all tools that upload packages (and setuptools, even though it's deprecated) to automatically transform that warning into an error overridable by --allow-no-python-requires
3. Add a backwards-compatibility API that allows you to upload packages without Requires-Python by configuring your upload endpoint to something not the default
4. Upgrade the warning to an error, dropping support for --allow-no-python-requires.

I think we could swap the order of 2 and 3 pretty easily - I'm guessing that updating the upload tools would be easier to do which is why I put them in this order, but 2 could be implemented in terms of 3.

In terms of time frame, I don't know how aggressive you (the Warehouse team) want to be. I think the first 3 can happen as soon as an implementation is available. It's probably a big ask to have all package maintainers switch over in a relatively short period of time, but I think a long deprecation period will be harmful given that it's likely a large number of packages are going to start dropping support for Python 2 soon, which will probably cause a ton of headaches if people aren't including Requires-Python in their metadata. Maybe a 6 months or so?

[di]

Since there is no good reason to upload a package without explicitly specifying a Requires-Python directive, I think we can assume that this is an error and reject packages that don't have a Requires-Python.

One use case would be a package that is compatible (or at least, thinks it's compatible) with all versions of Python. What should this field be for those packages, something like Requires-Python: >0?

1. Start throwing a warning when Requires-Python is missing, with a link to the documentation on how to add Requires-Python

I don't believe displaying a warning for a successful upload is possible at the moment without making changes to twine and the upload API, but this would probably be a beneficial change to make in general.

Worth mentioning that #726 might help with this a bit, because maintainers would be able to verify that the Requires-Python metadata they set actually made it to PyPI.

[pganssle]

@di Yes, agreed that #726 would be very helpful.

One use case would be a package that is compatible (or at least, thinks it's compatible) with all versions of Python. What should this field be for those packages, something like Requires-Python: >0?

Presumably python_requires="*.*" or something like that? >0 would also work, though I really doubt there are any packages out there that actually support Python<2.0 anyway.

don't believe displaying a warning for a successful upload is possible at the moment without making changes to twine and the upload API, but this would probably be a beneficial change to make in general.

Hm. That's a bit of a problem, because half the issue here is that people tend to install setuptools or twine once and then never upgrade it. The people with updated versions of twine and setuptools are the least in need of this change.

Other possible options:

1. Rolling blackouts as was done for TLS and the PyPI.org transition, skipping the "raise a warning" phase entirely.
2. Send an e-mail to people who upload new packages without a Requires-Python directive (this might be way more e-mail than can be done safely)
3. Do some verification to check if the source distribution includes a python_requires directive, but the metadata failed to upload and send an e-mail only to those people (with instructions on how to fix it).
4. Implement the "warning" as a "fail only the first time" - parse the metadata and if it doesn't contain Requires-Python, hash the payload and store it, then return an error explaining why it failed. Uploading the exact same payload a second time would actually allow the upload to proceed.

If #726 is happening soon, option 4 could be implemented as "if you upload and it doesn't have Requires-Python, you must go through the 2-phase upload process and check a big red box that says, "I don't want my packages to properly advertise its requirements" before allowing it to proceed.

[edmorley]

It seems that the other half of this issue is "how can we make sure users update setuptools/twine more often?" - which would help for more than just this case.

For example:

• making twine perform version checks periodically and output a warning if it or setuptools/wheel out of date
• actively working with CI providers to ensure the tooling they use is up to date (it looks like Travis' PyPI deploy addon already self-updates, but there are likely other common workflows that don't use it, as well as other providers)
• updating as many of the "how to publish your Python package" docs/guides to begin with a pip install -U ...
• (plus the idea of adding support for passing back a warning message on successful upload mentioned above, so at least new installs are more likely to stay up to date in the future)

Regarding users already on older broken versions - less disruptive options (vs brownouts/blocking) might be:

• displaying a banner on the package page or when the user logs in
• emailing the user after upload

[Carreau]

It seems that the other half of this issue is "how can we make sure users update setuptools/twine more often?"

Have it send a header with which tool and version. Slowly refuse upload from old-versions, tools that don't set these headers.