Lift pip.pep425tags into packaging

[leorochael]

While discussing how to implement wheel installation support into setuptools, @dholth mentioned that the wheel package had its own implementation of PEP-425 parsing, that was different than the one in pip.

In order not to duplicate another piece of code, wouldn't it be better to lift pep425tags into packaging, which is vendored in both pip and setuptools?

And since the main consumer of these tags are functions that parse wheel filenames, what do you think about lifting the pip.wheel:Wheel class as well? (which would be better named WheelFilename, since pretty much all it does is parse information from a potential wheel filename to help deciding whether it should be downloaded/installed or not).

[dstufft]

I'm generally +1 on moving this functionality into packaging. Though I want to be careful about how we do it, I don't want to just blindly copy the code that is in either wheel or pip into packaging but take the chance to figure out what the ideal API is for this (and not an API that suited one or the other's specific purpose best, but maybe isn't as good for other purposes).

I don't have anything specific about what the PEP 425 tags implementation would look like, but I think that the filename thing should be something that can be extended to support sdists as well (likely with two different classes and a singular parse function or something).

If you have any ideas for what an API for either of these might look like in an ideal world, feel free to show an example or submit a PR (and to be clear, I'm not saying that it needs a drastic overhaul from what exists already either, I'm just saying that we don't need to be constrained by the specific API decisions made by either of those projects).

[dholth]

The tag parsing is not appreciably different between pip and wheel.

pep425tags is not a parser. Its job is to generate a list of all supported tags on the current machine, in order of preference. pip's copy started from the one in wheel but they have diverged as no one realized it was a vendored file.

[dstufft]

That's kind of what I'm talking about in terms of not blindly copying code. I don't think we should just copy something that only works with the current machine. Take a look at the marker implementation here where sure, by default it pulls it's information from the current machine, but in general the API is designed in such a way that lets you evaluate the markers in the context of another machine. Not only does this make the API more flexible, but it makes it much easier to test since it's working to a defined interface that is passed into it rather than needing to mock random bits of the standard library.

[leorochael]

Since the great API internalizing of pip 10, I think this issue was responsible for a large number of breakages, as the logic to decide which tags are valid for a certain machine is something that no one wants to reimplement and get wrong.

@dstufft said:

That's kind of what I'm talking about in terms of not blindly copying code. I don't think we should just copy something that only works with the current machine. Take a look at the marker implementation here where sure, by default it pulls it's information from the current machine, but in general the API is designed in such a way that lets you evaluate the markers in the context of another machine. Not only does this make the API more flexible, but it makes it much easier to test since it's working to a defined interface that is passed into it rather than needing to mock random bits of the standard library.

In fact the get_supported() method, which is what most people are after, behaves like you described, accepting parameters to override versions=None, noarch=False, platform=None, impl=None, abi=None, but defaulting to the ones for the currently running interpreter.

If a PR moving pep425tags into packaging "as is" was proposed. What would be its probability of being accepted?

BTW, setuptools already forked it (discussion here). I think it's better to stop the proliferation of forks sooner rather than later.

[anthrotype]

and also pex forked pip's pep425tags.py the other day
https://github.com/pantsbuild/pex/blob/master/pex/pep425tags.py