Add initial support for license expression (PEP 639) by cdce8p · Pull Request #4706 · pypa/setuptools

cdce8p · 2024-10-28T05:13:58Z

Summary of changes

Update validate_pyproject to 0.23
https://github.com/abravalheri/validate-pyproject/blob/v0.23/CHANGELOG.rst
https://github.com/abravalheri/validate-pyproject/releases/tag/v0.23
Initial support for license expression in project metadata

 [project]
 name = "spam"
 ...
-license = {text = "MIT"}
+license = "MIT"

If present, the normalized license expression is written to the License field until metadata version 2.4 is supported.

Refs: #4629

abravalheri · 2024-10-28T17:33:16Z

Hi @cdce8p thank you very much for having a look on this.

We probably have to organise so that PEP 639 lands after PEP 685 implementation (ideally we want a couple of days/weeks between the two so that users have time to report potential bugs).

cdce8p · 2024-11-05T13:57:13Z

We probably have to organise so that PEP 639 lands after PEP 685 implementation (ideally we want a couple of days/weeks between the two so that users have time to report potential bugs).

Maybe I'm missing something obvious but I'm not quite sure how PEP 685 is related. Would you mind explaining it a bit more?

The way I've designed the PR, it shouldn't depend on anything. It just allows users to specify license = MIT in pyproject.toml which will be used for the License-Expression field. Note, the metadata version is still 2.2 (not 2.4 which is the first to "require" support for PEP 639).

cdce8p · 2024-11-06T11:12:03Z

Opened #4734 with just the validate-pyproject update. This will make the changes here easier to review.

abravalheri · 2024-11-11T16:36:03Z

Maybe I'm missing something obvious but I'm not quite sure how PEP 685 is related. Would you mind explaining it a bit more?

The problem is that the metadata version of the core metadata is incremental:

PEP 685 stablishes 2.2
PEP 639 stablishes 2.3

So we cannot implement PEP 639 before 685, otherwise metadata validation may fail.

Even if the PR is independent we cannot merge it now because of the nature of the releases of setuptools (we cannot risk PEP 639 support being released before PEP 685 support).

cdce8p · 2024-11-11T17:23:55Z

So we cannot implement PEP 639 before 685, otherwise metadata validation may fail.

The validation will only happen once we update the metadata version.
https://github.com/pypa/packaging/blob/24.2/src/packaging/metadata.py#L826-L831

If not, a lot of wheel uploads would probably fail as we currently include License-File metadata that's not up to spec. #3596 (Fix: #4728).

What I'm proposing here is simply to add the basic support to setuptools. It won't get validated by PyPI until we're ready to move to 2.4 but then all projects using the "new" syntax for pyproject.license will already be ready to go.

abravalheri · 2024-11-11T17:46:49Z

For sure the existence of the License-File in the setuptools-produced metadata with version 2.1 is violating the specs. However, this is a well known violation that the Python ecosystem has grown acquainted to since the early prototype implementation of PEP 639, and for better or worse, learned how to deal with.

By accepting only some aspects of PEP 639, we add other kinds of spec violations that some tools may not be prepared to deal with.

So for the sake of reducing the unknown risks, I propose we pipeline the changes you kindly contributed after the implementation of 685. Unfortunately, it means that people cannot start modifying their pyproject.toml immediately, but it feels safer.

abravalheri · 2024-11-11T18:44:47Z

Sorry, I have been using the wrong PEP number, sorry for the confusion.

PEP 685 is important, but the really big one for us to implement before 639 is PEP 643 (the one with Dynamic).

cdce8p · 2024-11-12T20:19:25Z

So for the sake of reducing the unknown risks, I propose we pipeline the changes you kindly contributed after the implementation of 685. Unfortunately, it means that people cannot start modifying their pyproject.toml immediately, but it feels safer.

I understand your reasoning. One last suggestion. The last days I've been looking into how hatchling does it as I noticed it was one of the earliest adopters of License-Expression in the metadata. Turns out the packaging update caused upload failures pypa/hatch#1786. I can't reproduce it with PyPI at the moment. However, packaging==24.2 rejects any License-Expression and License-File keys with metadata <2.4. ~~That would be relevant for us too.~~ After the issue report, they removed the PEP 639 keys from the metadata and now include the project.license as part of the License key.

My suggestion

~~Remove the License-File keys from the generate metadata as this might cause upload failures once the new packaging version is rolled out to PyPI.~~
Keep the license files in the .dist-info folder but move them to the licenses subfolder in according to the spec. Essentially Store license-files in licenses subfolder #4728
Add support for project.license by back-populating the License key, similar to Back-populate string license fields hatch#1805 Together with Update validate-pyproject to 0.23.0 #4734, this would allow users to use the new syntax while keeping the old metadata format.

What do you think?

cdce8p · 2024-11-25T00:44:38Z

@abravalheri With the last updates, I removed the License-Expression field and instead back-populate the License field with the validated license expression. That's the approach currently taken by hatchling. Since it doesn't add any new fields, would it be ok to do now?

abravalheri · 2025-01-09T17:56:40Z

-    if "file" in val:
+    if isinstance(val, str):
+        _set_config(dist, "license_expression", _static.Str(val))
+    elif "file" in val:


@cdce8p I updated this part of the code to solve the merge conflict. I hope that is OK.

Thanks. Feel free to ping me, especially if one of my PRs need attention. I'm usually fairly quick to respond.

Small side note. If I review PRs from other people, I only update them with merge commits. The rebase one makes it unnecessarily difficult for the OP to follow and understand what's changed on their own PR. I myself only do force pushes if my PR hasn't been reviewed yet, afterwards only merge commits.

If that's ok with you, I'll force push the rebase once #4796 is done. Just so I'm aware what has changed.

If that's ok with you, I'll force push the rebase once #4796 is done. Just so I'm aware what has changed.

Thanks @cdce8p. That would be helpful.
I usually try to avoid merging the full main branch back to a PR, because it messes up the graph visualisation (which becomes difficult to follow), that is why I went with a rebase, sorry if that disrupted your workflow.

I usually try to avoid merging the full main branch back to a PR, because it messes up the graph visualisation (which becomes difficult to follow), that is why I went with a rebase, sorry if that disrupted your workflow.

Don't worry about it. Guess everyone has their own preferred workflow. Just wrote that as it took me a bit to actually see what you've changed during the rebase.

Thanks @cdce8p. That would be helpful.

👍🏻

abravalheri · 2025-02-14T16:47:51Z

            write_field(field, attr_val)

-    license = self.get_license()
+    license = self.license_expression or self.get_license()


So this mean that we need a follow-up PR for the License-Expression, right?

Correct. This PR was designed so that it itself didn't require a metadata_version bump. That could / should happen at a later point once all pieces are in place. I.e. setuptools has adopted version 2.3, the license files are stored in the correct folder and license expression parsing is handled correctly.

abravalheri · 2025-02-17T13:06:36Z

Thank you very much @cdce8p. How do you feel about rebasing this on the feature/pep639 branch?

cdce8p · 2025-02-17T13:14:33Z

Thank you very much @cdce8p. How do you feel about rebasing this on the feature/pep639 branch?

Done. As expected it just removed the first commit here.

Before the rebase I pushed 0d8f1f2 to change the error to the SetuptoolsDeprecationWarning and remove the license classifier.

Let me know if there is anything else I should address here.

abravalheri · 2025-02-17T15:00:57Z

Problem with cygwin in the CI possibly solved by #4832

abravalheri · 2025-02-17T15:36:11Z

+            license_classifiers_found = False
+            for cl in self.metadata.get_classifiers():
+                if not cl.startswith("License :: "):
+                    classifiers.append(cl)
+                    continue
+                license_classifiers_found = True
+                SetuptoolsDeprecationWarning.emit(
+                    "License classifier are deprecated in favor of the license expression.",
+                    f"Please remove the '{cl}' classifier.",
+                    see_url="https://peps.python.org/pep-0639/",
+                    due_date=(2027, 2, 17),  # Introduced 2025-02-17
+                )
+            if license_classifiers_found:
+                self.metadata.set_classifiers(classifiers)


So if the user does not use a license expression in pyproject.toml, we are still going to emit license classifiers in the core metadata right? (because the for-loop is nested inside the if license_expr).

Should we *always skip license classifiers (with the warning) even if the project does not explicitly use a license expression (e.g. instead it relies on the license files)?

(It might also be the case your intention is to introduce a more timid version of the skip/warning in this PR, and later un-nest it and make it more prominent in a follow up. Please let me know if that is the case).

Tbh I don't think setuptools should modify the classifiers in the first place. We should either emit an error or a warning to the user to do so themselves. Everything else would just be surprising in the end. Pushed a new commit to revert that part.

(It might also be the case your intention is to introduce a more timid version of the skip/warning in this PR, and later un-nest it and make it more prominent in a follow up. Please let me know if that is the case).

At some point it might make sense to add another warning to nudge users to use license expressions. However, I do not plan to add it at this time.

* PEP 639: use SPDX-expression for license * remove license classifier * require setuptools >= 77, see pypa/setuptools#4706

cdce8p force-pushed the license-expression branch from 4c8ccfb to 1237a7d Compare October 28, 2024 05:17

cdce8p marked this pull request as draft October 28, 2024 05:58

cdce8p force-pushed the license-expression branch from 1237a7d to 93d0c67 Compare October 28, 2024 12:35

cdce8p marked this pull request as ready for review October 28, 2024 12:58

abravalheri reviewed Oct 28, 2024

View reviewed changes

Comment thread setuptools/_distutils/dist.py Outdated

cdce8p force-pushed the license-expression branch from 93d0c67 to 390b95f Compare October 31, 2024 05:35

cdce8p mentioned this pull request Nov 11, 2024

Validate SPDX license expressions abravalheri/validate-pyproject#217

Merged

cdce8p force-pushed the license-expression branch from 390b95f to 97e8f94 Compare November 25, 2024 00:29

cdce8p changed the title ~~Add initial support for License-Expression (PEP 639)~~ Add initial support for license expression (PEP 639) Nov 25, 2024

cdce8p mentioned this pull request Nov 27, 2024

[BUG] tool.setuptools.license-files results in invalid metadata #4759

Closed

The-Compiler mentioned this pull request Nov 30, 2024

Apply PEP 639 pytest-dev/pytest#13014

Merged

cdce8p mentioned this pull request Dec 11, 2024

Update the specification with PEP 639 pypa/packaging.python.org#1662

Merged

abravalheri added the standardization label Jan 9, 2025

abravalheri force-pushed the license-expression branch from 97e8f94 to f3a1fdd Compare January 9, 2025 17:55

abravalheri reviewed Jan 9, 2025

View reviewed changes

cdce8p force-pushed the license-expression branch from f3a1fdd to e42c5c4 Compare January 9, 2025 18:32

webknjaz mentioned this pull request Jan 27, 2025

chore(release): update pypi packaging codecov/codecov-cli#604

Merged

thomasrockhu-codecov mentioned this pull request Jan 27, 2025

Update LICENSE to PEP639 standards codecov/codecov-cli#605

Open

cdce8p force-pushed the license-expression branch from e42c5c4 to c825f3e Compare January 28, 2025 09:56

bdice mentioned this pull request Feb 11, 2025

Use SPDX license tags in pyproject.toml rapidsai/build-planning#152

Closed

29 tasks

abravalheri changed the base branch from main to feature/pep639 February 14, 2025 16:26

abravalheri reviewed Feb 14, 2025

View reviewed changes

calestyo mentioned this pull request Feb 15, 2025

[BUG] license-files should be allowed in [project]?! #4828

Closed

abravalheri mentioned this pull request Feb 17, 2025

Implement PEP 639 #4829

Merged

6 tasks

cdce8p added 6 commits February 17, 2025 14:11

Add initial support for License-Expression (PEP 639)

285d681

Additional test case

420766c

Normalize license expression

346bf17

Remove License-Expression field

31d8340

Review

3744994

Replace error with warning and remove license classifier

0d8f1f2

cdce8p force-pushed the license-expression branch from 2b6ae9f to 0d8f1f2 Compare February 17, 2025 13:12

cdce8p mentioned this pull request Feb 17, 2025

Bump core metadata version to 2.4 #4830

Merged

abravalheri reviewed Feb 17, 2025

View reviewed changes

Revert removing the license classifier

28baa9b

abravalheri merged commit 016d24a into pypa:feature/pep639 Feb 17, 2025

cdce8p deleted the license-expression branch February 17, 2025 16:02

abravalheri mentioned this pull request Feb 17, 2025

Prevent deprecated license classifiers from being written to core metadata #4833

Merged

2 tasks

XuehaiPan mentioned this pull request Mar 20, 2025

[Docs] Need migration guide for license expression (PEP 639) about TOML-table-based project.license deprecation if v77 unavailable #4903

Open

1 task

di mentioned this pull request Apr 7, 2025

Request: more precise license classifiers pypa/trove-classifiers#17

Closed

robhudson mentioned this pull request May 6, 2025

PEP 639: use SPDX-expression for license mozilla/django-csp#279

Merged

prauscher added a commit to prauscher/django-csp that referenced this pull request May 7, 2025

require setuptools >= 77, see pypa/setuptools#4706

e8b1906

robhudson pushed a commit to mozilla/django-csp that referenced this pull request May 7, 2025

Fix #278: Follow PEP 639: use SPDX-expression for license

c6ed796

* PEP 639: use SPDX-expression for license * remove license classifier * require setuptools >= 77, see pypa/setuptools#4706

spalmurray mentioned this pull request Jul 15, 2025

Update LICENSE to PEP639 standards getsentry/prevent-cli#65

Open

di mentioned this pull request Feb 5, 2026

Deprecate all license classifiers pypa/trove-classifiers#236

Open

neutrinoceros mentioned this pull request Feb 5, 2026

BLD: update build-time dependencies pyproj4/pyproj#1575

Merged

1 task

This was referenced Mar 19, 2026

refactor(build): use SPDX license expression in pyproject.toml fosslight/fosslight_source_scanner#262

Merged

feat: Add test configuration and execution tool RedHatQE/mtv-api-tests#321

Merged

coderabbitai Bot mentioned this pull request Mar 29, 2026

Method lvl iterative analysis CodeBoarding/CodeBoarding#253

Merged

Uh oh!

Conversation

cdce8p commented Oct 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Uh oh!

abravalheri commented Oct 28, 2024

Uh oh!

Uh oh!

cdce8p commented Nov 5, 2024

Uh oh!

cdce8p commented Nov 6, 2024

Uh oh!

abravalheri commented Nov 11, 2024

Uh oh!

cdce8p commented Nov 11, 2024

Uh oh!

abravalheri commented Nov 11, 2024

Uh oh!

abravalheri commented Nov 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cdce8p commented Nov 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cdce8p commented Nov 25, 2024

Uh oh!

abravalheri Jan 9, 2025

Choose a reason for hiding this comment

Uh oh!

cdce8p Jan 9, 2025

Choose a reason for hiding this comment

Uh oh!

abravalheri Jan 9, 2025

Choose a reason for hiding this comment

Uh oh!

cdce8p Jan 9, 2025

Choose a reason for hiding this comment

Uh oh!

abravalheri Feb 14, 2025

Choose a reason for hiding this comment

Uh oh!

cdce8p Feb 16, 2025

Choose a reason for hiding this comment

Uh oh!

abravalheri commented Feb 17, 2025

Uh oh!

cdce8p commented Feb 17, 2025

Uh oh!

abravalheri commented Feb 17, 2025

Uh oh!

abravalheri Feb 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cdce8p Feb 17, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cdce8p commented Oct 28, 2024 •

edited

Loading

abravalheri commented Nov 11, 2024 •

edited

Loading

cdce8p commented Nov 12, 2024 •

edited

Loading

abravalheri Feb 17, 2025 •

edited

Loading