Skip to content

Sort file lists#1068

Merged
jaraco merged 3 commits into
pypa:masterfrom
bmwiedemann:sortzip
Sep 3, 2017
Merged

Sort file lists#1068
jaraco merged 3 commits into
pypa:masterfrom
bmwiedemann:sortzip

Conversation

@bmwiedemann

Copy link
Copy Markdown
Contributor

to generate reproducible zip files
that do not differ depending on (random) filesystem order

See https://reproducible-builds.org/ for why this matters.

@jaraco jaraco left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this change is reasonable, though I'd really like to see the functionality encapsulated in a single place that might have a docstring to explain why it's there. The PR would also be substantially better with a test case to capture the expectation (as otherwise this change could get optimized away without any test failures).

Importantly, a change like this needs an entry in the CHANGES.txt file (probably 0.1 bump).

Comment thread setuptools/command/bdist_egg.py Outdated
for base, dirs, files in os.walk(self.bdist_dir):
for filename in files:
dirs.sort()
for filename in sorted(files):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use sorted for files but .sort for dirs?

Comment thread setuptools/command/bdist_egg.py Outdated
"""Walk an unpacked egg's contents, skipping the metadata directory"""
walker = os.walk(egg_dir)
base, dirs, files = next(walker)
dirs.sort()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's starting to feel like this change is getting sprinkled around a few places. Maybe it would make more sense to have a "sorted walk" function or similar.

to generate reproducible zip files
that do not differ depending on (random) filesystem order

See https://reproducible-builds.org/ for why this matters.
@bmwiedemann

Copy link
Copy Markdown
Contributor Author

Thanks for your nice feedback.

I added a sorted_walk function with docstring, but I have not fully tested it
and I could use some help with adding a test, because my python-foo is not the best.
Something like:

  1. create numbered file entries 1..9 (in random order)
  2. create an .egg from it
  3. check that entries in the zipfile are ordered 1..9

@jaraco

jaraco commented Sep 3, 2017

Copy link
Copy Markdown
Member

This new PR is much cleaner. And given that eggs are largely deprecated in favor of wheels, I'm not going to be a stickler about the test.

@jaraco jaraco merged commit 24f240e into pypa:master Sep 3, 2017
@bmwiedemann bmwiedemann deleted the sortzip branch July 22, 2020 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants