UnicodeDecodeError since 18.7 when determining version from file with LC_ALL=C

Originally reported by: The-Compiler (Bitbucket: The-Compiler, GitHub: The-Compiler)

---

When an user has LC_ALL=C set (which unfortunately happens more often than I'd like), starting a setuptools script seems to fail:

$ LC_ALL=C /usr/bin/qutebrowser
Traceback (most recent call last):
  File "/usr/bin/qutebrowser", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 3143, in <module>
    @_call_aside
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 3129, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 3156, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 649, in _build_master
    ws = cls()
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 642, in __init__
    self.add_entry(entry)
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 698, in add_entry
    for dist in find_distributions(entry, True):
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2136, in find_on_path
    path_item, entry, metadata, precedence=DEVELOP_DIST
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2518, in from_location
    version = cls._version_from_metadata(dist_path) or version
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2841, in _version_from_metadata
    return _version_from_file(strm)
  File "/usr/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2483, in _version_from_file
    line = next(iter(version_lines), '')
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 159: ordinal not in range(128)

This seems due to some other packages using non-ASCII chars in their long_descriptions, and pkg_resources trying to read PKG_INFO with the system encoding to find out the version.

---

• Bitbucket: https://bitbucket.org/pypa/setuptools/issue/469

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Add test for regression on Python 3 when LANG=C and there is non-ascii in the metadata file before the version. Ref #469.

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Annoyingly, I can't get the tests to fail. That test fails on my local machine (Python 3.5.0, OS X 10.11.1) with LANG=C and PYTHONHASHSEED=0. Why not on Travis? I don't know, so I'm going to punt, but if someone can show me how to make that test fail on Travis, I'd like to have a regression test there.

Original comment by The-Compiler (Bitbucket: The-Compiler, GitHub: The-Compiler):

---

Hm, I have no idea - have you tried with LC_ALL=C instead of LANG=C?

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Aha! Setting LC_ALL, not just LANG, did the trick. It found another error too, curiously, and failures on Python 2.7. Sigh.

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Re-use metadata loading functionality from Provider. Fixes #469.

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Released as 18.7.1.

Original comment by The-Compiler (Bitbucket: The-Compiler, GitHub: The-Compiler):

---

Thanks for the quick fix!