Use raw sys.path technique by default

[jaraco]

Setuptools 18.3 added support for omitting the sys path manipulation in easy-install.pth by setting SETUPTOOLS_SYS_PATH_TECHNIQUE=raw.

This change seems to have been well received and even revealed benefits in #400, despite the apparent undesirable behavior encountered in #447.

It's time for Setuptools to use this technique by default. This backward-incompatible change will require environments to set SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite to retain the existing behavior.

[jaraco]

@minrk, I'm mentioning you here, as this work is the follow-up to pull request 135.

[minrk]

Awesome, thanks @jaraco!

[fungi]

As suggested in the changelog, I'm reporting this change as a regression. It seems to break pip editable (-e) installs if there's already a non-editable install of the same package. To reproduce:

1. create a virtualenv and make sure it contains setuptools 25.0.0 or later
2. pip install something from PyPI
3. pip install -e a local source tree of the same software

You'll end up with the package you installed in step 2 rather than the editable version from step 3. (Note that if you use pip install without -e things still work properly and you end up with the version from step 3 instead.)

For example, the old (working) behavior is demonstrated by:

git clone git://git.openstack.org/openstack/openstackdocstheme.git
virtualenv foo
./foo/bin/pip install setuptools==24.3.1
./foo/bin/pip install openstackdocstheme==1.5.0
./foo/bin/pip install -e ./openstackdocstheme
./foo/bin/pip list|grep openstackdocstheme

You should see something like openstackdocstheme 1.5.1.dev1 referencing a path to the source directory you just installed.

Now blow away the foo virtualenv and repeat except use setuptools==25.0.0 (or any later version). You'll see that instead the result is that you still have openstackdocstheme 1.5.0 referencing the foo virtualenv's site-packages directory instead of the editable install.

[fungi]

Sorry, now I see the issue we've encountered is basically a dupe of #447 and #729, with a fix (pypa/pip#3904) coming in an eventual pip update.

[rbtcollins]

Its a shame the fix wasn't released when setuptools released :/.

[dhood]

As per the release notes, I am raising concern that support for the rewrite technique may be removed one day.

In our project it's common to have two versions of the same package on the PYTHONPATH. This is because our project does not install to the system, it is typically on the PYTHONPATH, and if a user wants to make any modifications to one of our tools, they will clone into an overlay workspace which is ahead of it on the PYTHONPATH.

In general this workflow is unaffected, but if the overlay workspace uses develop, for setuptools>=25.0.0 we rely on SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite to achieve the desired behaviour (preference to the package in the overlay). This looks to be tracked in #447.

If #447 gets addressed we may not rely on the rewrite technique, but in the meantime we do (unless you have suggestions for other approaches that do not use this environment variable that we can try).