spacer

classification
Title: Windows installer should add Python and Scripts directories to the PATH environment variable
Type: enhancement Stage: committed/rejected
Components: Installation, Windows Versions: Python 3.3
process
Status: open Resolution: fixed
Dependencies: Superseder:
Assigned To: brian.curtin Nosy List: AaronR, Jim.Jewett, brian.curtin, christian.heimes, eric.araujo, giampaolo.rodola, helder-magalhaes, jasonspiro, jdigital, jonathan.hartley, lambacck, loewis, ncoghlan, pekka.klarck, pitrou, python-dev
Priority: normal Keywords: needs review, patch

Created on 2008-08-15 17:16 by jasonspiro, last changed 2012-07-04 21:50 by jasonspiro.

Files
File name Uploaded Description Edit
issue3561.diff brian.curtin, 2012-04-07 15:45
issue3561_v2.diff brian.curtin, 2012-04-18 04:48 review
issue3561_v3.diff brian.curtin, 2012-04-18 22:43 review
news_dialog.diff brian.curtin, 2012-05-25 00:11 review
path_option.diff brian.curtin, 2012-06-23 03:55 review
Messages (47)
msg71172 - (view) Author: Jason Spiro (jasonspiro) Date: 2008-08-15 17:16
The Python Windows installer[1] should automatically add the Python and 
Scripts directories to the PATH environment variable.  (If you like, 
you can also provide a checkbox in the installer GUI that users can 
uncheck if they don't want this behavior.)

This issue was discussed at nabble.com/Why-does-Python-never-add-
itself-to-the-Windows-path--td8044465.html and the majority consensus 
was that this is a good idea, but nobody has volunteered to implement 
it.

^ [1].  The installer is generated by the code at 
svn.python.org/view/python/trunk/Tools/msi/
msg71174 - (view) Author: Martin v. Löwis (loewis) * spacer Date: 2008-08-15 18:03
I still don't think such a change should be made, hence I reject this
report as "won't fix". Discussion shouldn't start again on this matter
until there is an actual patch to review.
msg71185 - (view) Author: Jason Spiro (jasonspiro) Date: 2008-08-15 21:22
Martin, at the time I read the python-list thread, I didn't pay any 
attention to the posts' authors.  Only now did I realize you were one 
of the posters.  Oops.

I already know the basic ideas about creating MSIs with Wise from a 
past job.  So now I researched and thought about this particular 
problem for several hours.  I've learned that Windows Installer is able 
to add ...\python and ...\python\scripts to the PATH during 
installation.  It cannot[1] remove them at uninstallation.  [ WiX, and 
its superior competitor MAKEMSI, each provide slightly higher-level 
abstractions[2][3] on top of what Windows Installer provides[4] to make 
this slightly easier, but not by that much.  But I will assume you 
don't plan to spend days or weeks of your spare time on migrating away 
from msilib. :-) ]

Here are the rough notes I've made up so far on how to do this:

-  make a new method add_environment.  In it:
    -  call start_component to create components "modify_path_per_user" 
and "modify_path_per_machine"
    -  call add_data to create an Environment table.  It should have 
two rows[5]:
        -  Name:"=PATH" Value:"[TARGETDIR];[TARGETDIR]\Scripts;[~]" 
Component:"modify_path_per_user"
        -  Name:"=*PATH" Value:"[TARGETDIR];[TARGETDIR]\Scripts;[~]" 
Component:"modify_path_per_machine"

Another difficult part is the UI.  Then there's the issue of switching 
which of the two components are installed based on whether it's per-
user or per-machine and also based on whether the user specifies via 
the UI that they want their PATH changed.  I have to think more about 
that, and I'm already tired of researching.

Remember that Windows Installer cannot undo its PATH changes at 
uninstall time.  So, before I consider proceeding further, let me 
verify a few things with you.

1.  [TARGETDIR] will stay on the path.  I think that is fine, since the 
python.exe will be gone, so will never be executed.  Do you agree?

2.  [TARGETDIR]\scripts will also stay on the path.  And it may still 
contain scripts installed by the user or by third-party installers like 
the SCons installer.  I don't know enough about how Python works to 
know if that's a problem.  Is it a problem?

P.S.  Would you prefer to discuss this by something more synchronous 
like telephone (I will pay the tolls) or instant messaging?

P.P.S.  Now that I have realized how complicated Python installation 
actually is, and how hard it is to design the tables of and write 
raw .MSI files, I have a new appreciation for the work you've done on 
making a Python MSI installer.  Thank you very much for having done 
so.  Also, now that I have started researching how much work is 
necessary to get this done, I realize why you don't want to code it 
yourself.  :-)

I don't know if I will end up actually managing to come up with a patch.

^ [1].  I have inferred this fact based on 
www.isg.ee.ethz.ch/tools/realmen/det/msi.en.html -- scroll down 
to the "Setting the PATH" section
^ [2].  blogs.technet.com/alexshev/archive/2008/03/28/from-msi-
to-wix-part-13-installable-items-environment-variable.aspx
^ [3].  makemsi-manual.dennisbareis.com/path.htm
^ [4].  msdn.microsoft.com/en-us/library/aa368369(printer).aspx
^ [5].  Search inside the page msdn.microsoft.com/en-
us/library/aa368369(printer).aspx for "If the package can be installed 
per-user or per-machine" to see why you need two rows.
msg71190 - (view) Author: Martin v. Löwis (loewis) * spacer Date: 2008-08-15 23:01
> 1.  [TARGETDIR] will stay on the path.  I think that is fine, since the 
> python.exe will be gone, so will never be executed.  Do you agree?

Completely disagree. Adding something to PATH is nearly unacceptable
clutter even during installation, and completely unacceptable (to me)
after uninstallation. If you install Python several times, will the path
get longer and longer?

In principal, uninstallation should completely undo installation. There
should be only plausible exceptions, e.g. when the user still has files
in a directory, deleting the directory is unacceptable as it would also
delete the files. The big problem where the Python installer (and MSI
in general) fails is uninstallation of file associations, which doesn't
restore the file assocations to what they were (of course, that software
might be gone)

> 2.  [TARGETDIR]\scripts will also stay on the path.  And it may still 
> contain scripts installed by the user or by third-party installers like 
> the SCons installer.  I don't know enough about how Python works to 
> know if that's a problem.  Is it a problem?

To me, any change to PATH is a problem... (I really think that software
installation should never ever touch it - this aspect of the operating
system completely belongs to the user resp. the system administrator)

> P.S.  Would you prefer to discuss this by something more synchronous 
> like telephone (I will pay the tolls) or instant messaging?

I don't think this can be done for 2.6, so there is no need for
synchrony - there is plenty of time to come up with a solution
for 2.7/3.1. In any case, I'll be away for the next three weeks.

> It cannot[1] remove them at uninstallation.
> ^ [1].  I have inferred this fact based on 
> www.isg.ee.ethz.ch/tools/realmen/det/msi.en.html -- scroll down 
> to the "Setting the PATH" section

I think this is incorrect. You cannot uninstall the path through
unsetting the PATH variable, sure. So perhaps using the Environment
table is the wrong approach in the first place. You can do nearly
anything in a custom action, so it should be possible to remove the
path entry on uninstallation by means of a custom action.

If such a solution was designed, there would still be many questions,
such as:
- what is the actual problem being solved? Is it real? Could there
  be other solutions to that problem (such as installing things into
  system32, or somewhere else that is on the PATH already)?
- if the path is modified, should the Python directory be added to
  the beginning or the end?
- IMO, this feature needs to be customizable, and IMO, it must be
  turned off by default. So how should such customization be offered?
These questions can partially be discussed in the tracker item proposing
the patch. The very end-user related aspects of it need to be discussed
on python-dev (perhaps taking a poll whether this is desirable, whether
it should be optional, and if so, what the default should be)
msg134326 - (view) Author: Nick Coghlan (ncoghlan) * spacer Date: 2011-04-24 09:43
Reopening this issue since #9228 was closed as a duplicate of this one. Given the significant level of user demand for this behaviour, it should NOT be closed again until a PEP has been written and either accepted or rejected. If such a PEP is not written and championed through to a final decision, then this issue should remain open indefinitely.

Such a PEP should canvas the option of installing development shell launch scripts into the Start menu and updating the documentation accordingly as an alternative to messing with the system-wide PATH setting.

An install-time option to allow power users to disable the PATH manipulation should also be considered. The interests of novice Python users and experienced Windows system administrators are not in alignment on this topic, and the current installer behaviour favours the latter at the expense of the former.
msg134328 - (view) Author: Nick Coghlan (ncoghlan) * spacer Date: 2011-04-24 10:07
Now, another factor to consider is that Windows 7 makes manipulating the system PATH even more difficult to do correctly (e.g. see www.symantec.com/connect/forums/wise-7-win-7-problems-updating-environment-variable-current-user).

I believe ActiveState handle this by making the PATH modification optional and having it off by default (I found docs for ActivePerl stating this explicitly, but no equivalent for ActivePython).

Regardless, for a product intended for heavy command line use, we should definitely do better in either providing a way for users to request that the installer modify PATH directly, or else a way to easily launch a command shell with PATH updated appropriately.
msg134333 - (view) Author: Sridhar Ratnakumar (srid) Date: 2011-04-24 16:59
> I believe ActiveState handle this by making the PATH modification 
> optional and having it off by default (I found docs for ActivePerl 
> stating this explicitly, but no equivalent for ActivePython).

ActivePython 2.x has it on by default.
msg152706 - (view) Author: Helder Magalhães (helder-magalhaes) Date: 2012-02-05 20:40
See tightly related ActivePython issue [1] regarding the way directories are added to the PATH environment variable.

Also, here's qooxdoo (a well-known JavaScript framework) giving out instructions [2] on how to use the official python distribution due to the missing path and file association treatment (and also redirecting the preference to ActivePython for that).

(Setting to all versions as I believe this isn't addressed in any previous or current release.)

[1] bugs.activestate.com/show_bug.cgi?id=92615
[2] manual.qooxdoo.org/current/pages/getting_started/troubleshooting.html#windows
msg152707 - (view) Author: Brian Curtin (brian.curtin) * spacer Date: 2012-02-05 20:43
FWIW I have an installer built which optionally adds to the path. It's not complete - still needs some GUI work to hook it all up, but I'll be proposing it shortly.
msg152709 - (view) Author: Martin v. Löwis (loewis) * spacer Date: 2012-02-05 21:11
Helder: please don't change tracker settings unless you know what they are for. Python 2.6 and 3.1 can't possibly see any change here since they are in security fix mode, and can't see any fixes affecting this issue. Likewise, 2.7 and 3.2 will only see bug fixes. As the lack of supporting changes to Path isn't a bug (but possibly a missing feature), these versions can't see the bug, either. Putting 3.4 and 3.3 into an issue is just nonsense at this point: if the issue gets resolved for 3.3 before 3.4 is released, it will never be an issue for 3.4.
msg152711 - (view) Author: Helder Magalhães (helder-magalhaes) Date: 2012-02-05 22:02
@Brian: glad to know about it. It will surely help, as many Windows users aren't much comfortable with the console already, and even less comfortable for tweaking the system at this level (apart from permission issues, etc.). ;-)

@Martin: you're right, sorry. I'd swear I several versions selected and only a couple missing, but probably I was tricked by a related/duplicate issue. Better read the manual next time... :-)
msg153761 - (view) Author: Brian Curtin (brian.curtin) * spacer Date: 2012-02-20 02:34
Attached is an in-progress patch.

I still need to figure out how to hook up the GUI to condition the path manipulation. A few locations have said to do conditional environment modification via the Component table, but I haven't been able to get some placeholder in there based on whether or not PREPEND is true/false from the GUI.

Right now if you build an installer with this it will always modify the path even if you haven't chosen the option to do so.

briancurtin.com/python-dev/python-3.3.15390.msi is an example installer. Note that it's not a signed installer, it's unofficial, and I don't support it whatsoever. It's merely provided as a sample of what the GUI looks like, and you can install it, see what happens, then uninstall and see that the path modification is properly undone.
msg157124 - (view) Author: Jeff Dean (jdigital) Date: 2012-03-30 03:03
I just saw Brian Curtin's Pycon 2012 presentation.  If a goal is to make it easy for new users to run python, consider installing a desktop shortcut.  This would make it very easy for new users (easier than starting up a shell).

This is independent of the Path changes discussed here.  Those should be made (or not made) on their own merit, since most users will eventually need the Path to be properly adjusted.
msg157735 - (view) Author: Brian Curtin (brian.curtin) * spacer Date: 2012-04-07 15:45
Attached is issue3561.diff which adds a path option, off by default, as a feature to be installed. I've tested installation and un-installation with the feature both installed and not installed and it seems to work fine for me.

briancurtin.com/python-dev/python-3.3.15437.msi is an installer built with this patch. briancurtin.com/python-dev/CustomizePage.png is simply a screenshot of the page where you choose to enable this feature.
msg157892 - (view) Author: Jim Jewett (Jim.Jewett) Date: 2012-04-09 20:26
@Brian -- to clarify, 

(1)  Does issue3561.diff completely supersede prependpath_in-progress.diff?  (And should that be the one currently subject to review?)

(2)  What happens with multiple installations?  Do users have to manually unset the path to avoid surprises?  Does this become obsolete if the py.exe shim delegating to the appropriate py* is included with 3.3?  Does installing 3.3.2 in over top of 3.3.1 add the directory to the path twice?
msg157893 - (view) Author: Antoine Pitrou (pitrou) * spacer Date: 2012-04-09 20:36
UI-wise, I'm not sure why it looks like an installable component rather than a separate checkbox. Is it a limitation of the installation software?
msg157895 - (view) Author: Brian Curtin (brian.curtin) * spacer Date: 2012-04-09 20:45
I unlinked the old diff. issue3561.diff is the one that matters.

As for what happens with multiple installations, it's no different than how you'd already be managing it or anything else like it. If you install 2.7 with the path option enabled and then you install 3.2 with the path option enabled, 3.2 goes in front of 2.7. The installations don't touch each other. If we want to get smart and detect other installations on the Path, I believe this gets a lot more complicated.

It does not become obsolete with the launcher. The launcher solves a wider problem in a different way. You could just switch to the launcher if you wanted to, but because we (probably will) have that feature doesn't mean the bare python.exe can't grow this functionality. People have been asking for it for years. It's the first step of every tutorial on the internet. It takes up a special page on the information for a sprint I just saw.

As for the question of 3.3.2 over 3.3.1: I'm not sure yet. I'll build a few installers with different versions and run some upgrades to see what happens. We're using MSI's builtin ability to manage everything here, so I would imagine it knows what to do, but I'll confirm.
msg157897 - (view) Author: Brian Curtin (brian.curtin) * spacer Date: 2012-04-09 20:47
> UI-wise, I'm not sure why it looks like an installable component rather than a separate checkbox. Is it a limitation of the installation software?

I originally did it as a separate check box UI-wise but couldn't hook that into be an actual "Feature" in MSI terms. The way it's currently done allows it to be added to certain tables, allowing us to conditionally modify the "Environment" table which contains the optional path addition.

I am by no means an MSI expert. This is just the way Martin and I talked about it at PyCon and the way I've seen it done around the web. If there's a way to tie a checkbox to the Feature table, I would like that.
msg157909 - (view) Author: Martin v. Löwis (loewis) * spacer Date: 2012-04-09 21:52
> UI-wise, I'm not sure why it looks like an installable component  
> rather than a separate checkbox. Is it a limitation of the  
> installation software?

You are misinterpreting the UI. The list is not of "installable components",
but of "features". Adjusting the path is a feature just as registering the
.py extension. That feature can be installed and uninstalled.
msg157911 - (view) Author: Martin v. Löwis (loewis) * spacer Date: 2012-04-09 22:05
IANANSOE (I am not a native speaker of English), but it seems to me that "Prepend path" is a bit terse, in particular since "Path" is being prepended *to*. How about "Add python.exe to the search path"? That it is added to the beginning could be elaborated (i.e. prepended) in the feature description.
msg157921 - (view) Author: Brian Curtin (brian.curtin) * spacer Date: 2012-04-09 22:59
Agreed. I will work up a more friendly text to go along with the feature.
msg158588 - (view) Author: Brian Curtin (brian.curtin) * spacer Date: 2012-04-18 04:48
Here's a patch with better wording, and here's a screenshot of what the feature selection looks like with that text: i.imgur.com/k7e12.png
msg158672 - (view) Author: Éric Araujo (eric.araujo) * spacer Date: 2012-04-18 22:08
A minor thing: The capitalization of the feature names is inconsistent.
msg158680 - (view) Author: Brian Curtin (brian.curtin) * spacer Date: 2012-04-18 22:43
The attached patch changes the feature text to "Add python.exe to Path". I'm not sure the word "search" adds much there anyway.

An additional change here that I think would be beneficial is a better description text, immediately covering the benefit of this feature. I had to do some minor resizing of the relevant text boxes, but the description is now...

Prepend C:\Python33 to the system Path variable. This allows you to type 'python' into a command prompt without needing the full path.
msg159259 - (view) Author: Martin v. Löwis (loewis) * spacer Date: 2012-04-25 06:58
Brian: The patch is fine, please apply.
msg159280 - (view) Author: Roundup Robot (python-dev) Date: 2012-04-25 13:13
New changeset 4e9f1017355f by Brian Curtin in branch 'default':
Fix #3561. Add an option to place the Python installation into the Windows Path environment variable.
hg.python.org/cpython/rev/4e9f1017355f
msg159283 - (view) Author: Brian Curtin (brian.curtin) * spacer Date: 2012-04-25 13:26
Now that the feature is in, I'm going to track the few places we need to document it in #14668.
msg159288 - (view) Author: Chris Lambacher (lambacck) Date: 2012-04-25 14:35
I am really happy to see this as an option in the Windows installer. This has a potential to really reduce the support burden on training new Windows users to use Python and will really help normalize the experience for new users between Windows and POSIX platforms.

Unfortunately, from what I can tell, this is OFF by default. I think that is a mistake. The default for something like this is really important because without new users being explicitly told to set it, new users will not. Most new Python users are just going to take the default values and still be confused by not being able to open the console and run python as in the instructions for various new user tutorials (i.e. web frameworks, scientific computing, etc).

I understand the issue with installing multiple Python versions, but generally speaking people are going to want to install the latest version and have that be on their path. My sense is that there is a relative minority of Windows users that care about multiple versions of Python would not want their path to be updated. Unfortunately the ones who don't want their path updated are the ones that have driven the installer development at this point and will be most vocal in the Python community. 

I suggest a compromise which makes adding to the PATH default to ON if another Python version is not installed and/or on the PATH and default to OFF otherwise.

It looks like you can use a condition table plus the feature table to check for Python in the path buy I am not exactly an expert with MSI. My quick check found:
msdn.microsoft.com/en-us/library/aa368012(VS.85).aspx which says you can do a search for an installed component and then set a property. This should then allow setting of the default state of the feature based on a condition.

I really recommend that while we still have a chance to make a change that we think about the goal of this feature and whether making it disabled by default achieves those goals.
msg159289 - (view) Author: Martin v. Löwis (loewis) * spacer Date: 2012-04-25 14:50
lambacck: I'm -1, but I'm willing to yield to anybody who wants to be "in charge" of this feature (i.e. Brian, or the release manager). I'm not willing yield to "mere" user requests, as regular users won't have to deal with negative consequences that enabling this by default may have.

I'm quite opposed to your proposed conditional approach; I think this will be highly confusing to users.
msg159295 - (view) Author: Chris Lambacher (lambacck) Date: 2012-04-25 15:11
The reason for the conditional approach was to attempt to account for the "negative consequences" of adding enabling this by default. i.e. if you are already a Python developer and install a new version, it will be status quo, but if you are a new Python developer then you will be able to run instructions for packages that work perfectly fine on Windows but are maintained and documented by POSIX users and don't understand that the steps to make things work on Windows are different.

I don't care one way or the other about the compromise position but I strongly believe that if someone is installing Python on Windows for the first time they should get a good experience without having to know of the existence of docs like: docs.python-guide.org/en/latest/starting/install/win/

This tweet https://twitter.com/#!/zedshaw/status/194853198006198272 is characteristic of the sentiment that non-Windows users have about training Windows users to develop with Python. At the PyCon web summit several people who have been doing training lately have said that their solution is to get windows users to run a Linux Virtual Machine because the Windows Python experience is so bad for new Windows developers. This is one relatively easy place where we can make that experience better.
msg159298 - (view) Author: Antoine Pitrou (pitrou) * spacer Date: 2012-04-25 15:20
> Unfortunately, from what I can tell, this is OFF by default. I think
> that is a mistake. The default for something like this is really
> important because without new users being explicitly told to set it,
> new users will not. Most new Python users are just going to take the
> default values and still be confused by not being able to open the
> console and run python as in the instructions for various new user
> tutorials (i.e. web frameworks, scientific computing, etc).

To put things in perspective, the default under POSIX ("make install")
is to make the installed version the default (by copying it
into /usr/bin/python3). To change that behaviour, you have to use "make
altinstall" explicitly.
msg159305 - (view) Author: Éric Araujo (eric.araujo) * spacer Date: 2012-04-25 15:42
[Jeff Dean]
> If a goal is to make it easy for new users to run python, consider installing a desktop shortcut.
> This would make it very easy for new users (easier than starting up a shell).
> This is independent of the Path changes discussed here.

Hi Jeff; thanks for the suggestion.  Could you open another bug report for it?  It’s much more manageable to have one request per report.  Thanks in advance.
msg161528 - (view) Author: Pekka Klärck (pekka.klarck) Date: 2012-05-24 21:00
I found about this enhancement via Python Insider blog post [1] that also asked adding comments to this issue. Here are mine:

1) Great to see that this is finally done!

2) Is only Python installation directory added into PATH? Why not also Scripts directory under it as originally requested in this issue? As a developer of a Python based test automation tool, just adding Python into PATH doesn't help us because our tool's launcher scripts still aren't found unless PATH is manually edited.

3) I would also prefer this to be on by default.

[1] blog.python.org/2012/05/recent-windows-changes-in-python-33.html
msg161530 - (view) Author: Brian Curtin (brian.curtin) * spacer Date: 2012-05-24 21:28
2. Yes


gipoco.com is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.