Develop Reference

python: why is user script directory shared across python versions?

From PEP-370:
user script directory
A directory for binaries and scripts. [10] It's shared across Python versions and the destination directory for scripts.
Unix (including Mac)
~/.local/bin
Windows
%APPDATA%/Python/Scripts
Why does it propose version-specific user site dirs but not user script dirs? Wouldn't scripts coming from different python versions conflict each other then?
Edit. Yes, they would. I did a test with python2-pytest and python3-pytest. When installing both into user dir with pip, one pytest script overwrote another without a warning.
There seems to be a relevant link but it's dead:
Discussion about the bin directory http://permalink.gmane.org/gmane.comp.python.devel/91095

The shell doesn't support per-Python-version binaries. There is a single namespace for command-line executables, the first name found in the directories listed on PATH is used.
The point of the ~/.local/bin directory here is that it is added to the PATH environment variable, and that scripts and other executables are put there for command line use. And because there is only a single namespace for such executables, there is no point in putting commands into per-version directories here.
Instead, it is up to the project to provide you with versioned executables. The pip project uses setup.py configuration to pip, pipX and pipX.Y scripts when installed with Python X.Y, and so you'll always have a more specific version of the script when there are multiple Python versions. And you can also use the module as a script with pythonX.Y -m pip. Lots of Python command-line tools have similar support.
As for the lost GMane link (still available on the web archive); there are other archives of the Python-dev discussion still available, such as this grokbase.com rendering of the same post; that discussion was about what directory to place the scripts in, ~/bin or ~/.local/bin, and was never about per-Python-version directories.

Copy Python App to a new Machine?

I have a Python app running on windows that has imports for the following packages:
requests
json
psycopg2
I copy the entire project (I used Pycharms to write the app and import the packages) to a new machine and expected it would work. The new machine is also windows and I'm trying to run my script from the command line (i.e. no Pycharm on the new machine).
Instead, I get an error saying "ModuleNotFoundError: No module named 'requests'"
If I look at the project, I have the directories:
venv
Lib
site-packages
requests
What am I missing/doing wrong?

You have a couple of options here but first the problem. You are exporting your code base to a new machine without the required Modules installed on that machine and/or within your Python project's environment. After you have python installed on your new machine, you need to be sure to point your PyCharm Project to the proper environment.
File > Default Preferences > Project Interpreter
The window that appears on the right will contain a drop down menu labeled Project Interpreter. If you click on the drop down, it should reveal a list of the available Python environments on your machine.
Based on your description of your site-packages directory I would assume you do not have your interpreter pointed the proper environment on your new machine. With that said, you would be better served creating a new virtual python environment on your machine and installing each relevant dependency within that environment.
Take a look at this post here for your first best option on re-creating your old python environment on your new machine.
EDIT: I apologize for not reading the question more thoroughly before answering the questions. If this is running on a Windows machine you will need to double check the environment path python is using. It is very easy to install python at a different PATH than the command line environment is checking on a Windows box. If for example your PATH is pointing to a different version of Python and PIP is installing packages somewhere else this issue can occur. Double check your System PATH for python and which version the command line is running.

On the new machine you must source venv/bin/activate so your path environment variables are set properly. In particular, which python should say venv/bin/python rather than /usr/bin/python. Also, take care to conda env update or pip install -r requirements.txt so you'll have suitable venv libraries on the new machine.

Associating a python project with a virtual environment

been searching for this with no success, i don't know if i am missing something but i have a virtualenv already but how do i create a project to associate the virtualenv with, thanks
P.S. Am on windows

I could be wrong here, but I do not believe that a virtualenv is something that is by its very nature something that you associate with a project. When you use a virtualenv, you're basically saying, "I'm taking this Python interpreter, installing what I want on it, and setting it aside from the Python interpreter that the entire computer uses by default." Virtualenv does not have a concept of a Python "project"; it is just a custom version of a Python interpreter that you run code through. There are tools in IDEs like PyCharm that enable you to associate a project with a virtualenv, but those are another layer on top of the base software.
In order to use a virtualenv with a project, you will need to "activate" it every time you wish to use it. The documentation for activating a virtualenv on Windows is found here.
EDIT:
Saw that you had virtualenvwrapper tagged in your post, so I did a bit of hunting on that. It would appear that there is the mkproject command, which creates a project folder and then associates it with a virtualenv interpreter. Documentation on it can be found here.

Requirements:
Virtual Env
Pycharm
Go to Virtual env and type which python
Add remote project interpreter (File > Default Settings > Project Interpreter (cog) add remote)
You'll need to set up your file system so that PyCharm can also open the project.
NOTE:
do not turn off your virtual environment without saving your run configurations that will cause pycharm to see your run configurations as corrupt
There's a button on the top right that reads share enable this and your run configs will be saved to a .idea file and you'll have a lot less issues

If you already have your virtualenv installed you just need to start using it.
Create your projects virtual environment using virtualenv env_name on cmd. To associate a specific version of python with your environment use: virtualenv env_name -p pythonx.x;
Activate your environment by navigating into its Scripts folder and executing activate.
Your terminal now is using your virtual environment, that means every python package you install and the python version you run will be the ones you configured inside your env.
I like to create environments with the names similar to my projects, I always use one environment to each project, that helps keeping track of which packages my specific projects need to run.
If you haven't read much about venvs yet, try googling about requirements.txt along with pip freeze command those are pretty useful to keep track of your project's packages.

I like Pipenv: Python Dev Workflow for Humans to manage environments:
Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. Windows is a first-class citizen, in our world.
It automatically creates and manages a virtualenv for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages. It also generates the ever-important Pipfile.lock, which is used to produce deterministic builds.
Pipenv is primarily meant to provide users and developers of applications with an easy method to setup a working environment.

Package a command line application for distribution?

I am currently writing a command line application in Python, which needs to be made available to end users in such a way that it is very easy to download and run. For those on Windows, who may not have Python (2.7) installed, I intend to use PyInstaller to generate a self-contained Windows executable. Users will then be able to simply download "myapp.exe" and run myapp.exe [ARGUMENTS].
I would also like to provide a (smaller) download for users (on various platforms) who already have Python installed. One option is to put all of my code into a single .py file, "myapp.py" (beginning with #! /usr/bin/env python), and make this available. This could be downloaded, then run using myapp.py [ARGUMENTS] or python myapp.py [ARGUMENTS]. However, restricting my application to a single .py file has several downsides, including limiting my ability to organize the code and making it difficult to use third-party dependencies.
Instead I would like to distribute the contents of several files of my own code, plus some (pure Python) dependencies. Are there any tools which can package all of this into a single file, which can easily be downloaded and run using an existing Python installation?
Edit: Note that I need these applications to be easy for end users to run. They are not likely to have pip installed, nor anything else which is outside the Python core. Using PyInstaller, I can generate a file which these users can download from the web and run with one command (or, if there are no arguments, simply by double-clicking). Is there a way to achieve this ease-of-use without using PyInstaller (i.e. without redundantly bundling the Python runtime)?

I don't like the single file idea because it becomes a maintenance burden. I would explore an approach like the one below.
I've become a big fan of Python's virtual environments because it allows you to silo your application dependencies from the OS's installation. Imagine a scenario where the application you are currently looking to distribute uses a Python package requests v1.0. Some time later you create another application you want to distribute that uses requests v2.3. You may end up with version conflicts on a system where you want to install both applications side-by-side. Virtual environments solve this problem as each application would have its own package location.
Creating a virtual environment is easy. Once you have virtualenv installed, it's simply a matter of running, for example, virtualenv /opt/application/env. Now you have an isolated python environment for your application. Additionally, virtual environments are very easy to clean up, simply remove the env directory and you're done.
You'll need a setup.py file to install your application into the environment. Say your application uses requests v2.3.0, your custom code is in a package called acme, and your script is called phone_home. Your directory structure looks like this:
acme/
__init__.py
models.py
actions.py
scripts/
phone_home
setup.py
The setup.py would look something like this:
from distutils.core import setup
install_requires = [
'requests==2.3.0',
]
setup(name='phone_home',
version='0.0.1',
description='Sample application to phone home',
author='John Doe',
author_email='john#doe.com',
packages=['acme'],
scripts=['scripts/phone_home'],
url='http://acme.com/phone_home',
install_requires=install_requires,
)
You can now make a tarball out of your project and host it however you wish (your own web server, S3, etc.):
tar cvzf phone_home-0.0.1.tar.gz .
Finally, you can use pip to install your package into the virtual environment you created:
/opt/application/env/bin/pip install http://acme.com/phone_home-0.0.1.tar.gz
You can then run phone_home with:
/opt/application/env/bin/phone_home
Or create a symlink in /usr/local/bin to simply call the script using phone_home:
ln -s /opt/application/env/bin/phone_home /usr/local/bin/phone_home
All of the steps above can be put in a shell script, which would make the process a single-command install.
And with slight modification this approach works really well for development environments; i.e. using pip to install / reference your development directory: pip install -e . where . refers to the current directory and you should be in your project directory alongside setup.py.
Hope this helps!

You could use pip as suggested in the comments. You need to create a MANIFEST.in and setup.py in your project to make it installable. You can also add modules as prerequisites. More info can be found in this question (not specific to Django):
How do I package a python application to make it pip-installable?
This will make your module available in Python. You can then have users run a file that runs your module, by either python path/run.py, ./path/run.py (with +x permission) or python -c "some code here" (e.g. for an alias).
You can even have users install from a git public reporitory, like this
pip install git+https://bitbucket.org/yourname/projectname.git
...in which case they also need git.

How do I uninstall python from web platform installer?

I've installed python and some other packages using web platform installer, but I was having some issues getting a Django project to work so I uninstalled everything and am trying to get it going from scratch. Web Platform Installer still shows that I have 'Windows Azure SDK for Python" and "Python 2.7 (32-bit)" installed however and I can't mark them as uninstalled.
I don't see where to uninstall from WPI at all, I uninstalled them using control panel. I think I had originally installed python from the python site and had version 3.3 and 2.7 (64-bit), but now there are no entries containing 'python' when I try to uninstall a program from the control panel.
Does anyone know what is going on or can I download the setups from somewhere and try them manually? Is there a way to reset what shows as installed in WPI? I tried uninstalling and reinstalling WPI but that didn't help.

For what it's worth, I just deleted the folder containing the installed PHP versions(5.3,5.4,5.5), which for me was \Program Files (x86)\IIS Express\PHP. Also, I removed "\Program Files (x86)\iis express\php\5.3" from the search path.
When I return to the web installer the 'Add' buttons are enabled.
I opened the options, set the Web Server to IIS, then installed PHP v5.5. It was installed into \Program Files (x86)\PHP\v5.5 and added to the search path.

This is how IIS recommends doing it:
http://forums.iis.net/t/1178803.aspx
Open %userprofile%\documents\iisexpress\config\applicationhost.config file and
1. Find following entry (or similar entry) in applicationhost.config file and comment it or delete it.
Find following entry in hanlders section and comment this as well or delete.
3.By default Web Platform Installer installs PHP to %programfiles%\iis express\php\. so open %programfiles%\iis express\php\ folder and delete the php version folder that you no longer need (don't forget to remove relavant entries from applicationhost.config as mentioned in step 1 and 2 above)
Of course you would need to find the python one instead of php but it is the jist of it

I found a PowerShell script in the WPI directory that checked for python installs and I had to delete the registry keys specified in it.
Let me add some context:
Open the below path,
%LOCALAPPDATA%\Microsoft\Web Platform Installer\installers\PythonDetectInstalls
in the sub-folder of the above path, there is a PowerShell script "DetectPythonInstalls.ps1" which contains two script lines for checking if Python is installed:
$regKey = "hklm:SOFTWARE\Python\PythonCore\$pythonVersion\InstallPath";
$correctRegKey = "hklm:SOFTWARE\Wow6432Node\Python\PythonCore\$pythonVersion\InstallPath";
Uninstall all Python versions you do not neet. However, we need to remove some registry keys manually using "regedit".
(Safety Note: Please take a backup of the registry keys before removing the above-mentioned registry key)
Ref: Social.Tecnet