Develop Reference

How to move all modules to new version of Python (from 3.6 to 3.7)

I just upgraded to python 3.7 and I realized that all my modules stuck with the previous version. Even Django is not recognised anymore. How can I do to transfer everything to the new version? I am a little lost right now, don't even know where the new version has been installed.
Edit:
When I do $ which python3.6 the terminal tells me it doesn't exist, but I have a python3.6 directory in /usr/local/lib/, where all modules are installed.
In the same directory /usr/local/lib/ I also have a python3.7 directory with some modules installed but many are missing. However when I search for the file python3.7 in my finder it doesn't appear. when I do $ which python3.7 the path is /usr/local/bin so not the same path as the directory.
Anyone sees what happened and knows how I can transfer all modules to python3.7?

Even if the old python version has been removed, it is possible to use the pip of the current python version with the --path option to list all the modules installed in the previous version.
For example, migrating all my user installed python modules from 3.7 to 3.8
pip freeze --path ~/.local/lib/python3.7/site-packages > requirements.txt
pip install --user -r requirements.txt
Incidentally, I always use pip install with --user and leave the system wide installations to the package manager of my linux distro.

It is safer to re-install all packages due to possible compatibility issues:
pip3.6 list | awk '{print $1}' | xargs -I{} pip3.7 install {}

in older version of Python --run the command
pip freeze > requirements.txt
download and install newer version on python.. change the PATH variable to the new version
and run the command
pip install -r requirements.txt

I'm not sure about all modules...but if you want to install a module specifically in python3.7, try this:
python3.7 -m pip install *module_name*

In some cases, we don't have the opportunity to pip freeze in old version--because I've already updated and old version have been purged! There are some measures I've taken to recover some of the packages but I'm NOT sure every package would work with this fix.(e.g. the packages built with wheels)
mv /your/path/to/python3.{6,7}/site-packages/
If the case is packages installed outside venv (in /usr/local/ or ~/.local), reinstall pip with get-pip.py, just to be safe.
If you are recovering a virtualenv. Activate your virtualenv and use my script
Most of your packages should work by now. If anything malfunctions, pip reinstall would works. If you still want it 100% works, pip freeze now.😉

I have an alternative
(Not sure if works outside Windows 10)
I'm currently migrating from 3.7 to 3.8 and the way I found to re-install my previous libraries was by using a script I had that updates all packages via pip install. (Assuming you installed your new Python version as your main version) This checks for all the packages I had and updates/install them in the new Python version.
Note: I prefer to run the script from the command line
Use the file explorer to go to the folder where you have the script;
Click on the path box, write "cmd" and press enter to open a command line from the folder where you are;
Write "python name_of_your_script.py" and press enter to run the command.
The script (adapted from this solution):
import pkg_resources
from subprocess import call
packages = [dist.project_name for dist in pkg_resources.working_set]
[call("pip install " + name + " --upgrade") for name in packages]

I faced a similar problem, now that I upgraded from python 3.7 to python 3.8 (new)
I installed Python 3.8, but the system kept the python37 subfolder with the already installed packages(...\Python37-32\Lib\site-packages) even with the Pyhton38 subfolder created, with the new python.exe.
Usually, it's possible to keep on using the old libraries on your new Python version, because the existent libraries installation folder are already registered in your local computer system path (*).
Even though, I've had problems to use some libraries (some worked in Jupyter Notebook but not in Spyder). I tried the alternatives others proposed to migrate libraries but it did not worked (maybe I did not
So I used brutal force solution.. Not elegant at all, but it worked:
remove the OLD python version folders from the system path or even remove the folder itself for good..
Folders: C:\Users\USERNAME\AppData\Roaming\Python\Python37
C:\Users\USERNAME\AppData\Local\Programs\Python\Python37
Reinstall the packages you need, preferably via Anaconda prompt.
python -mpip install library_name
OR
pip install --user --force-reinstall package_name
OR
pip install --user --force-reinstall package_name == specify_package_version
The libraries will be installed at c:\users\USERNAME\anaconda3\lib\site-packages and recognized by your new python version.
(*) to add the folder to the PATH: System properties --> environment variables --> click "Path"--> edit --> add folder name)

Unable to install MySQL driver for Python [duplicate]

First off, yeah, I've already seen this:
pip install mysql-python fails with EnvironmentError: mysql_config not found
The problem
I am trying to use Django on a Google App Engine project. However, I haven't been able to get started as the server fails to start properly due to:
ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb
I did some research and it all pointed to having to install Mysql-python, as apparently it isn't on my system. I actually tried uninstalling it and got this:
Cannot uninstall requirement mysql-python, not installed
Whenever I actually do try to install via:
sudo pip install MySQL-python
I get an error stating:
raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found
I've already tried running:
export PATH=$PATH:/usr/local/mysql/bin
but that didn't seem to help, as I ran the installation command again and it still failed.
Any ideas?
Please note I'm not in a virtualenv.

Ok, well, first of all, let me check if I am on the same page as you:
You installed python
You did brew install mysql
You did export PATH=$PATH:/usr/local/mysql/bin
And finally, you did pip install MySQL-Python (or pip3 install mysqlclient if using python 3)
If you did all those steps in the same order, and you still got an error, read on to the end, if, however, you did not follow these exact steps try, following them from the very beginning.
So, you followed the steps, and you're still geting an error, well, there are a few things you could try:
Try running which mysql_config from bash. It probably won't be found. That's why the build isn't finding it either. Try running locate mysql_config and see if anything comes back. The path to this binary needs to be either in your shell's $PATH environment variable, or it needs to be explicitly in the setup.py file for the module assuming it's looking in some specific place for that file.
Instead of using MySQL-Python, try using 'mysql-connector-python', it can be installed using pip install mysql-connector-python. More information on this can be found here and here.
Manually find the location of 'mysql/bin', 'mysql_config', and 'MySQL-Python', and add all these to the $PATH environment variable.
If all above steps fail, then you could try installing 'mysql' using MacPorts, in which case the file 'mysql_config' would actually be called 'mysql_config5', and in this case, you would have to do this after installing: export PATH=$PATH:/opt/local/lib/mysql5/bin. You can find more details here.
Note1: I've seen some people saying that installing python-dev and libmysqlclient-dev also helped, however I do not know if these packages are available on Mac OS.
Note2: Also, make sure to try running the commands as root.
I got my answers from (besides my brain) these places (maybe you could have a look at them, to see if it would help): 1, 2, 3, 4.
I hoped I helped, and would be happy to know if any of this worked, or not. Good luck.

I had been debugging this problem forever - 3 hours 17 mins. What particularly annoyed me was that I already had sql installed on my system through prior uni work but pip/pip3 wasn't recognising it. These threads above and many other I scoured the internet for were helpful in eluminating the problem but didn't actually solve things.
ANSWER
Pip is looking for mysql binaries in the Homebrew Directory which is located relative to Macintosh HD #
/usr/local/Cellar/
so I found that this requires you making a few changes
step 1: Download MySql if not already done so https://dev.mysql.com/downloads/
Step 2: Locate it relative to Macintosh HD and cd
/usr/local/mysql/bin
Step 3: Once there open terminal and use a text editor of choice - I'm a neovim guy myself so I typed (doesn't automatically come with Mac... another story for another day)
nvim mysql_config
Step 4: You will see at approx line 112
# Create options
libs="-L$pkglibdir"
libs="$libs -l "
Change to
# Create options
libs="-L$pkglibdir"
libs="$libs -lmysqlclient -lssl -lcrypto"
*you'll notice that this file has read-only access so if your using vim or neovim
:w !sudo tee %
Step 5: Head to the home directory and edit the .bash_profile file
cd ~
Then
nvim .bash_profile
and add
export PATH="/usr/local/mysql/bin:$PATH"
to the file then save
Step 6: relative to Macintosh HD locate paths and add to it
cd /private/etc/
then
nvim paths
and add
/usr/local/mysql/bin
*you'll again notice that this file has read-only access so if your using vim or neovim
:w !sudo tee %
then
cd ~
then refresh the terminal with your changes by running
source .bash_profile
Finally
pip3 install mysqlclient
And Viola. Remember it's a vibe.

If you don't want to install full mysql, we can fix this by just installing mysql-client
brew install mysql-client
Once cmd is completed it will ask to add below line to ~/.bash_profile:
echo 'export PATH="/usr/local/opt/mysql-client/bin:$PATH"' >> ~/.bash_profile
Close terminal and start new terminal and proceed with pip install mysqlclient.

I am running Python 3.6 on MacOS Catalina. My issue was that I tried to install mysqlclient==1.4.2.post1 and it keeps throwing mysql_config not found error.
This is the steps I took to solve the issue.
Install mysql-connector-c using brew (if you have mysql already install unlink first brew unlink mysql) - brew install mysql-connector-c
Open mysql_config and edit the file around line 112
# Create options
libs="-L$pkglibdir"
libs="$libs -lmysqlclient -lssl -lcrypto"
brew info openssl - this will give you more information on what needs to be done about putting openssl in PATH
in relation to step 3, you need to do this to put openssl in PATH - echo 'export PATH="/usr/local/opt/openssl/bin:$PATH"' >> ~/.bash_profile
for compilers to find openssl - export LDFLAGS="-L/usr/local/opt/openssl/lib"
for compilers to find openssl - export CPPFLAGS="-I/usr/local/opt/openssl/include"

Also this happens when I was installing mysqlclient,
$ pip install mysqlclient
As user3429036 said,
$ brew install mysql

If you have installed mysql using Homebrew by specifying a version then mysql_config would be present here. - /usr/local/Cellar/mysql#5.6/5.6.47/bin
you can find the path of the sql bin by using ls command in /usr/local/ directory
/usr/local/Cellar/mysql#5.6/5.6.47/bin
Add the path to bash profile like this.
nano ~/.bash_profile
export PATH="/usr/local/Cellar/mysql#5.6/5.6.47/bin:$PATH"

This answer is for MacOS users who did not install from brew but rather from the official .dmg/.pkg. That installer fails to edit your PATH, causing things to break out of the box:
All MySQL commands like mysql, mysqladmin, mysql_config, etc cannot be found, and as a result:
the "MySQL Preference Pane" fails to appear in System Preferences, and
you cannot install any API that communicates with MySQL, including mysqlclient
What you have to do is appending the MySQL bin folder (typically /usr/local/mysql/bin in your PATH by adding this line in your ~/.bash_profile file:
export PATH="/usr/local/mysql/bin/:$PATH"
You should then reload your ~/.bash_profile for the change to take effect in your current Terminal session:
source ~/.bash_profile
Before installing mysqlclient, however, you need to accept the XcodeBuild license:
sudo xcodebuild -license
Follow their directions to sign away your family, after which you should be able to install mysqlclient without issue:
pip install mysqlclient
After installing that, you must do one more thing to fix a runtime bug that ships with MySQL (Dynamic Library libmysqlclient.dylib not found), by adding this line to your system dynamic libraries path:
export DYLD_LIBRARY_PATH=/usr/local/mysql/lib/:$DYLD_LIBRARY_PATH

brew install mysql added mysql to /usr/local/Cellar/..., so I needed to add :/usr/local/Cellar/ to my $PATH and then which mysql_config worked!

The problem in my case was that I was running the command inside a python virtual environment and it didn't had the path to /usr/local/mysql/bin though I have put it in the .bash_profile file. Just exporting the path in the virtual env worked for me.
For your info sql_config resides inside bin directory.

Install brew or apt-get is also not easy for me so I downloaded mysql via: https://dev.mysql.com/downloads/connector/python/, installed it. So I can find mysql_config int this directory: /usr/local/mysql/bin
the next step is:
export PATH=$PATH:/usr/local/mysql/bin
pip install MySQL-python==1.2.5

Python pip: install in specific non-user directory and ignore system wide packages

I want to set up a Python environment for a whole team and I don't have root access to the server.
I have done a similar thing with Perl and expected to be able to do this for Python in a similar way but I keep running into a problem.
Basically, I want to be able to install a package into /SOME/DIR on the system while ignoring any system wide versions of that package.
However, when I run
pip install --install-option="--prefix=/SOME/DIR/" --up --ignore-installed SOME-MODULE
I keep getting a "permission denied" error because pip keeps trying to remove system-wide packages when upgrading.
What does work is this
pip install --user --up --ignore-installed SOME-MODULE
Which does not try to touch the system-wide packages but it installs the module into a directory in $HOME/.lib, which is not what I need.
It seems impossible to combine --user and a "--prefix" option, so it sems that I can either install into an arbitrary path but then get conflicts with already install system-wide packages or install into my home directory. Neither of them are what I need.
For now I have been using the --user option and then moved the installed files across to /SOME/DIR which works but seems odd.
Am I missing something? I have read up on virtualenv but this also doesn't quite sound like what I need. Thanks for your help!

Note that --install-options is passed directly to the packages setup.py install command - this requires the installation directory to be in your python path.
add it to your PYTHONPATH i.e.
set -gx PYTHONPATH $PYTHONPATH '/home/user/temp/lib/python3.4/site-packages'
and run pip
pip install django==1.6 --ignore-installed --install-options="--prefix=/home/user/temp"
Mostly this is a pain in the ass if you have to do this for each library (note that you will still have potential conflicts with imports if you want to use certain standard libraries from the default site-packages dir, and others from your custom dir). And the best choice is probably, as the comment says, to install virtualenv and virtualenvwrapper

I had a similar problem (permission denied + no root access), --build option made it work: pip install --install-option="--prefix=/path/to/local/lib" --build=/tmp wget

pip install on OS X Mavericks fails with error: /usr/local/bin/pip: No such file or directory

I am running the Python 2.7.5 version that came standard with OSX.
I have tried to install pip using sudo easy_install pip AND python get-pip.py
When I try to install pip I get the following error with easy_install:
Processing pip
Running setup.py -q bdist_egg --dist-dir /Users/XXXX/pip/egg-dist-tmp-VtRwku
warning: no files found matching 'pip/cacert.pem'
warning: no files found matching '*.html' under directory 'docs'
warning: no previously-included files matching '*.rst' found under directory 'docs/_build'
no previously-included directories found matching 'docs/_build/_sources'
no previously-included directories found matching 'tasks'
pip 1.6.dev1 is already the active version in easy-install.pth
Installing pip script to /usr/local/bin
error: /usr/local/bin/pip: No such file or directory
When I try to install pip with get-pip.py
Requirement already up-to-date: pip in /Library/Python/2.7/site-packages/pip-1.6.dev1-py2.7.egg
Nothing shows when I type pip help

# mirra.p
the problem is probably you have two versions of python 2 installed on your PC in different locations and pip of one of the copies of python got removed somehow. so when you try to install the pip for a copy, system says that there is a pip already installed in another location and also the /usr/local/bin folde is also missing the pip file used to installed pip.
already installed pip file's location is this:
Library/Python/2.7/site-packages/pip-1.6.dev1-py2.7.egg
i just solved this issue so i will try to explain it in a way so that beginners also understand it properly
copy the already installed pip file to the location where it is missing instead of installing it through commands as command will detect the already installed pip.
the solution is very simple:
run these commands in a terminal (command prompt) in Ubuntu
to get the location of the pip file already installed
which pip
cp <location provided by which pip> <location where to copy pip file>
cp /Library/Python/2.7/site-packages/pip /usr/local/bin
i used the locations for mirra.p problem.
hope it helps.

The error message and especially the additional diagnostics in comments mentioning Cellar indicate that you have installed another Python version using Homebrew.
Probably simply renaming /usr/local/bin/pip to something else will fix the immediate problem. I'll advise against removing it entirely, at least until you understand what precisely is wrong.
In more detail; to use the system Python, you have to undo whatever Homebrew did when it installed a second Python. For a start, you need to unset all the variables which brew installed (or made you install) in your .bashrc and other locations previously.
If you have orphaned symbolic links in /usr/local/bin which point to locations where no command is installed any longer, you'll want to simply remove those (needs root access). This is a sign of a dirty uninstall; next time, follow the Homebrew instructions for how to remove it and the packages you installed with it.
In more detail, Howebrew will have done something like
ln -s /usr/local/Cellar/something/something/python-2.7.3 /usr/local/bin/python
and if /usr/local/Cellar/something/something now no longer exists, attempting to write something there through the symlink will obviously fail. This seems to be exactly what the pip installer is attempting to do at the very end of the installation.
If you have removed Homebrew permanently, you can edit .bashrc and revert whatever changes Homebrew required there. Some settings may remain active in your already-running shell instances until the next reboot; if you want to fix those, read on.
To temporarily override settings, you can unset your PYTHONPATH to get back the system default behavior. If you have commands in /usr/local/bin shadowing the system default ones, you can temporarily set your PATH to prefer /usr/bin by simply prepending it (this is a temporary hack only);
PATH=/usr/bin:$PATH
The shell commands are suitable for Bash; if you use another shell, adapt accordingly.

If you are getting error: /usr/local/bin/pip: No such file or directory and pip already is installed, do following things to solve this problem:
Run which pip. It will give answer like :/usr/local/bin
just copy pip file in desired location
here,
cp /usr/local/bin /usr/local/bin/pip
This solved my problem.

beginner installing nosetests package

I'm trying to install the following as per the learnpythonthehardway tutorial:
pip from http://pypi.python.org/pypi/pip
distribute from http://pypi.python.org/pypi/distribute
nose from http://pypi.python.org/pypi/nose/
virtualenv from http://pypi.python.org/pypi/virtualenv
I've visited these links and clicked the download button; each file is in my downloads folder now, and I unarchive/unzipped them- not sure what this means but it seemed required. Are they "installed"? If not, what does it mean to really install them? I've tried typing nosetests in the terminal (as the book says you should), as well as tried easy_install but that doesn't seem to work. It appears my understanding is limited in a number of ways here.
I get the following -bash: nosetests: command not found, but am trying to get:
nosetests . ---------------------------------------------------------------------- Ran 1 test in 0.007s
OK

This works on mac, it may work on linux
1)Open terminal (Be prepared to enter your password)
2)Type: sudo easy_install pip
3)Type: sudo pip install distribute
4)Type: sudo pip install nose
5)Type: sudo pip install virtualenv
Hope that helps, cheers!

They are not yet installed.
Each has its own vagaries about how exactly the install process works and in fact some of those packages will include the other packages with them. I.e.
"If you use virtualenv, a copy of pip will be automatically be
installed in each virtual environment you create."
http://pypi.python.org/pypi/pip
As a relatively new python user myself, (and having gone thru a similar process not long ago) I am eagerly awaiting the more complete answers that come back for this one.

Find out where the nosetests script is. On OSX:
/usr/local/share/nosetests
Execute directly, or set up a bash alias, perhaps in .bash_profile:
alias nosetests='/usr/local/share/python/nosetests'
alias nosetests3='/usr/local/share/python3/nosetests'

You were on the right track, but the package is actually called "nose". The documentation for nose (nosetests is the script) can be found here:
https://nose.readthedocs.org/en/latest/
The short answer is that you have 2 basic options: easy_install or pip. Install using setuptools/distribute: easy_install nose. Install using pip: pip install nose

Install Python 2.7 and after just do this (only on Windows - Windows PowerShell):
Setup PATH env:
C:\Python27\;C:\Python27\Scripts\", "User"
pip is already installed
3.
.\pip install distribute
4.
.\pip install nose
5.
.\pip install virtualenv