Develop Reference

Why doesn't virtualenv contain a module for pip when it creates a python environment for me?

I might have multiple problems here but they both deal with virtualenv and python3 (3.6.13).
Problem 1:
I had installed virtualenv==20.14.1.
However, it's complaining that there's a version conflict for some reason.
$ python3 -V
Python 3.6.13
$ virtualenv -h
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 573, in _build_master
ws.require(__requires__)
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 891, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 782, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (virtualenv 20.14.1 (/home/mylogin/.local/lib/python3.6/site-packages), Requirement.parse('virtualenv==15.0.1'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/virtualenv", line 5, in <module>
from pkg_resources import load_entry_point
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 3266, in <module>
#_call_aside
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 3241, in _call_aside
f(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 3279, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 575, in _build_master
return cls._build_from_requirements(__requires__)
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 588, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 777, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'virtualenv==15.0.1' distribution was not found and is required by the application
$
Problem 2:
I downgraded virtualenv to be 15.0.1 like it said it needed but then it complains that there's a missing module called dataclasses. For some reason, my co-workers don't have this problem but we can't figure out what is different between my setup and their setup.
$ virtualenv mydir -p python3.6
Running virtualenv with interpreter /usr/bin/python3.6
Using base prefix '/usr'
New python executable in /tmp/mydir/bin/python3.6
Also creating executable in /tmp/mydir/bin/python
Installing setuptools, pip, wheel...done.
/tmp/mydir$ source bin/activate
(mydir) myhost:/tmp/mydir$ python -V
Python 3.6.13
(mydir) myhost:/tmp/mydir$ python -m pip -V
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmp/mydir/lib/python3.6/site-packages/pip/__main__.py", line 29, in <module>
from pip._internal.cli.main import main as _main
File "/tmp/mydir/lib/python3.6/site-packages/pip/_internal/cli/main.py", line 9, in <module>
from pip._internal.cli.autocompletion import autocomplete
File "/tmp/mydir/lib/python3.6/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
from pip._internal.cli.main_parser import create_main_parser
File "/tmp/mydir/lib/python3.6/site-packages/pip/_internal/cli/main_parser.py", line 8, in <module>
from pip._internal.cli import cmdoptions
File "/tmp/mydir/lib/python3.6/site-packages/pip/_internal/cli/cmdoptions.py", line 23, in <module>
from pip._internal.cli.parser import ConfigOptionParser
File "/tmp/mydir/lib/python3.6/site-packages/pip/_internal/cli/parser.py", line 12, in <module>
from pip._internal.configuration import Configuration, ConfigurationError
File "/tmp/mydir/lib/python3.6/site-packages/pip/_internal/configuration.py", line 20, in <module>
from pip._internal.exceptions import (
File "/tmp/mydir/lib/python3.6/site-packages/pip/_internal/exceptions.py", line 14, in <module>
from pip._vendor.rich.console import Console, ConsoleOptions, RenderResult
File "/tmp/mydir/lib/python3.6/site-packages/pip/_vendor/rich/console.py", line 7, in <module>
from dataclasses import dataclass, field
ModuleNotFoundError: No module named 'dataclasses'
This seems to be a chicken and the egg problem. I would run pip to install this missing module but pip is the program that is complaining about this missing module so I can't install it. Like I said above, none of my co-workers I asked know what the problem is.
For debugging purposes, I installed python 2.7.12 and virtualenv==20.14.1 to see what would happen. Apparently things work fine with python 2.7.
$ pip freeze | grep virtual
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
virtualenv==20.14.1
virtualenv-clone==0.5.3
virtualenvwrapper==4.8.4
myhost:/tmp$ virtualenv zzz -p python
Running virtualenv with interpreter /usr/bin/python
New python executable in /tmp/zzz/bin/python
Installing setuptools, pip, wheel...done.
myhost:/tmp$ cd zzz
myhost:/tmp/zzz$ source bin/activate
(zzz) myhost:/tmp/zzz$ python -V
Python 2.7.12
(zzz) myhost:/tmp/zzz$ pip -V
pip 20.3.4 from /tmp/zzz/local/lib/python2.7/site-packages/pip (python 2.7)
(zzz) myhost:/tmp/zzz$ pip freeze
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
(zzz) myhost:/tmp/zzz$
So the questions are
Why do I need to use virtualenv 15.0.1? Why can't I use a newer version of virtualenv?
How come python3 is complaining about the missing dataclasses module but python2 is ok?

Like mentioned in this pypi docs; https://pypi.org/project/dataclasses/ dataclasses only exist in 3.7 and above, which i would recommend to use instead, otherwise, you could also download this package in hopes that it fixes your problem. Are you limited to a python version?

I want to convert Python to exe that imported tensorflow

There was a problem with exe conversion using the pyinstaller.
Traceback (most recent call last):
File "Find.py", line 1, in <module>
File "c:\users\hsk15\python_project\javaproject\venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "lib\site-packages\tensorflow\__init__.py", line 98, in <module>
File "c:\users\hsk15\python_project\javaproject\venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "lib\site-packages\tensorflow_core\__init__.py", line 40, in <module>
File "lib\site-packages\tensorflow\__init__.py", line 50, in __getattr__
File "lib\site-packages\tensorflow\__init__.py", line 44, in _load
File "importlib\__init__.py", line 126, in import_module
File "c:\users\hsk15\python_project\javaproject\venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 627, in exec_module
exec(bytecode, module.__dict__)
File "lib\site-packages\tensorflow_core\python\__init__.py", line 49, in <module>
ImportError: cannot import name 'pywrap_tensorflow'
[20188] Failed to execute script Find
this error.. i can find problem
why can not import pywrap_tensorflow? and what is pywrap_tensorflow?

Pywrap tensorflow is a wrapper for TensorFlow SWIG-generated bindings.
How to use pyinstaller:
First, install the pyinstaller using command prompt by executing
pip install pyinstaller:
Run this line in cmd:
pip install auto-py-to-exe
To open the application run this line in cmd:
auto-py-to-exe
There are few main options you need to choose:
Pick your .py file
Pick "One Directory" or "One File" option
Pick additional files
Then, open the application run this line in cmd, enter the command
pyinstaller filename.py replace the filename with your python code file.

First; create a virtual environment. To do this, you should run this on cmd:
pip install virtualenv
And then, run this:
python -m venv VIRTUAL_ENVIRONMENT'S_NAME
VIRTUAL_ENVIRONMENT'S_NAME/Scripts/activate
pip install pyinstaller
And then, download all the libraries that you use. For example:
pip install requests
pip install datetime
Move your code file to:
VIRTUAL_ENVIRONMENT'S_NAME/Scripts
Now, you can convert it using:
pyinstaller --onefile PYTHON_FILE'S_NAME

broken easy_install and pip after upgrading to OS X Mavericks

Upgraded to OS X 10.9 Mavericks and installed XCode, Command Line Tools, XQuartz, etc. Trying to run a pip install now, but it says that the distribution is not found:
Traceback (most recent call last):
File "/usr/local/bin/pip", line 5, in <module>
from pkg_resources import load_entry_point
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 2603, in <module>
working_set.require(__requires__)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 666, in require
needed = self.resolve(parse_requirements(requirements))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 565, in resolve
raise DistributionNotFound(req) # XXX put more info here
pkg_resources.DistributionNotFound: pip==1.4.1
So I tried to install pip with an easy_install. Turns out that's borked too:
Traceback (most recent call last):
File "/usr/local/bin/easy_install", line 5, in <module>
from pkg_resources import load_entry_point
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 2607, in <module>
parse_requirements(__requires__), Environment()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 565, in resolve
raise DistributionNotFound(req) # XXX put more info here
pkg_resources.DistributionNotFound: setuptools==1.1.6
So some of the other threads say to reinstall setuptools with a sudo python ez_setup.py. It seems to work fine:
Installed /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-1.1.6-py2.7.egg
Processing dependencies for setuptools==1.1.6
Finished processing dependencies for setuptools==1.1.6
But when running the easy_install pip, the same pkg_resources.DistributionNotFound: setuptools==1.1.6 error occurs. Anyone else have this problem? Any ideas how to fix this?

Install easy_install:
Download ez_setup.py module from https://pypi.python.org/pypi/setuptools
$ cd path/to/ez_setup.py
$ python ez_setup.py
Install pip:
$ sudo easy_install pip

try sudo python -m easy_install pip

I ran into a similar problem with git-review.
$ git review -s
Traceback (most recent call last):
File "/usr/local/bin/git-review", line 11, in <module>
sys.exit(main())
File "/Library/Python/2.7/site-packages/git_review/cmd.py", line 1132, in main
(os.path.split(sys.argv[0])[-1], get_version()))
File "/Library/Python/2.7/site-packages/git_review/cmd.py", line 180, in get_version
provider = pkg_resources.get_provider(requirement)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 197, in get_provider
return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 666, in require
needed = self.resolve(parse_requirements(requirements))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 565, in resolve
raise DistributionNotFound(req) # XXX put more info here
pkg_resources.DistributionNotFound: git-review
The git-review team said it was a bug with pkg_resources that could be fixed with
sudo pip install --upgrade setuptools
This worked fine for me.

Why can't I install the openstack nova client on OS X?

I am attempting to install the openstack nova client on my Mac (10.4.8)
nova = https://github.com/openstack/python-novaclient#command-line-api
python --version
Python 2.7.2
I successfully got nova installed (after installing pip)
When I run the client, I get the following error
foo#bar-macbook-pro:~$ nova
Traceback (most recent call last):
File "/usr/local/bin/nova", line 6, in <module>
from novaclient.shell import main
File "/Library/Python/2.7/site-packages/novaclient/__init__.py", line 15, in <module>
import pbr.version
ImportError: No module named pbr.version
In my research, I have found conflicting information about pbr, some say it is required for nova, while others say it isn't required for nova.
https://github.com/rackspace/pyrax/issues/121
When I attempt to install pbr, I see the following error.
foo#bar-macbook-pro:~$ sudo python ~/Downloads/pbr/setup.py install
Traceback (most recent call last):
File "setup.py", line 22, in <module>
**util.cfg_to_args())
File "/Volumes/WDBlack750/spencerowen/Downloads/pbr/pbr/util.py", line 241, in cfg_to_args
pbr.hooks.setup_hook(config)
File "/Volumes/WDBlack750/spencerowen/Downloads/pbr/pbr/hooks/__init__.py", line 27, in setup_hook
metadata_config.run()
File "/Volumes/WDBlack750/spencerowen/Downloads/pbr/pbr/hooks/base.py", line 29, in run
self.hook()
File "/Volumes/WDBlack750/spencerowen/Downloads/pbr/pbr/hooks/metadata.py", line 28, in hook
self.config['name'], self.config.get('version', None))
File "/Volumes/WDBlack750/spencerowen/Downloads/pbr/pbr/packaging.py", line 817, in get_version
version = _get_version_from_git(pre_version)
File "/Volumes/WDBlack750/spencerowen/Downloads/pbr/pbr/packaging.py", line 776, in _get_version_from_git
"git --git-dir=\"" + git_dir + "\" describe --always").replace(
File "/Volumes/WDBlack750/spencerowen/Downloads/pbr/pbr/packaging.py", line 220, in _run_shell_command
stderr=err_location)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1228, in _execute_child
raise child_exception
TypeError: must be encoded string without NULL bytes, not str
Is there anything apparent that would explain why I can not get the library installed?
Surely I must not be the first person to try and install nova on OS X.

Over a year later, I finally got this working on OS X Yosemite
sudo pip install python-novaclient
I did not have to install pbr.

I did the install based on venv:
virtualenv venv_name
source venv_name/bin/activate
pip install python-novaclient fabric
In my case, I had a mixup in which python I was using by way of fabric being installed globally.
Prior: rf -rf all my virtualenvs, rf -rf all references to novaclient (locally, and globally), and deleted a global install of fabric which was calling the novaclient.
Also as a precaution, I do not install pip globally, and only use it without sudo in virtualenvs.

Installing Fabric On Windows (Error No Module Called Readline)

I'm trying to use the Fabric 0.1.1 deploy tool (http://docs.fabfile.org/) on Windows and we're running into an issue with the readline module. I've been through various threads but can't seem to solve the issue. It's important because we can't deploy applications from Windows based machines.
C:\Documents and Settings\dev\Desktop\deploy>fab
Traceback (most recent call last):
File "C:\python\Scripts\fab-script.py", line 8, in <module>
load_entry_point('fabric==0.1.1', 'console_scripts', 'fab')()
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\pkg_resources.py"
, line 277, in load_entry_point
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\pkg_resources.py"
, line 2180, in load_entry_point
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\pkg_resources.py"
, line 1913, in load
File "build\bdist.win32\egg\fabric.py", line 25, in <module>
**ImportError: No module named readline**
Installing the module results in:
**easy_install readline**
Searching for readline
Reading http://pypi.python.org/simple/readline/
Reading http://www.python.org/
Best match: readline 2.6.4
Downloading http://pypi.python.org/packages/source/r/readline/readline-2.6.4.tar
.gz#md5=7568e8b78f383443ba57c9afec6f4285
Processing readline-2.6.4.tar.gz
Running readline-2.6.4\setup.py -q bdist_egg --dist-dir c:\docume~1\ji81b9~1.che
\locals~1\temp\easy_install-pzkz1a\readline-2.6.4\egg-dist-tmp-szs2ps
Traceback (most recent call last):
File "C:\python\Scripts\easy_install-script.py", line 8, in <module>
load_entry_point('setuptools==0.6c9', 'console_scripts', 'easy_install')()
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 1671, in main
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 1659, in with_ei_usage
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 1675, in <lambda>
File "c:\python\lib\distutils\core.py", line 152, in setup
dist.run_commands()
File "c:\python\lib\distutils\dist.py", line 975, in run_commands
self.run_command(cmd)
File "c:\python\lib\distutils\dist.py", line 995, in run_command
cmd_obj.run()
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 211, in run
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 446, in easy_install
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 476, in install_item
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 655, in install_eggs
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 930, in build_and_install
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\comman
d\easy_install.py", line 919, in run_setup
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\sandbo
x.py", line 27, in run_setup
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\sandbo
x.py", line 63, in run
File "c:\python\lib\site-packages\setuptools-0.6c9-py2.6.egg\setuptools\sandbo
x.py", line 29, in <lambda>
File "setup.py", line 93, in <module>
AttributeError: 'module' object has no attribute 'symlink'
Has anybody solved this issue or can anybody suggest a workaround?

The readline module you are trying to install with easy_install is for OS X, not windows. There are Windows-compatible replacements for readline out there but perhaps you should first try updating fabric itself to a more current version (0.9 is out there now).
Grepping the source of the 0.9 version does not find any dependencies on readline.

Following these steps exactly worked for me:
1) Installed using the MSI installer for x86 from here.
2) Installed in the default C:\Python27 directory.
3) Create a new directory: C:\Python27\Scripts
4) Added C:\Python27 and C:\Python27\Scripts to the system path:
5) Download the distribute_setup.py from here into C:\Python27\Scripts
6) Open a command line, navigate to C:\Python27\Scripts, run: 'python distribute_setup.py'
7) now run 'easy_install pip'
8) now run 'pip install fabric'
9) You should get an error saying PyCrypto couldn't install. You can download the pre-build Windows binary from here. Run this to install PyCrypto.
10) run 'pip install fabric' again and it should say everything is installed.
11) in a different directory (let's say c:\dev\hello) create a fabfile.py and add the following code:
def hello(name="world"):
print("Hello %s!" % name)
12) cd to this directory and run 'fab hello:working'. You should see output say
Hello working!
Done.

Give this readline a try. It is a module for Windows that allows additional features in IPython that aren't native and might work with what you are trying to do.

0.1.1 is an older version, I believe. I have no problem installing Fabric on Windows with ActivePython (w/ PyPM):
C:\> pypm install fabric
Ready to perform these actions:
The following packages will be installed:
fabric-0.9.0 pycrypto-2.0.1
Get: [pypm.activestate.com] fabric 0.9.0-1
Get: [pypm.activestate.com] pycrypto 2.0.1-1
Installing fabric-0.9.0
Fixing script C:\Users\sridharr\AppData\Roaming\Python\Scripts\fab-script.py
Installing pycrypto-2.0.1

Download and run easy_install installer for your python version from http://pypi.python.org/pypi/setuptools#downloads . ie: setuptools-0.6c11.win32-py2.6.exe
On the command prompt, lauch easy_install -U fabric to install the last fabric release.

Readline should be available with Cygwin, if you want to move your entire stack in that direction.