Develop Reference

PyWin32 (226) and virtual environments

[PyPI]: pywin32 226 has been released on 20191110. It works on most of the Python installations (e.g. works on the official versions downloaded from Python), but not on virtual environments (e.g. created with VirtualEnv (v16.7.7), and (based on further research) Python's venv). I used Python v3.8.0 and v3.7.3 as lab rats. Here's the output for the former:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q058805040]> "e:\Work\Dev\VEnvs\py_064_03.08.00_test0\Scripts\python.exe" -c "import win32api"
Fatal Python error: init_import_size: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
File "e:\Work\Dev\VEnvs\py_064_03.08.00_test0\lib\site.py", line 769, in <module>
main()
File "e:\Work\Dev\VEnvs\py_064_03.08.00_test0\lib\site.py", line 746, in main
paths_in_sys = addsitepackages(paths_in_sys)
File "e:\Work\Dev\VEnvs\py_064_03.08.00_test0\lib\site.py", line 279, in addsitepackages
addsitedir(sitedir, known_paths)
File "e:\Work\Dev\VEnvs\py_064_03.08.00_test0\lib\site.py", line 202, in addsitedir
addpackage(sitedir, name, known_paths)
File "e:\Work\Dev\VEnvs\py_064_03.08.00_test0\lib\site.py", line 170, in addpackage
exec(line)
File "<string>", line 1, in <module>
File "e:\Work\Dev\VEnvs\py_064_03.08.00_test0\lib\site-packages\win32\lib\pywin32_bootstrap.py", line 14, in <module>
for maybe in site.getsitepackages():
AttributeError: partially initialized module 'site' has no attribute 'getsitepackages' (most likely due to a circular import)

Note: I branched this answer (and also the question) from [SO]: PyWin32 and Python 3.8.0 (#CristiFati's answer), as it's a different issue. You might want to check that one before going further.
After some digging, it turns out it's a VirtualEnv bug (or at least, that's how I see things, because VirtualEnv's site.py doesn't contain getsitepackages - although it was present in Python's site.py since v2.7). There are several issues (that were) open revolving this absence (e.g. [GitHub]: pypa/virtualenv - site.getsitepackages() missing), but they appear to be closed without a fix (many of them due to inactivity). Also, [GitHub]: [WIP] The next-gen virtualenv (rewrite) which is a big refactor, doesn't seem to address it.
Recap: this error (in VirtualEnv context):
Also applies to other Python versions (I can confirm for v3.7.3 (32bit))
Happens every time when the interpreter starts
In conclusion, do not install PyWin32 226 in (VirtualEnv) virtual environments, as they will end up in a broken state!
PyWin32 (official .whls) "compatibility table" (didn't check Anaconda (or other such tools)):
"Normal" (official) Python installations:
v226 is OK
VirtualEnvs (and also Python's standard venvs):
v225 is OK
For Python 3.8.0, [GitHub]: CristiFati/Prebuilt-Binaries - (master) Prebuilt-Binaries/PyWin32/v225 should be used
Submitted [GitHub]: mhammond/pywin32 - Workaround for virtual environments (VirtualEnv) (merged on 20191114). Applying the changes locally (check the referenced question for details on how to do it), fixed it:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q058805040]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]> ".\venv_py_064_030800\Scripts\python.exe" -m pip list
Package Version
---------- -------
pip 19.3.1
pywin32 226
setuptools 41.6.0
wheel 0.33.6
[prompt]> ".\venv_py_064_030800\Scripts\python.exe" -c "import win32api"
[prompt]>
Update #0
[PyPI]: pywin32 227 (which addresses this issue), was published on 20191114!

librosa.load: file not found error on loading a file

I am trying to use librosa to analyze .wav files. I started with creating a list which stores the names of all the .wav files it detected.
data_dir = '/Users/raghav/Desktop/FSU/summer research'
audio_file = glob(data_dir + '/*.wav')
I can see the names of all the files in the list 'audio_file'. But when I load any of the audio file, it gives me file not found error.
audio, sfreq = lr.load(audio_file[0])
error output:
Traceback (most recent call last):
File "read_audio.py", line 10, in <module>
audio, sfreq = lr.load(audio_file[1])
File "/usr/local/lib/python3.7/site-packages/librosa/core/audio.py", line 119, in load
with audioread.audio_open(os.path.realpath(path)) as input_file:
File "/usr/local/lib/python3.7/site-packages/audioread/__init__.py", line 107, in audio_open
backends = available_backends()
File "/usr/local/lib/python3.7/site-packages/audioread/__init__.py", line 86, in available_backends
if ffdec.available():
File "/usr/local/lib/python3.7/site-packages/audioread/ffdec.py", line 108, in available
creationflags=PROC_FLAGS,
File "/usr/local/lib/python3.7/site-packages/audioread/ffdec.py", line 94, in popen_multiple
return subprocess.Popen(cmd, *args, **kwargs)
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1522, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'avconv': 'avconv'

Two things:
It looks like you are using Homebrew
avconv is not in your path
Assuming, you have never installed it, you should be able to solve this simply by installing it. I.e. run:
$ brew install libav
(see here)
If avconv is already installed, you probably need to look into your PATH environment and check whether it is in the path.
That said, using a system-wide Python as installed by Homebrew is a bad idea, because it does not quickly let you change Python versions and dependency sets. It all becomes one big mess within weeks.
One (among multiple) solutions for this is to use miniconda. It quickly lets you activate Python interpreters with defined dependency sets.
So to really solve the issue, I'd recommend to install miniconda and create a plain Python 3.6 environment:
$ conda create -n librosa_env python=3.6
Activate the environment:
$ source activate librosa_env
Then add the conda-forge channel (a repository that contains many libraries like librosa):
$ conda config --add channels conda-forge
Then install librosa:
$ conda install librosa
By installing librosa this way, conda should take care of all dependencies, incl. libav.

That error suggests librosa is unable to find FFMPEG executables, of which avconv is an example.
When not converting different samplerates, FFMPEG is often not needed. You can try to specify the sample size to be the original of the audio file during load()

Upgrading audioread to 2.1.8 fixed this issue for me. The bugfix can be inspected here: https://github.com/beetbox/audioread/commit/8c4e236fda38ce1d1f6dafc4715074a790e62849

I had to install FFMPEG to solve the issue.. You may follow along the link https://www.wikihow.com/Install-FFmpeg-on-Windows for the same.

python - pyinstaller "RuntimeWarning: Parent module 'PyInstaller.hooks.hook-PIL' not found while handling absolute import" and "tcl" related errors

I get a warning message while trying to create exectable file using pyinstaller. This warning appeared after installing Pillow. Previously i nevre got any warnings and was able to make it through.
the warning i get by pyinstaller is:
7314 INFO: Analyzing main.py
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/hooks/hook-PIL.Image.py:14: RuntimeWarning: Parent module 'PyInstaller.hooks.hook-PIL' not found while handling absolute import
from PyInstaller.hooks.shared_PIL_Image import *
Also when i tried to run the executable's exe/consol version of my code that lies inside the dist folder created by the pyinstaller (dist/main/main), these are displayed..
Traceback (most recent call last):
File "<string>", line 26, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
exec(bytecode, module.__dict__)
File "/Users/..../build/main/out00-PYZ.pyz/PIL.PngImagePlugin", line 40, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
exec(bytecode, module.__dict__)
File "/Users/..../build/main/out00-PYZ.pyz/PIL.Image", line 53, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
exec(bytecode, module.__dict__)
File "/Users/..../build/main/out00-PYZ.pyz/FixTk", line 74, in <module>
OSError: [Errno 20] Not a directory: '/Users/.../dist/main/tcl'
logout
[Process completed]
so, i tried by uninstalling pillow, installing tk tcl dev version. And then installed pillow. Even that didnt helped.
I also tried reinstalling pyinstaller,. didnt help too
Update 1:
It seems Pyinstaller.hooks.hook-PIL.py file was missing in the Pyinstaller/hooks directory. And it was missing on all platforms(Mac, windows and linux). This is the warning/error message that i get on windows, which is the same i got on mac and on linux.
Later i found a link which said, its just to need Python import machinery happy. so i created as said so. Then i dont get the same error on all platforms, But on mac i still get the PILImagePlugin,Image and FixTk errors

Solution for tcl:
I found what was going wrong,.. Every problem that i faced on OSX was the OS itself(exactly the macport). Python by default comes with the mac OS. And this version of python may be useful for just learning basic python, but is not suitable for Development purpose.
Installing brew's python helped. I followed this SO link. After doing these i was still getting errors. Later i had to change the paths on /etc/paths. Basically rearranging them should work. But still then i wasn't getting it right.
Then i had to change the .bash_profile, which worked for most users, But still i was getting mac's version of python and pip, not the brews version of python.
Finally i had to restart the machine for a couple of times and do the /etc/paths and .bash_profile steps repeatedly to get the system wide effect to accept brews version of python and pip
Solution for PIL:
just adding a file called hook-PIL.py with an empty content would serve the purpose. I found a link which was having the hook files content of pyinstaller.
The location to create
for mac : /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/hooks/ Actually for mac this step wouldn’t be required. When we install python through brew and change the path, everything that you try to install later either through pip install or from source packages tend to choose a different path. And everything will be taken care of.
for windows:C:\Python27\lib\site-packages\PyInstaller-2.1.1.dev0-py2.7.egg\PyInstaller\hooks
**Please check if this is a valid path on your machine before creating the file and then create the file. And im not sure or i don't know if just adding an empty file is the right way. But it worked for me

Python: OSError: cannot load library libcairo.so.2

I'm having some trouble running a python script on my Windows 7 platform. I've installed Python and also cairo, last one using "pip". I'm running the script using this command:
C:\Python34>python.exe label/make_label.py
and I get the following error message:
Traceback (most recent call last):
File "label/make_label.py", line 6, in <module>
import cairocffi as cairo
File "C:\Python34\lib\site-packages\cairocffi\__init__.py", line 41, in <modul
e>
cairo = dlopen(ffi, *CAIRO_NAMES)
File "C:\Python34\lib\site-packages\cairocffi\__init__.py", line 34, in dlopen
return ffi.dlopen(names[0]) # pragma: no cover
File "C:\Python34\lib\site-packages\cffi\api.py", line 118, in dlopen
lib, function_cache = _make_ffi_library(self, name, flags)
File "C:\Python34\lib\site-packages\cffi\api.py", line 411, in _make_ffi_libra
ry
backendlib = _load_backend_lib(backend, libname, flags)
File "C:\Python34\lib\site-packages\cffi\api.py", line 400, in _load_backend_l
ib
return backend.load_library(name, flags)
OSError: cannot load library libcairo.so.2: error 0x7e
What I've already done is the following:
Added the PATH to GTK/bin in the environmental variable
I check the folder GTK/bin and found "libcairo-2.dll" so I renamed it to libcairo.so
I don't know what other information could be useful in solving this but please let me know and I'll try to add it.

On Mac OS X using homebrew:
brew install cairo
brew install pango

It seems cairo depends a shared library which is not in standard search library, however, the python is calling dlopen to dynamic load the library, so you could try to put the libcairo.so.2(if it's a link, then make sure the reference locates at the same folder) in the working directory. You can also try pkg-config to set the environment. see here http://people.freedesktop.org/~dbn/pkg-config-guide.html

I just had the same issue ("OSError: cannot load library libcairo.so.2: error 0x7e"), and this is how I solved the problem on Windows (Windows 7 x64, Python 3.4.2 x86 (MSC v.1600 32 bit)):
downloaded an all-in-one bundle of the GTK+ stack including 3rd-party dependencies (which contains libcairo-2.dll and other Cairo-related libraries)
extracted this archive to a path which does NOT contain spaces (e.g. C:\Programs\gtk+)
added the extracted directory's bin subdirectory (which contains the mentioned libcairo-2.dll and other necessary files) to the PATH
Win+R, SystemPropertiesAdvanced
Environment Variables...
added this directory to the Path variable (either to the user variables or system variables, after a semicolon) (e.g. ...;C:\foo;C:\Programs\gtk+)
OK
pip install cairosvg
tested it with a very simple code, which already had worked:
import cairosvg
testsvg = '<svg height="30" width="30">\
<text y="10">123</text>\
</svg>'
svgConvertedToPng = cairosvg.svg2png(bytestring=testsvg)
print(svgConvertedToPng)

Solved on Windows 10 as follows:
download the headless UniConverter installer
Find where it installed and add its dll subdirectory to the system path.
Close and re-open the command window to get the updated path.

I just fixed this on Mac OSX 10.13 with an Anaconda Python installation and cairosvg:
$ conda install cairo pango gdk-pixbuf libffi cairosvg
$ cairosvg image.svg -o image.png
I got the idea from https://cairosvg.org/documentation/, which says that all its dependencies can be installed with WeasyPrint. WeasyPrint's documentation for installation on MacOSX at https://weasyprint.readthedocs.io/en/latest/install.html#macos says to get the dependencies from HomeBrew:
brew install python3 cairo pango gdk-pixbuf libffi
So I tried it with conda instead and it worked fine.

Python: Installation issues with pygraphviz and graphviz

I see many questions on the difficulties of properly installing pygraphviz and graphviz on Windows for Python 2.7. But no answers that I have found is solving my problem. Here's what I did:
I first installed pygraphviz using unofficial windows binaries
with this link in my anaconda (python) folder (
C:\Users\chamar\AppData\Local\Continuum\Anaconda )
Downloaded graphviz-2.36.msi and installed it under the default
path C:\Program Files (x86)\Graphviz2.36
The command import pygraphviz in Python works. But when I want to use say this function nx.graphviz_layout I get raise ValueError("Program %s not found in path."%prog)
What may cause this problem is that pygraphviz cannot locate the path of graphviz. Now, since I installed pygraphviz using the unofficial windows binary, which file can I modify to link both the library and include for graphviz's path? You would you usually find in the setup.py of pygraphviz the library and include paths when you don't use the unofficial binaries.
UPDATE 1
I added to PATH in Regedit under SOFTWARE a folder GRAPHIZ with a new key (default) with value C:\Program Files (x86)\Graphviz2.36\bin
UPDATE 2
I was having an error in the pydot.py file regarding the difficulty of Python locating the path of Graphviz. I made the changes as follow:
def _graphviz():
if os.sys.platform == 'win32':
path = r"C:/Program Files (x86)/Graphviz2.36/bin/"
progs = __find_executables(path)
return progs
find_graphviz()
{'fdp': 'C:/Program Files (x86)/Graphviz2.36/bin/fdp.exe', 'twopi': 'C:/Program Files (x86)/Graphviz2.36/bin/twopi.exe', 'neato': 'C:/Program Files (x86)/Graphviz2.36/bin/neato.exe', 'dot': 'C:/Program Files (x86)/Graphviz2.36/bin/dot.exe', 'circo': 'C:/Program Files (x86)/Graphviz2.36/bin/circo.exe'}
That seems ok with me but when I execute say:
positions = nx.graphviz_layout(G, prog='twopi', root=0)
I get:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\chamar.stu\AppData\Local\Continuum\Anaconda\lib\site-packages\networkx\drawing\nx_agraph.py", line 229, in graphviz_layout
return pygraphviz_layout(G,prog=prog,root=root,args=args)
File "C:\Users\chamar.stu\AppData\Local\Continuum\Anaconda\lib\site-packages\networkx\drawing\nx_agraph.py", line 264, in pygraphviz_layout
A.layout(prog=prog,args=args)
File "C:\Users\chamar.stu\AppData\Local\Continuum\Anaconda\lib\site-packages\pygraphviz\agraph.py", line 1305, in layout
data=self._run_prog(prog,' '.join([args,"-T",fmt]))
File "C:\Users\chamar.stu\AppData\Local\Continuum\Anaconda\lib\site-packages\pygraphviz\agraph.py", line 1251, in _run_prog
runprog=r'"%s"'%self._get_prog(prog)
File "C:\Users\chamar.stu\AppData\Local\Continuum\Anaconda\lib\site-packages\pygraphviz\agraph.py", line 1239, in _get_prog
raise ValueError("Program %s not found in path."%prog)
ValueError: Program twopi not found in path.
Why?

Here are the steps I followed to get pygraphviz working for Python 3.4 (I think if you follow the analogous steps, it should work for Python 2.x). I'm just documenting it here for future visitors to the page :
Pre-requisites :
wheel (Should be present by default in newer distributions)
The correct Windows build of pygraphviz (unofficial builds). On Win7 x64, I selected "pygraphviz‑$version-cp34‑none‑win_amd64.whl".
(Note the cp34 indicating the python version.)
The Graphviz installer version 2.38 (for which the above wheel is built)
Steps:
Run the Graphviz installer
Add the Graphviz\bin folder to your user or system PATH
Check: Open a command prompt and execute twopi -V. You should be able to see the Graphviz version printed onto the console.
Now go to your Python environment (e.g. by running anaconda.bat, a prompt where you can run python)
Run pip install pygraphviz‑*$version*-cp34‑none‑win_amd64.whl
You're done :) ! Run an example script to see if everything went well.

You'll find loads of install-ready packages on this site: http://www.lfd.uci.edu/~gohlke/pythonlibs/ including the ones you tried to install. I know I'm way too late with the answer but I just became a member.

You may first install "easy_install" (look at
How to use Python's "easy_install" on Windows ... it's not so easy)
then 2 packages are required: 'python-pygraph' & 'libgv-python'.