Develop Reference

pychrm cannot import my own package, but the terminal can

Problem solved.
If you add your path to PYTHONPATH, you can import the packages inside that path. But what I am doing is try to import that path, this is wrong.
So in this case, I make a sub Dir at my path and that dir is now a package can be imported.
Still I have to include my path in file>settings>project structure as source.
==========================================================================
I am using python3.6 in anaconda, Ubuntu16.04.
I have my own package in path /home/gph/pyutils_gph.
Inside this dir are utils.py files. I have include this path in PYTHONPATH.
I can do
Python 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyutils_gph import utils
>>>
in terminal. But the same import code shows error in pycharm. It can find my package, indicating it with red lines.
What else should I do to make pycharm know my own package?
==========================================================================
I opened the terminal inside pycharm, get output like below. I have that dir in PYTHONPATH, but I can not import it. What's wrong?
Python 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/home/gph/pyutils_gph', '/home/gph/Desktop/BorderSecure/detection_cnn/src', '/home/gph/anaconda3/envs/py36_torch0.4/lib/python36.zip', '/home/gph/anaconda3/envs/py36_torch0.4/lib/python3.6', '/home/gph/anaconda3/envs/py36_torch0.4/lib/python3.6/lib-dynload', '/home/gph/anaconda3/envs/py36_torch0.4/lib/python3.6/site-packages']
>>> from pyutils_gph import utils
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pyutils_gph'
>>>

If you have /home/gph/pyutils_gph in PYTHONPATH and you do from pyutils_gph import utils then it is looking for
"/home/gph/pyutils_gph/" + "pyutils_gph/utils.py`
because from pyutils_gph import utils means pyutils_gph/utils.py and it adds it to every path from PYTHONPATH
You have to add to PYTHONPATH
/home/gph
and then it will give
"/home/gph/" + "pyutils_gph/utils.py`
so you get correct path
First version can works in terminal if you run Anaconda in folder /home/gph because Python search packages also in current working directory so it finds pyutils_gph/utils.py directly in /home/gph without using PYTHONPATH
If you will go to different folder then it will not find local pyutils_gph/utils.py and you get the same error.

Subprocess popen interaction with virtualenv

I am using virtualenv to run a script that uses subprocess popen to run another program that requires the system wide python and original environment variables. How do I prevent virtualenv from affecting it?

You can pass in a modified PATH for the subprocess with env=.
from subprocess import Popen
from os import environ
from os.path import join as path_join
myenv = environ.copy()
if 'VIRTUAL_ENV' in environ:
myenv['PATH'] = ':'.join(
[x for x in environ['PATH'].split(':')
if x != path_join(environ['VIRTUAL_ENV'], 'bin')])
Popen(['command', '--with', 'arguments'], env=myenv)

virtualenv creates a copy of python executable and you can activate it to your current shell:
This will change your $PATH so its first entry is the virtualenv’s
bin/ directory. (You have to use source because it changes your shell
environment in-place.) This is all it does; it’s purely a convenience.
If you directly run a script or the python interpreter from the
virtualenv’s bin/ directory (e.g. path/to/ENV/bin/pip or
/path/to/ENV/bin/python-script.py) there’s no need for activation.
So when I've activated python in a virtualenv for my project it's the one that will be used:
gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ source ../venv/bin/activate
(venv) gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ which python
/home/gonczor/Projects/papukurier/venv/bin/python
(venv) gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ python
Python 2.7.14 (default, Sep 23 2017, 22:06:14)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.executable
'/home/gonczor/Projects/papukurier/venv/bin/python'
>>>
But at the same time you can give full path to execute any other python instance on your computer:
(venv) gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ /usr/bin/python
Python 2.7.14 (default, Sep 23 2017, 22:06:14)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.executable
'/usr/bin/python'
>>>

I believe subprocess doesn't care about you venv
subprocess.run('/path/to/system/python program.py', stdout=PIPE, stderr=PIPE)

"python3" command not doing anything

When I run the python3 command, nothing happens. No stdout, no stderr... nothing. When I try the python2 command, however- I get the expected result. What is going on?
ubuntu#ip-172-91-23-255:~$ python3
ubuntu#ip-172-91-23-255:~$
ubuntu#ip-172-91-23-255:~$ which python3
usr/bin/python3
ubuntu#ip-172-91-23-255:~$ python2
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

Just solved the same issue
If you renamed the python executable change it back, else, ignore this.
add the folder of python3 instalation to the PATH,
the instalation folder is possibly
this: C:\Users\RV420\AppData\Local\Programs\Python\Python37-32
To add folders to your path you can just search 'environment
variables' on windows, then click on environment variables and find
one called path
Now make sure that the new folder comes first in the path that the Python2 instalation folder
You're good to go

Ubuntu VTK for Python compiled from source

I compiled VTK 7.0 (6.3 has the same effect) over cmake with following params:
-LIBRARY_OUTPUT_PATH:PATH="" -CMAKE_INSTALL_PREFIX:PATH="/usr/local" -VTK_ENABLE_VTKPYTHON:BOOL="1" -Module_vtkPython:BOOL="1" - -VTK_Group_Qt:BOOL="1" -CMAKE_OBJCOPY:FILEPATH="/usr/bin/objcopy" -VTK_RENDERING_BACKEND:STRING="OpenGL2" -VTK_INSTALL_PYTHON_MODULE_DIR:PATH="/usr/local/lib/python2.7/site-packages" -DVTK_EGL_DEVICE_INDEX:STRING="0" -VTK_WRAP_PYTHON:BOOL="1" -Module_vtkGUISupportQtOpenGL:BOOL="1"
Now i can find the binary "vtkpython" at /usr/local/bin .
Good news:
I am allowed to enter python shell with the command "vtkpython" out of this directory (/usr/local/bin) with all the needed vtk bindings.
markovich#markovich-desktop:~$ cd /usr/local/bin/
markovich#markovich-desktop:/usr/local/bin$ vtkpython
vtk version 7.0.0
Python 2.7.10 (default, Oct 14 2015, 16:09:02)
[GCC 5.2.1 20151010] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
>>> vtk
<module 'vtk' from '/usr/local/lib/python2.7/site-packages/vtk/__init__.py'>
>>>
Thats a bit irritating because i am expecting to run my default python environement and the vtk bindings are available.
so the bad news:
if I type python in my shell or vtkpython from another location on my system the shell says "no modulen named vtk found" on calling import vtk .
markovich#markovich-desktop:~$ vtkpython
vtk version 7.0.0
Python 2.7.10 (default, Oct 14 2015, 16:09:02)
[GCC 5.2.1 20151010] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named vtk
>>>
Question 1: Maybe I missed something in the make configuration ?
Question 2: If I take the actual status (which is somehow working): Is it possible integrate the "vtkpython" bindings in my default python environment? If I am not totally wrong. The binding is correctly loaded out of my python2.7 path like you can see in the terminal:
<module 'vtk' from '/usr/local/lib/python2.7/site-packages/vtk/__init__.py'>
So how can I add this module to be loaded in Python environment?

Since loading vtkpython clearly shows that you have the module available somewhere on your system, you should be able to add the location of the vtk module to your PYTHONPATH variable.
Find where the vtk module is installed (try /usr/local/lib/python2.7/site-packages, you should see a /vtk folder). If you're unsure, you could try to find it in vtkpython with
import vtk
import imp
imp.find_module('vtk')
You can check what paths are stored in PYTHONPATH by entering in terminal:
echo $PYTHONPATH
(In my install, it was empty by default.)
Then you can add the vtk folder location to your PYTHONPATH in terminal:
export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python2.7/site-packages
Check if vtk is availabe:
$ python
>>> import vtk
If it works, you can add the export... line above into your ~/.bashrc or ~/.profile (as appropriate per your distro installation) to permanently load the option in PYTHONPATH.

Only able to import python module inside the installation directory

I installed 64bit package of cefpython in Ubuntu 12.04 LTS (http://code.google.com/p/cefpython/). The problem is I am not able to run the examples. It says no module named wx.
But when I navigate to the directory /usr/local/lib/python2.7/dist-packages/cefpython1 and do import wx it works. So basically I am not able to import wx outside that directory. I am using python interpreter on terminal.
rishi:cefpython1 ls
cefclient cefpython_py27.pyc chrome.pak examples __init__.pyc LICENSE.txt wx
cefpython_py27.py cefpython_py27.so devtools_resources.pak __init__.py libcef.so locales
rishi:cefpython1 python
Python 2.7.3 (default, Sep 26 2013, 20:03:06)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
>>>
[2]+ Stopped python
rishi:cefpython1 cd ..
rishi:dist-packages python
Python 2.7.3 (default, Sep 26 2013, 20:03:06)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named wx
>>>
KeyboardInterrupt
>>>
My PATH is as follows and PYTHONPATH is empty:
installed 64bit package of cefpython in Ubuntu 12.04 LTS (http://code.google.com/p/cefpython/). The problem is I am not able to run the examples. It says no module named wx. But when I navigate to the directory /usr/local/lib/python2.7/dist-packages/cefpython1 and do import wx it works. So basically I am not able to import wx outside that directory. I am using python interpreter on terminal.
rishi:dist-packages echo $PATH
/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
rishi:dist-packages echo $PYTHONPATH
rishi:dist-packages echo $PYTHONPATH
rishi:dist-packages

Your problem here is that python has no idea where you are importing from. Refer to This post for instructions. Basically, what is happening is that when you're in the directory, python knows to look for it (it looks for python files and packages in the directory). You need to add the python libraries into the PYTHONPATH.