I am using virtualenv to run a script that uses subprocess popen to run another program that requires the system wide python and original environment variables. How do I prevent virtualenv from affecting it?
You can pass in a modified PATH for the subprocess with env=.
from subprocess import Popen
from os import environ
from os.path import join as path_join
myenv = environ.copy()
if 'VIRTUAL_ENV' in environ:
myenv['PATH'] = ':'.join(
[x for x in environ['PATH'].split(':')
if x != path_join(environ['VIRTUAL_ENV'], 'bin')])
Popen(['command', '--with', 'arguments'], env=myenv)
virtualenv creates a copy of python executable and you can activate it to your current shell:
This will change your $PATH so its first entry is the virtualenv’s
bin/ directory. (You have to use source because it changes your shell
environment in-place.) This is all it does; it’s purely a convenience.
If you directly run a script or the python interpreter from the
virtualenv’s bin/ directory (e.g. path/to/ENV/bin/pip or
/path/to/ENV/bin/python-script.py) there’s no need for activation.
So when I've activated python in a virtualenv for my project it's the one that will be used:
gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ source ../venv/bin/activate
(venv) gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ which python
/home/gonczor/Projects/papukurier/venv/bin/python
(venv) gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ python
Python 2.7.14 (default, Sep 23 2017, 22:06:14)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.executable
'/home/gonczor/Projects/papukurier/venv/bin/python'
>>>
But at the same time you can give full path to execute any other python instance on your computer:
(venv) gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ /usr/bin/python
Python 2.7.14 (default, Sep 23 2017, 22:06:14)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.executable
'/usr/bin/python'
>>>
I believe subprocess doesn't care about you venv
subprocess.run('/path/to/system/python program.py', stdout=PIPE, stderr=PIPE)
Related
I'm trying to run the following commands in a Python script
import subprocess
image_name = "alpine:3.10"
scan_image = "trivy -q image -f json {}".format(image_name)
scan_result = subprocess.check_output(scan_image.split()).decode('utf-8')
If I run it from a Python script, it raises the following error: [Errno 2] No such file or directory: 'trivy'
But if I run the command using python interpreter (interactive mode) it works fine.
-bash-4.2# python3
Python 3.6.8 (default, Apr 2 2020, 13:34:55)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> image_name = "alpine:3.10"
>>> scan_image = "trivy -q image -f json {}".format(image_name)
>>> subprocess.check_output(scan_image.split()).decode('utf-8')
'[\n {\n "Target": "alpine:3.10 (alpine 3.10.5)",\n "Type": "alpine",\n "Vulnerabilities": null\n }\n]'
>>>
What could I be doing wrong? I've run the commands using Python2 and Python3 interpreters.
Security concerns aside, you need to provide the full path to your executable:
Replace trivy in the script with the results of which trivy from a shell
When I run the python3 command, nothing happens. No stdout, no stderr... nothing. When I try the python2 command, however- I get the expected result. What is going on?
ubuntu#ip-172-91-23-255:~$ python3
ubuntu#ip-172-91-23-255:~$
ubuntu#ip-172-91-23-255:~$ which python3
usr/bin/python3
ubuntu#ip-172-91-23-255:~$ python2
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
Just solved the same issue
If you renamed the python executable change it back, else, ignore this.
add the folder of python3 instalation to the PATH,
the instalation folder is possibly
this: C:\Users\RV420\AppData\Local\Programs\Python\Python37-32
To add folders to your path you can just search 'environment
variables' on windows, then click on environment variables and find
one called path
Now make sure that the new folder comes first in the path that the Python2 instalation folder
You're good to go
What is the $PYTHONPATH variable, and what's the significance in setting it?
Also, if I want to know the content of my current pythonpath, how do I find that out?
As already answered PYTHONPATH is the search path used by python to find modules when you ''import' them.
Echoing $PYTHONPATH is only useful if you have set it.
However in python itself you can list out what it thinks it is.
$ python3
Python 3.4.3 (default, Oct 14 2015, 20:28:29)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']
>>>
PYTHONPATH is the default search path for importing modules. If you use bash, you could type echo $PYTHONPATH to look at it.
I installed 64bit package of cefpython in Ubuntu 12.04 LTS (http://code.google.com/p/cefpython/). The problem is I am not able to run the examples. It says no module named wx.
But when I navigate to the directory /usr/local/lib/python2.7/dist-packages/cefpython1 and do import wx it works. So basically I am not able to import wx outside that directory. I am using python interpreter on terminal.
rishi:cefpython1 ls
cefclient cefpython_py27.pyc chrome.pak examples __init__.pyc LICENSE.txt wx
cefpython_py27.py cefpython_py27.so devtools_resources.pak __init__.py libcef.so locales
rishi:cefpython1 python
Python 2.7.3 (default, Sep 26 2013, 20:03:06)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
>>>
[2]+ Stopped python
rishi:cefpython1 cd ..
rishi:dist-packages python
Python 2.7.3 (default, Sep 26 2013, 20:03:06)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named wx
>>>
KeyboardInterrupt
>>>
My PATH is as follows and PYTHONPATH is empty:
installed 64bit package of cefpython in Ubuntu 12.04 LTS (http://code.google.com/p/cefpython/). The problem is I am not able to run the examples. It says no module named wx. But when I navigate to the directory /usr/local/lib/python2.7/dist-packages/cefpython1 and do import wx it works. So basically I am not able to import wx outside that directory. I am using python interpreter on terminal.
rishi:dist-packages echo $PATH
/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
rishi:dist-packages echo $PYTHONPATH
rishi:dist-packages echo $PYTHONPATH
rishi:dist-packages
Your problem here is that python has no idea where you are importing from. Refer to This post for instructions. Basically, what is happening is that when you're in the directory, python knows to look for it (it looks for python files and packages in the directory). You need to add the python libraries into the PYTHONPATH.
When I start Python from Mac OS' Terminal.app, python recognises the encoding as UTF-8:
$ python3.0
Python 3.0.1 (r301:69556, May 18 2009, 16:44:01)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.stdout.encoding
'UTF-8'
This works the same for python2.5.
But inside Emacs, the encoding is US-ASCII.
Python 3.0.1 (r301:69556, May 18 2009, 16:44:01)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.stdout.encoding
'US-ASCII'
How do I make Emacs communicate with Python so that sys.stdout knows to use UTF-8?
Edit: Since I don't have the rep to edit the accepted answer, here is precisely what worked for me on Aquaemacs 1.6, Mac OS 10.5.6.
In the python-mode-hook, I added the line
(setenv "LANG" "en_GB.UTF-8")
Apparently, Mac OS requires "UTF-8", while dfa says that Ubuntu requires "UTF8".
Additionally, I had to set the input/output encoding by doing C-x RET p and then typing "utf-8" twice. I should probably find out how to set this permanently.
Thanks to dfa and Jouni for collectively helping me find the answer.
Here is my final python-mode-hook:
(add-hook 'python-mode-hook
(lambda ()
(set (make-variable-buffer-local 'beginning-of-defun-function)
'py-beginning-of-def-or-class)
(define-key py-mode-map "\C-c\C-z" 'py-shell)
(setq outline-regexp "def\\|class ")
(setenv "LANG" "en_GB.UTF-8"))) ; <-- *this* line is new
check your environment variables:
$ LANG="en_US.UTF8" python -c "import sys; print sys.stdout.encoding"
UTF-8
$ LANG="en_US" python -c "import sys; print sys.stdout.encoding"
ANSI_X3.4-1968
in your python hook, try:
(setenv "LANG" "en_US.UTF8")