Develop Reference

How to execute a system call in python using PyCharm on Debian Linux?

I am not able to run a system call using PyCharm and can't figure out what variables or environment settings to change.
Given this simple script:
import os
cmd = 'ifconfig -a'
os.system(cmd)
...which runs fine at the command line in terminal, yields the following error:
sh: ifconfig: command not found
This is happening with really any process I'm trying to run such as CSVSQL, PSQL, etc.
I have tried: Displaying my python interpreter paths dispayed at the command line, I tried adding them to the PyCharm interpreter paths, to no avail.
There are several other threads out there describing similar problems, but there doesn't seem to be a good solution that I have come across.
I'm running Linux Mint 19, though this works on my Windows installation (PATH output is much different).
My apologies if this is really simple... Thank you!

Run printenv on both Python and terminal, and check the PATH variable. Use os.environ['PATH'] = 'My path' to set it to what you saw on the terminal.
For future issues (That I've run into):
A quick way to check if it's an exported environment variable is to run os.system("/bin/sh -c \"MYCMD\""), and then run the same "/bin/sh -c \"MYCMD\"" string in your terminal. If there's still a problem, then it must be an export (And this is the likely issue).
To resolve this, try printenv in both python and the terminal to see the list of exports. You should see a discrepancy. The format is simple as you can simple copy the output of the terminal's printenv (Which should be a series of declares), and paste it into python so python will get the same variables. Then your "/bin/sh CMD" calls should align.
The wrapped /bin/sh is in case they're running different shells or have different local variables. echo $SHELL can confirm this, at which point you can compare sets and printenvs and copy paste in the same way. Wrapped you only have to compare exports, as that's what get passed to child processes.

Looks like pycharm isn't getting the PATH from your profile or rc. Try giving the absolute path of the command.
import os
cmd = '/sbin/ifconfig -a'
os.system(cmd)
You can also verify your path using following.
print(os.environ['PATH'])
And use following to add your custom path to current env path.
os.environ['PATH'] += ':/sbin'

How can I run the Python3 interpreter with variables/references/etc as defined in a python file?

I'm new to python and enjoying learning the language. I like using the interpreter in real time, but I still don't understand completely how it works. I would like to be able to define my environment with variables, imports, functions and all the rest then run the interpreter with those already prepared. When I run my files (using PyCharm, Python 3.6) they just execute and exit.
Is there some line to put in my .py files like a main function that will invoke the interpreter? Is there a way to run my .py files from the interpreter where I can continue to call functions and declare variables?
I understand this is a total newbie question, but please explain how to do this or why I'm completely not getting it.

I think you're asking three separate things, but we'll count it as one question since it's not obvious that they are different things:
1. Customize the interactive interpreter
I would like to be able to define my environment with variables, imports, functions and all the rest then run the interpreter with those already prepared.
To customize the environment of your interactive interpreter, define the environment variable PYTHONSTARTUP. How you do that depends on your OS. It should be set to the pathname of a file (use an absolute path), whose commands will be executed before you get your prompt. This answer (found by Tobias) shows you how. This is suitable if there is a fixed set of initializations you would always like to do.
2. Drop to the interactive prompt after running a script
When I run my files (using PyCharm, Python 3.6) they just execute and exit.
From the command line, you can execute a python script with python -i scriptname.py and you'll get an interactive prompt after the script is finished. Note that in this case, PYTHONSTARTUP is ignored: It is not a good idea for scripts to run in a customized environment without explicit action.
3. Call your scripts from the interpreter, or from another script.
Is there a way to run my .py files from the interpreter where I can continue to call functions and declare variables?
If you have a file myscript.py, you can type import myscript in the interactive Python prompt, or put the same in another script, and your script will be executed. Your environment will then have a new module, myscript. You could use the following variant to import your custom definitions on demand (assuming a file myconfig.py where Python can find it):
from myconfig import *
Again, this is not generally a good idea; your programs should explicitly declare all their dependencies by using specific imports at the top.

You can achieve the result you intend by doing this:
Write a Python file with all the imports you want.
Call your script as python -i myscript.py.
Calling with -i runs the script then drops you into the interpreter session with all of those imports, etc. already executed.
If you want to save yourself the effort of calling Python that way every time, add this to your .bashrc file:
alias python='python -i /Users/yourname/whatever/the/path/is/myscript.py'

You set the environment variable PYTHONSTARTUP as suggested in this answer:
https://stackoverflow.com/a/11124610/1781434

Setting up a specific Python in Jenkins

I am quite new with configuring Jenkins or Python but I have to set up a unitary test in Jenkins. My program is in Python, but only works on Python 2.6 whereas the Jenkins version I should be using is 2.7, so I'm trying to set up Jenkins to set some environment variables so that it prepares launching the accurate Python for that specific test (it is part of a greater project that will successfully run several other tests that work well).
The idea I had was to set in the command to execute several environment variables like PATH, LD_LIBRARY_PATH and PYTHONPATH such as following in the "Execute shell" command line interpreter:
PYTHONPATH=/path/to/python2.6/lib:$PYTHONPATH
PATH=/path/to/python2.6/bin:$PATH
LD_LIBRARY_PATH=/path/to/python2.6/lib:$LD_LIBRARY_PATH
... however, it was still calling the wrong version of Python. Therefore, I forced these variables to:
PYTHONPATH=/path/to/python2.6/lib
PATH=/path/to/python2.6/bin
LD_LIBRARY_PATH=/path/to/python2.6/lib
... and I still get errors because the old version of Python is called instead, even if it should not appear in the PATH ... It appears Jenkins will remember the location of the old libraries however and will try loading them first.
How would I correctly set the environment in a "subproject" in Jenkins so that I can call a different version of Python?
Thank you and best regards,
~Stéphane

If you want your program to run with a specific version of the python interpreter, you indicate it in the shebang
#!/usr/bin/python2.6
#your code here

What i did in my Jenkins shell script using a specific python version was something like this when calling my unit test:
python3 src/test/unit_test.py
I was using it to use Python 3.X but it should work with 2.6 as well using:
python2.6 src/test/unit_test.py

Stupid me... I was indeed doing things correctly, I just had a part of my code that was overriding the PYTHONPATH value, so the solution I had found previously was good.
FYI, I modified my shebang, if it's of any help to anyone ;)

How to properly use PyDev with two different Python versions with scripts that are recalling other python scripts?

The story began with a very strange error while I was running my script from PyDev. Running the same script from outside will not encounter the same problem.
Fatal Python error: Py_Initialize: can't initialize sys standard streams
File "C:\Python26\lib\encodings\__init__.py", line 123
raise CodecRegistryError,\
^
SyntaxError: invalid syntax
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
I was able to find why this is happening: In PyDev I use two different Python versions: 3.1 that is the default installation and 2.6 as the alternative one.
My Windows Environment does not contains PYTHONHOME, CLASSPATH, PYTHONPATH but PyDev does add them.
Now the problem is at one stage my python script does execute another python script using os.system(python second.py) and the second script will fail with the above error.
Now I'm looking to find a way to prevent this issue, issue that is happening because it will run the execute the default python using the settings for the non-default one (added by PyDev).
I do not want to change the standard call (python file.py) but I want to be able to run my script from pydev without problem and being able to use default or alternative python environment.
Any ideas?

I found a solution that seams acceptable specially because it will not interfere with running the scripts on other systems, just to run python -E second.py - this will force Python to ignore PYTHON* environment variables.

I may not be understanding this quite right, but I think you're invoking a script from pydev that works okay, but this script executes another script which requires a different version.
While this would unfortunately be installation-specific, you could use os.system("c:\absolute\path\to\proper\version\of\python.exe second.py").
If PyDev is setting up conflicting environmental variables, you may want to look into subprocess over os.system.
http://docs.python.org/library/subprocess.html#using-the-subprocess-module
This will allow you to invoke a process with a handle, so you can optionally wait for it to terminate. It will also allow you to pass environment variables upon execution.

I believe your call should be:
import sys
os.system(sys.executable+ ' second.py')
So that you guarantee you're using the same interpreter you're currently running and not launching the other one (or did you really mean to use the other interpreter?)

Temporary PYTHONPATH in Windows

How do I set, temporarily, the PYTHONPATH environment variable just before executing a Python script?
In *nix, I can do this:
$ PYTHONPATH='.' python scripts/doit.py
In Windows, this syntax does not work, of course. What is the equivalent, though?

How temporarily? If you open a Windows console (cmd.exe), typing:
set PYTHONPATH=.
will change PYTHONPATH for that console only and any child processes created from it. Any python scripts run from this console will use the new PYTHONPATH value. Close the console and the change will be forgotten.

To set and restore an environment variable on Windows' command line requires an unfortunately "somewhat torturous" approach...:
SET SAVE=%PYTHONPATH%
SET PYTHONPATH=.
python scripts/doit.py
SET PYTHONPATH=%SAVE%
You could use a little auxiliary Python script to make it less painful, e.g.
import os
import sys
import subprocess
for i, a in enumerate(sys.argv[1:]):
if '=' not in a: break
name, _, value = a.partition('=')
os.environ[name] = value
sys.exit(subprocess.call(sys.argv[i:]))
to be called as, e.g.,
python withenv.py PYTHONPATH=. python scripts/doit.py
(I've coded it so it works for any subprocess, not just a Python script -- if you only care about Python scripts you could omit the second python in the cal and put 'python' in sys.argv[i-1] in the code, then use sys.argv[i-1:] as the argument for subprocess.call).

In Windows, you can set PYTHONPATH as an environment variable, which has a GUI front end. On most versions of Windows, you can launch by right click on My Computer and right click Properties.

You use SET on Windows:
SET PYTHONPATH=.
python scripts/doit.py

Windows can localize variables within the script. This will save having to set PYTHONPATH before running. It will also help when different scripts require different conflicting PYTHONPATHs.
setlocal
set PYTHONPATH=.
python.exe scripts\doit.py
endlocal
For more info, check MS documentation
setlocal
endlocal