Develop Reference

How can I run the Python3 interpreter with variables/references/etc as defined in a python file?

I'm new to python and enjoying learning the language. I like using the interpreter in real time, but I still don't understand completely how it works. I would like to be able to define my environment with variables, imports, functions and all the rest then run the interpreter with those already prepared. When I run my files (using PyCharm, Python 3.6) they just execute and exit.
Is there some line to put in my .py files like a main function that will invoke the interpreter? Is there a way to run my .py files from the interpreter where I can continue to call functions and declare variables?
I understand this is a total newbie question, but please explain how to do this or why I'm completely not getting it.

I think you're asking three separate things, but we'll count it as one question since it's not obvious that they are different things:
1. Customize the interactive interpreter
I would like to be able to define my environment with variables, imports, functions and all the rest then run the interpreter with those already prepared.
To customize the environment of your interactive interpreter, define the environment variable PYTHONSTARTUP. How you do that depends on your OS. It should be set to the pathname of a file (use an absolute path), whose commands will be executed before you get your prompt. This answer (found by Tobias) shows you how. This is suitable if there is a fixed set of initializations you would always like to do.
2. Drop to the interactive prompt after running a script
When I run my files (using PyCharm, Python 3.6) they just execute and exit.
From the command line, you can execute a python script with python -i scriptname.py and you'll get an interactive prompt after the script is finished. Note that in this case, PYTHONSTARTUP is ignored: It is not a good idea for scripts to run in a customized environment without explicit action.
3. Call your scripts from the interpreter, or from another script.
Is there a way to run my .py files from the interpreter where I can continue to call functions and declare variables?
If you have a file myscript.py, you can type import myscript in the interactive Python prompt, or put the same in another script, and your script will be executed. Your environment will then have a new module, myscript. You could use the following variant to import your custom definitions on demand (assuming a file myconfig.py where Python can find it):
from myconfig import *
Again, this is not generally a good idea; your programs should explicitly declare all their dependencies by using specific imports at the top.

You can achieve the result you intend by doing this:
Write a Python file with all the imports you want.
Call your script as python -i myscript.py.
Calling with -i runs the script then drops you into the interpreter session with all of those imports, etc. already executed.
If you want to save yourself the effort of calling Python that way every time, add this to your .bashrc file:
alias python='python -i /Users/yourname/whatever/the/path/is/myscript.py'

You set the environment variable PYTHONSTARTUP as suggested in this answer:
https://stackoverflow.com/a/11124610/1781434

How do you run a Python script in a newer Python interpreter?

I am using several applications that are using different versions of Python:
Nuke - 2.7
3Dequalizer - 2.6
linux - 2.6.6
I am getting various problems trying to get them all to communicate with one another, so I was wondering if it's possible to change Python interpreter during a script.
E.g. Start in 2.6, then run a Python script in 2.7 from a script in 2.6
EDIT:
nuke_install = "/path/to/nuke"
cmd = nukeLauncher + " -t"
os.system(cmd)
The -t flag allows for nuke to be run without a GUI. This code works when run in a Python interpreter, but when I run via a Python script in 3dequalizer it gives me the:
ImportError: No module named site
To add another level of confusion, I can import site inside 3dequalizer. The sys.path for 3dequalizer contains the same paths as when run directly from the interpreter, with a few additions for the python lib that comes with 3de.
Also PYTHONPATH is empty inside 3dequalizer. Does this matter if sys.path is pointing to the right paths?

I am not sure it is really the way to go; but if you really want to do it, you could use the os.system command with somthing like:
os.system("python2.7 myscript.py")
which will execute the program python2.7 (as long as it is in your executable path) with the name of the script as its argument (before returning to the current) statement in your initial script.
But honestly, I think you should do it in some other way. Regards.

launch external shell python instance in shell from python

I'd like to call a separate non-child python program from a python script and have it run externally in a new shell instance. The original python script doesn't need to be aware of the instance it launches, it shouldn't block when the launched process is running and shouldn't care if it dies. This is what I have tried which returns no error but seems to do nothing...
import subprocess
python_path = '/usr/bin/python'
args = [python_path, '&']
p = subprocess.Popen(args, shell=True)
What should I be doing differently
EDIT
The reason for doing this is I have an application with a built in version of python, I have written some python tools that should be run separately alongside this application but there is no assurance that the user will have python installed on their system outside the application with the builtin version I'm using. Because of this I can get the python binary path from the built in version programatically and I'd like to launch an external version of the built in python. This eliminates the need for the user to install python themselves. So in essence I need a simple way to call an external python script using my current running version of python programatically.
I don't need to catch any output into the original program, in fact once launched I'd like it to have nothing to do with the original program
EDIT 2
So it seems that my original question was very unclear so here are more details, I think I was trying to over simplify the question:
I'm running OSX but the code should also work on windows machines.
The main application that has a built in version of CPython is a compiled c++ application that ships with a python framework that it uses at runtime. You can launch the embedded version of this version of python by doing this in a Terminal window on OSX
/my_main_app/Contents/Frameworks/Python.framework/Versions/2.7/bin/python
From my main application I'd like to be able to run a command in the version of python embedded in the main app that launches an external copy of a python script using the above python version just like I would if I did the following command in a Terminal window. The new launched orphan process should have its own Terminal window so the user can interact with it.
/my_main_app/Contents/Frameworks/Python.framework/Versions/2.7/bin/python my_python_script
I would like the child python instance not to block the main application and I'd like it to have its own terminal window so the user can interact with it. The main application doesn't need to be aware of the child once its launched in any way. The only reason I would do this is to automate launching an external application using a Terminal for the user

If you're trying to launch a new terminal window to run a new Python in (which isn't what your question asks for, but from a comment it sounds like it's what you actually want):
You can't. At least not in a general-purpose, cross-platform way.
Python is just a command-line program that runs with whatever stdin/stdout/stderr it's given. If those happen to be from a terminal, then it's running in a terminal. It doesn't know anything about the terminal beyond that.
If you need to do this for some specific platform and some specific terminal program—e.g., Terminal.app on OS X, iTerm on OS X, the "DOS prompt" on Windows, gnome-terminal on any X11 system, etc.—that's generally doable, but the way to do it is by launching or scripting the terminal program and telling it to open a new window and run Python in that window. And, needless to say, they all have completely different ways of doing that.
And even then, it's not going to be possible in all cases. For example, if you ssh in to a remote machine and run Python on that machine, there is no way it can reach back to your machine and open a new terminal window.
On most platforms that have multiple possible terminals, you can write some heuristic code that figures out which terminal you're currently running under by just walking os.getppid() until you find something that looks like a terminal you know how to deal with (and if you get to init/launchd/etc. without finding one, then you weren't running in a terminal).

The problem is that you're running Python with the argument &. Python has no idea what to do with that. It's like typing this at the shell:
/usr/bin/python '&'
In fact, if you pay attention, you're almost certainly getting something like this through your stderr:
python: can't open file '&': [Errno 2] No such file or directory
… which is exactly what you'd get from doing the equivalent at the shell.
What you presumably wanted was the equivalent of this shell command:
/usr/bin/python &
But the & there isn't an argument at all, it's part of sh syntax. The subprocess module doesn't know anything about sh syntax, and you're telling it not to use a shell, so there's nobody to interpret that &.
You could tell subprocess to use a shell, so it can do this for you:
cmdline = '{} &'.format(python_path)
p = subprocess.Popen(cmdline, shell=True)
But really, there's no good reason to. Just opening a subprocess and not calling communicate or wait on it already effectively "puts it in the background", just like & does on the shell. So:
args = [python_path]
p = subprocess.Popen(args)
This will start a new Python interpreter that sits there running in the background, trying to use the same stdin/stdout/stderr as your parent. I'm not sure why you want that, but it's the same thing that using & in the shell would have done.

Actually I think there might be a solution to your problem, I found a useful solution at another question here.
This way subprocess.popen starts a new python shell instance and runs the second script from there. It worked perfectly for me on Windows 10.

You can try using screen command
with this command a new shell instance created and the current instance runs in the background.
# screen; python script1.py
After running above command, a new shell prompt will be seen where we can run another script and script1.py will be running in the background.
Hope it helps.

How to properly use PyDev with two different Python versions with scripts that are recalling other python scripts?

The story began with a very strange error while I was running my script from PyDev. Running the same script from outside will not encounter the same problem.
Fatal Python error: Py_Initialize: can't initialize sys standard streams
File "C:\Python26\lib\encodings\__init__.py", line 123
raise CodecRegistryError,\
^
SyntaxError: invalid syntax
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
I was able to find why this is happening: In PyDev I use two different Python versions: 3.1 that is the default installation and 2.6 as the alternative one.
My Windows Environment does not contains PYTHONHOME, CLASSPATH, PYTHONPATH but PyDev does add them.
Now the problem is at one stage my python script does execute another python script using os.system(python second.py) and the second script will fail with the above error.
Now I'm looking to find a way to prevent this issue, issue that is happening because it will run the execute the default python using the settings for the non-default one (added by PyDev).
I do not want to change the standard call (python file.py) but I want to be able to run my script from pydev without problem and being able to use default or alternative python environment.
Any ideas?

I found a solution that seams acceptable specially because it will not interfere with running the scripts on other systems, just to run python -E second.py - this will force Python to ignore PYTHON* environment variables.

I may not be understanding this quite right, but I think you're invoking a script from pydev that works okay, but this script executes another script which requires a different version.
While this would unfortunately be installation-specific, you could use os.system("c:\absolute\path\to\proper\version\of\python.exe second.py").
If PyDev is setting up conflicting environmental variables, you may want to look into subprocess over os.system.
http://docs.python.org/library/subprocess.html#using-the-subprocess-module
This will allow you to invoke a process with a handle, so you can optionally wait for it to terminate. It will also allow you to pass environment variables upon execution.

I believe your call should be:
import sys
os.system(sys.executable+ ' second.py')
So that you guarantee you're using the same interpreter you're currently running and not launching the other one (or did you really mean to use the other interpreter?)

Problem with Python interpreter in Eclipse

When trying to set the interpreter for python in Eclipse by choosing the executable, clicking OK displays "An error has occured." Does the interpreter name matter?

I had a similar problems with this on Mac OS X. My problem was that I had a space in Eclipse's application path, e.g. "/Applications/eclipse 3.3/Eclipse".
I changed the folder name to "/Applications/eclipse3.3" and it fixed it.

Testing/running your apps on the command line is the safest bet, especially when writing threaded applications (you can kill your threadlocked program without killing eclipse)