Running a simple .py or .pyw python file causes python.exe to show up under Task Manager.
python myApp.py
python myApp.pyw
However when we try to run it without using the console, the script does not appear to run, nor does python.exe or pythonw.exe appears under Task Manager
pythonw myApp.pyw
pythonw myApp.py
How do we troubleshoot the problem? The system is running Python 2.7.8 x64.
tl;dr
To troubleshoot, use output redirection on invocation:
pythonw myApp.py 1>stdout.txt 2>stderr.txt
This will capture stdout output, such as from print(), in file stdout.txt, and stderr output (such as from unhandled exceptions), in file stderr.txt; from PowerShell, use
cmd /c pythonw myApp.py 1>stdout.txt 2>stderr.txt).
Note that the very act of redirecting stdout may actually make your script work again, if the only reason for its failure with pythonw was the use of print (in Python 2.x - see below).
Caveat: This output redirection technique seemingly does not work when invoking *.pyw scripts directly (as opposed to by passing the script file path to pythonw.exe). Do let me know if you know why and/or if it does work for you.
To fix your script:
Place the following at the top of any Python 2.x or 3.x script that you want to run with pythonw.exe:
import sys, os
if sys.executable.endswith("pythonw.exe"):
sys.stdout = open(os.devnull, "w");
sys.stderr = open(os.path.join(os.getenv("TEMP"), "stderr-"+os.path.basename(sys.argv[0])), "w")
This ensures the following when a script is run with pythonw.exe:
print() calls and explicit calls to sys.stdout() are effectively ignored (are no-ops).
Stderr output, including from an unhandled fatal exception, is sent to file
%TEMP%\stderr-<scriptFileName>; %TEMP% is a standard Windows environment variable that points to the current user's folder for temporary files.
In other words: With the above code in place, check file %TEMP%\stderr-<scriptFileName> after your script has failed silently when invoked with pythonw.exe.
For an explanation, read on.
On Windows, pythonw.exe is for launching GUI/no-UI-at-all scripts, which means that the
standard in- and output streams - sys.stdin, sys.stdout, sys.stderr are NOT available.
This has two nasty side effects:
Using print() - which targets sys.stdout by default - causes an exception in Python 2.x.
This problem has been fixed in Python 3.x.
Any unhandled exception - including one triggered by print() in 2.x - causes the script to abort silently.
Exception error messages go to sys.stderr by default, which is the very thing not available in this scenario.
The above code fixes these problems by:
sending stdout output to the null device, effectively ignoring any attempt to output to sys.stdout - whether explicitly, or implicitly via print().
sending all stderr output to a temporary file.
Differences between Python 2.x and Python 3.x:
When a script is run with pythonw.exe, sys.stdin, sys.stdout, and sys.stderr:
in Python 2.x: have invalid file descriptors
The eventual result when trying to write to sys.stdout or sys.stderr is the following exception: IOError: [Errno 9] Bad file descriptor
Pitfall: Due to output buffering, this exception may not surface until you've output, say, 4K bytes; you can provoke it instantly by invoking pythonw.exe with -u (for unbuffered output).
print() blindly tries to sys.stdout (by default), so it provokes this exception sooner or later.
in Python 3.x: are set to None
This is complemented with the 3.x print() function performing a no-op (doing nothing) when it finds that sys.stdout is None, so that print() statements can by default safely be used - they'll simply be ignored when run with pythonw.exe
However, it follows that trying to use sys.stdout.write() and sys.stderr.write() still results in an exception.
See here for more background.
Try adding the line import sys; sys.stderr = open("errlog.txt", "w") to the start of myApp.py. Then look in errlog.txt for a traceback or any other error messages.
I faced the same problem on a script of my own and found that when adding the output from Ross' answer the script would actually run.
It appears that for some reason that redirecting output fixes the problem. Since I'm not interested in writing the output to disk I've instead written it to /dev/null (or the platform equivalent) with:
if ( sys.platform == 'win32' and sys.executable.split( '\\' )[-1] == 'pythonw.exe'):
sys.stdout = open(os.devnull, 'w')
sys.stderr = open(os.devnull, 'w')
The if statement ensures it only happens when the script is launched from pythonw.exe. I'm not sure if it is related but it was important to do this before other imports (including e.g. import logging).
I was having similar problem.
After debugging step by step by writing to a log file, I discovered that pythonw.exe crashed after a statement that tried to use the call: sys.stdout.write(). It turns out, when run with pythonw.exe, sys.stdout is None.
If you are using functions of sys.stdout/stderr/stdin, and intend to use your program with pythonw.exe, adding a check for "None" is a good idea.
Im not sure I understand your problem but I think this is what you need to know
you need to right click on a py or pyw file and select open with ... find python.exe (probably C:\Python27\python.exe) .. check the box that says always open ... now you can just double click it if you want to run it
(usually the installer sets this up for you ...)
This is an old answer, but I want to leave my solution here also:
Open CMD (with elevated privileges or not - depends on your needs)
Change to directory of .py / .pyw script - this is important
Run pythonw with script as argument
cd E:\my\script\folder\
pythonw script.py
I had a similar problem after an upgrade to my computer RAM. Turns out I had to reinstall Pillow (library used for image processing). So make sure it is installed and if it's not, install it using "pip install Pillow" in cmd.
Related
I have a MATLAB .m file and want a Python.py file to be able to run/call that .m file, however it seems calling MATLAB files using 'import matlab.engine' isn't supported in Python 3.9 (what Im running) and was cut after 3.7
My questions is what is the proper convention to call a MATLAB script in Python 3.9?
EDIT: When I say 'call a MATLAB script', I mean run the script on it's own. Using the .py file to open the MATLAB window kind of defeats the purpose of this project.
This answer is for Python 3.6 but it should be applicable to 3.9 as well. On a Linux machine (and MacOS, possibly with slight modifications), you can execute a system command which opens MATLAB, runs your script, and then closes MATLAB,
import subprocess
run_matlab = 'matlab -nodisplay -nojvm -nosplash -nodesktop -r "try, test_script, catch e, disp(getReport(e)), exit(1), end, exit(0);"'
subprocess.run(run_matlab, shell=True, check=True)
The purpose of the try-catch block is to properly exit from MATLAB after executing the script test_script in case when the script raises an exception - throws an error. One could instead just exit (exit or quit in MATLAB returns proper exit code by default on those two OS), but with a try-catch block there are more opportunities to fix, notify, or save things when an exception occurs. This may be generally useful when running outside an interactive session.
The rest of the command opens MATLAB with no display of any kind, runs the script test_script, and exits MATLAB - exit(1) signifying an exception was raised, and exit(0) meaning that all went well.
Without the try-catch the command is much simpler,
run_matlab = 'matlab -nodisplay -nojvm -nosplash -nodesktop -r "test_script; exit;"'
On the Python side, I use subprocess.run() (or .call()) since it is newer, and recommended. The option shell means that the command will be executed through the shell. check means that if the run_matlab process (command) throws an error, the Python script will also throw one. The default is False and along with the try-catch block may be used to swiftly ignore crashing code.
As #CrisLuengo said in his comment, starting with MATLAB 2019a (MATLAB 2018b indeed does not have it) there is no need for the elaborated command above. All you need is
run_matlab = 'matlab -batch test_script.m'
Looking at the documentation, -batch invokes all the necessary options, and more.
This is more of a hack fix, but you can edit the Python package for the matlab engine on your system and bypass the version checker. Since Python is backwards compatible it shouldn't be a problem, depending on your use case.
Open the following file on your system (or the equivalent file under matlab engine):
vim /usr/lib/python3.9/site-packages/matlab/engine/__init__.py
In that file, change line 31 to include Python 3.9:
_supported_versions = ['2_7', '3_6', '3_7', '3_8', '3_9']
Finally, in line 37, change the version number to a valid version.
_PYTHONVERSION = '3_7' # or another new, supported version
After this, you should be able to run matlab's engine as if Python 3.9 were supported.
Long story short: pythonw.exe does nothing, python.exe accepts nothing (which one should I use?)
test.py:
print "a"
CMD window:
C:\path>pythonw.exe test.py
<BLANK LINE>
C:\path>
C:\path>python.exe test.py
File "C:\path\test.py", line 7
print "a"
^
SyntaxError: invalid syntax
C:\path>
Please tell me what I'm doing terrible wrong.
To summarize and complement the existing answers:
python.exe is a console (terminal) application for launching CLI-type scripts (console applications).
Unless run from an existing console window, python.exe opens a new console window.
Standard streams sys.stdin, sys.stdout and sys.stderr are connected to the console window.
Execution is synchronous when launched from a cmd.exe or PowerShell console window: See eryksun's 1st comment below.
If a new console window was created, it stays open until the script terminates.
When invoked from an existing console window, the prompt is blocked until the script terminates.
pythonw.exe is a GUI app for launching GUI/no-UI-at-all scripts.
NO console window is opened.
Execution is asynchronous:
When invoked from a console window, the script is merely launched and the prompt returns right away, whether the script is still running or not.
Standard streams sys.stdin, sys.stdout and sys.stderr are NOT available.
Caution: Unless you take extra steps, this has potentially unexpected side effects:
Unhandled exceptions cause the script to abort silently.
In Python 2.x, simply trying to use print() can cause that to happen (in 3.x, print() simply has no effect).
To prevent that from within your script, and to learn more, see this answer of mine.
Ad-hoc, you can use output redirection:Thanks, #handle.
pythonw.exe yourScript.pyw 1>stdout.txt 2>stderr.txt
(from PowerShell:
cmd /c pythonw.exe yourScript.pyw 1>stdout.txt 2>stderr.txt) to capture stdout and stderr output in files.
If you're confident that use of print() is the only reason your script fails silently with pythonw.exe, and you're not interested in stdout output, use #handle's command from the comments:
pythonw.exe yourScript.pyw 1>NUL 2>&1
Caveat: This output redirection technique does not work when invoking *.pyw scripts directly (as opposed to by passing the script file path to pythonw.exe). See eryksun's 2nd comment and its follow-ups below.
You can control which of the executables runs your script by default - such as when opened from Explorer - by choosing the right filename extension:
*.py files are by default associated (invoked) with python.exe
*.pyw files are by default associated (invoked) with pythonw.exe
If you don't want a terminal window to pop up when you run your program, use pythonw.exe;
Otherwise, use python.exe
Regarding the syntax error: print is now a function in 3.x
So use instead:
print("a")
See here: http://docs.python.org/using/windows.html
pythonw.exe "This suppresses the terminal window on startup."
If you're going to call a python script from some other process (say, from the command line), use pythonw.exe. Otherwise, your user will continuously see a cmd window launching the python process. It'll still run your script just the same, but it won't intrude on the user experience.
An example might be sending an email; python.exe will pop up a CLI window, send the email, then close the window. It'll appear as a quick flash, and can be considered somewhat annoying. pythonw.exe avoids this, but still sends the email.
I was struggling to get this to work for a while. Once you change the extension to .pyw, make sure that you open properties of the file and direct the "open with" path to pythonw.exe.
In my experience the pythonw.exe is faster at least with using pygame.
I am running a C++ binary which runs a python script. I do this in the following way
stream = popen(cmd.c_str(), "r");
if (stream) {
while (!feof(stream))
if (fgets(buffer, max_buffer, stream) != NULL) data.append(buffer);
status = pclose(stream);
}
This python script also runs some shell commands using subprocess
subprocess.check_call(arg_list)
Suddenly I have started getting errors like "Vim: Warning: Output not to a terminal" and the C++ binary hangs.
What is the easiest way to fix it?
The popen call is going to read whatever the subprocess writes to its standard output. Some of that is likely useful (such as status messages). You would not get useful results from vim's standard output, because it would be partly escape sequences (for moving the cursor, changing the colors).
If you can modify the script so that it handles vim specially, there is a fix: modify the commands which run vim so that they redirect to /dev/tty. That would change a command-string that looks like this
vim filename
to
vim filename >/dev/tty
You can try using
export TERM=vt100
before you call Perl or as the first command in each Unix subprocess.
By some need I was forced to correct os.environ['PATH'] to be able to run dir\to\fake\python.cmd script which adds some extra parameters to the original one before execution.
Also I have two python scripts:
test1.py:
# ...
p = subprocess.call("test2.py") # shell=True works fine
# ...
test2.py:
# ...
print "Hello from test2.py"
# ...
When I run python test1.py my "fake" python.cmd doing its stuff, refers to the original python in c:\Python25 and runs test1.py with my extra arguments. But, sadly, test2.py, script is never called. If I put shell=True as subprocess.call argument - everythin's fine, test2.py is called.
I know, Windows is trying to find python interpreter to use for the call in the real c:\Python25 working directory when shell=False is by default.
The question to you is: how can I achieve the goal without changing my code in test1.py and test2.py? Maybe virtualenv library may be very useful in this case?
Thank you very much for your help
As stated in the docs:
The shell argument (which defaults to False) specifies whether to use the shell as the program to execute.
and
On Windows with shell=True, the COMSPEC environment variable specifies the default shell. The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.
So when you call subprocess.call("test2.py"), the system tries to call test2.py as an executable, which it is not, so it fails. However, you don't capture the return value from subprocess.open to check for error conditions, so it fails silently. When you call it with shell=True, it calls the system shell with the argument test2.py, which in turn looks up the default executable for .py files on your system and then executes the file that way.
All that said though, the deeper problem here is that your code is very poorly engineered. The probability that you have a legitimate reason to dispatch a python script via the OS from another python script is vanishingly small. Rather than patching another kludgy hack over the top of what you already have, save yourself headaches in the future and refactor the system so that it does things in an easier, more sensible, more naturally integrated way.
I'm trying to log the terminal display produced by a script which contains a Scipy optimization routine. I've tried three different ways, all with disappointing results.
open() and cmd-prompt redirection (">") did not log the warnings nor the per-iteration information I would like to keep track of; it only logged the parameter solutions.
logging.basicConfig() produced a blank file.
If possible, I would like to be able to log the exact terminal display of a script file. I've begun looking into using the logging module, but it seems pretty complex for what I would think is a very basic task (I'm coming from a Matlab background).
Environment:
. Python 2.7.3
. Operating Systems: Windows Vista and XP
Probably the stuff you're interested in is being sent to stderr rather than stdout. Try redirecting both to a file. You can either do that in Python:
import sys
with open("log.txt", "w") as logfile:
sys.stdout, sys.stderr = logfile, logfile
# do your stuff here, e.g. import the module that you want to log
Or from the command line:
python myscript.py > log.txt 2>&1
Take a look at linux script command. It saves everything that is printed in your terminal.
user#host:~ $ script
Script started, file is typescript
user#host:~ $ ls
bin Documents Pictures Videos
user#host:~ $ exit # ^D stops script
Script done, file is typescript
Then, the result is in a file named typescript.