I would like to use a script to call an executable program and input some instructions in the exe program (a DOS window) to run this exe program (output not required)
For example if I directly run the program I'll double click it and type as follows:
load XXX.txt
oper
quit
Here's my code, fwiw I do not have a deep understanding about subprocess.
import subprocess
import os
os.chdir('D:/Design/avl3.35/Avl/Runs')
Process=subprocess.Popen(['avl.exe'], stdin=subprocess.PIPE)
Process.communicate(b'load allegro.avl\n')
When I run this code, I get the following:
===================================================
Athena Vortex Lattice Program Version 3.35
Copyright (C) 2002 Mark Drela, Harold Youngren
This software comes with ABSOLUTELY NO WARRANTY,
subject to the GNU General Public License.
Caveat computor
===================================================
==========================================================
Quit Exit program
.OPER Compute operating-point run cases
.MODE Eigenvalue analysis of run cases
.TIME Time-domain calculations
LOAD f Read configuration input file
MASS f Read mass distribution file
CASE f Read run case file
CINI Clear and initialize run cases
MSET i Apply mass file data to stored run case(s)
.PLOP Plotting options
NAME s Specify new configuration name
AVL c>
Reading file: allegro.avl ...
Configuration: Allegro-lite 2M
Building surface: WING
Reading airfoil from file: ag35.dat
Reading airfoil from file: ag36.dat
Reading airfoil from file: ag37.dat
Reading airfoil from file: ag38.dat
At line 145 of file ../src/userio.f (unit = 5, file = 'stdin') #!!!!Error here!!
Fortran runtime error: End of file #!!!!Error here!!
Building duplicate image-surface: WING (YDUP)
Building surface: Horizontal tail
Building duplicate image-surface: Horizontal tail (YDUP)
Building surface: Vertical tail
Mach = 0.0000 (default)
Nbody = 0 Nsurf = 5 Nstrp = 64 Nvor = 410
Initializing run cases...
I have no idea what's wrong with this, nor do I know why the error prompt shows inside the codes. After searching I see that communicate method is to wait for the process to finish and return all the output, though I do not need output, but I still don't know what to do.
Could you explain what is happening here and how could I finish what I want to do?
Related
I'm not sure if it's allowed to seek for help(if not, I don't mind not getting an answer until the competition period is over).
I was solving the Interactive Problem (Dat Bae) on CodeJam. On my local files, I can run the judge (testing_tool.py) and my program (<name>.py) separately and copy paste the I/O manually. However, I assume I need to find a way to make it automatically.
Edit: To make it clear, I want every output of x file to be input in y file and vice versa.
Some details:
I've used sys.stdout.write / sys.stdin.readline instead of print / input throughout my program
I tried running interactive_runner.py, but I don't seem to figure out how to use it.
I tried running it on their server, my program in first tab, the judge file in second. It's always throwing TLE error.
I don't seem to find any tutorial to do the same either, any help will be appreciated! :/
The usage is documented in comments inside the scripts:
interactive_runner.py
# Run this as:
# python interactive_runner.py <cmd_line_judge> -- <cmd_line_solution>
#
# For example:
# python interactive_runner.py python judge.py 0 -- ./my_binary
#
# This will run the first test set of a python judge called "judge.py" that
# receives the test set number (starting from 0) via command line parameter
# with a solution compiled into a binary called "my_binary".
testing_tool.py
# Usage: `testing_tool.py test_number`, where the argument test_number
# is 0 for Test Set 1 or 1 for Test Set 2.
So use them like this:
python interactive_runner.py python testing_tool.py 0 -- ./dat_bae.py
I am trying to do Incomplete Cholesky Decomposition in Python, but no direct Python package I can find.
Since the most available codes I can find online are written in Matlab, I want to take a detour by
compiling the matlab code to a shared library (I am using Mac OS and MATLAB_R2014a, so it should produce .dylib file)
load library in Python by using Ctypes
The following lists the detailed steps:
0. Download Matlab Source Code
The code can be downloaded from F. Bach's webpage link to zip file, which contains the following files:
panc:csi-1.0 panc25$ ls
center.m csi.dll csi.mexglx csi_gaussian.dll csi_gaussian.mexglx readme.txt
csi.c csi.m csi_gaussian.c csi_gaussian.m demo_csi.m sqdist.m
1. Compiling the matlab code to a shared library
Then by following this post, I run the command:
mcc -v -W cpplib:libcsi -T link:lib csi
After around a minute, the terminal prints MEX completed successfully and in my folder there are
panc:csi-1.0 panc25$ ls
center.m csi.m csi_gaussian.dll demo_csi.m libcsi.exports readme.txt
csi.c csi.mexglx csi_gaussian.m libcsi.cpp libcsi.h sqdist.m
csi.dll csi_gaussian.c csi_gaussian.mexglx libcsi.dylib mccExcludedFiles.log
where libcsi.dylib is the shared library I want.
2. Loading library in Python
Then I open IPython and try to load the library:
In [1]: import ctypes
In [2]: ctypes.C
ctypes.CDLL ctypes.CFUNCTYPE
In [2]: ctypes.CDLL('libcsi.dylib')
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-2-b6d0c1a91651> in <module>()
----> 1 ctypes.CDLL('libcsi.dylib')
/Users/panc25/anaconda/lib/python2.7/ctypes/__init__.pyc in __init__(self, name, mode, handle, use_errno, use_last_error)
363
364 if handle is None:
--> 365 self._handle = _dlopen(self._name, mode)
366 else:
367 self._handle = handle
OSError: dlopen(libcsi.dylib, 6): Library not loaded: #rpath/libmwmclmcrrt.8.3.dylib
Referenced from: /Users/panc25/Downloads/csi-1.0/libcsi.dylib
Reason: image not found
This problem persists even after I replace file name in ctypes.CDLL('libcsi.dylib') with the full path.
So I am confused. The shared library is there, but why Python says "image not found"?
BTW
SInce the source code also provide C implementation through mex.h, I also tried to first create a .mex file, then compile the .mex to a shared library as follows:
panc:csi-1.0 panc25$ mex csi.c
which created the csi.mexmaci64 file. Then according to this link, I called:
panc:csi-1.0 panc25$ mcc -B csharedlib:csi2 csi.mexmaci64
which produced csi2.dylib file.
But when I tried to load it in Python, I had the same error.
Could anyone let me know what is wrong?
I would avoid Matlab altogether, and instead use the Incomplete Cholesky Decomposition available in PyMC2:
from pymc.gp.incomplete_chol import ichol_full
The f2py wrapped Fortran code, that was actually adapted from a MEX file, can be found here. So you could use this independently of PyMC2 if need be.
If you are interested, you could also propose to add this function to scipy (see this githib issue ).
I would like to use Elaphe to generate barcodes.
I am working on a 64-bit windows machine. This is on Windows 7, Python 2.7, I have Elaphe 0.6.0 and Ghostscript 9.10 installed.
When I run the simple example usage, nothing seems to be happening. The barcode does not show up. When I execute _.show(), it hangs but nothing shows up. I have to do a KeyboardInterrupt to get back to the prompt. What viewer is supposed to launch when I do _.show()? I however see a gswin32.exe process in the Windows Task Manager.
Please refer to my Python traceback at http://dpaste.com/hold/1653582/
Is there a way to see the PS code generated? How can I troubleshoot?
Please help.
The object returned by elaphe.barcode is an EpsImageFile (where EPS means Encapsulated PostScript), but after calling barcode it hasn't yet run Ghostscript to convert the code into a bitmap image.
You can dump out the code that it has generated by looking at the fp attribute - there's a lot of it, because it embeds the full PS library code for all the different barcode types it supports. So it's probably best to write it out to a file:
b = el.barcode('qr', 'slamacow')
with open('code.eps') as outfile:
outfile.write(b.fp.getvalue()) # fp is a StringIO instance
In the file you'll see something like this:
%!PS-Adobe-2.0
%%Pages: (attend)
%%Creator: Elaphe powered by barcode.ps
%%BoundingBox: 0 0 42 42
%%LanguageLevel: 2
%%EndComments
% --BEGIN RESOURCE preamble--
... A whole lot of included library ...
% --END ENCODER hibccodablockf--
gsave
0 0 moveto
1.000000 1.000000 scale
<74686973206973206d792064617461>
<>
/qrcode /uk.co.terryburton.bwipp findresource exec
grestore
showpage
If you want to see how PIL or pillow runs Ghostscript so you can try it yourself at the commandline, the key part from the PIL/pillow code is this (from site-packages/PIL/EpsImagePlugin.py, line 84):
# Build ghostscript command
command = ["gs",
"-q", # quiet mode
"-g%dx%d" % size, # set output geometry (pixels)
"-r%d" % (72*scale), # set input DPI (dots per inch)
"-dNOPAUSE -dSAFER", # don't pause between pages, safe mode
"-sDEVICE=ppmraw", # ppm driver
"-sOutputFile=%s" % outfile, # output file
"-c", "%d %d translate" % (-bbox[0], -bbox[1]),
# adjust for image origin
"-f", infile, # input file
]
But on Windows the gs command will be replaced with the path to the executable.
I have a Python script that cleans up and performs basic statistical calculations on a large panel dataset (2,000,000+ observations).
I find that some of these tasks are better suited to Stata, and wrote a do file with the necessary commands. Thus, I want to run a .do file within my Python code. How would I go about calling a .do file from Python?
I think #user229552 points in the correct direction. Python's subprocess module can be used. Below an example that works for me with Linux OS.
Suppose you have a Python file called pydo.py with the following:
import subprocess
## Do some processing in Python
## Set do-file information
dofile = "/home/roberto/Desktop/pyexample3.do"
cmd = ["stata", "do", dofile, "mpg", "weight", "foreign"]
## Run do-file
subprocess.call(cmd)
and a Stata do-file named pyexample3.do, with the following:
clear all
set more off
local y `1'
local x1 `2'
local x2 `3'
display `"first parameter: `y'"'
display `"second parameter: `x1'"'
display `"third parameter: `x2'"'
sysuse auto
regress `y' `x1' `x2'
exit, STATA clear
Then executing pydo.py in a Terminal window works as expected.
You could also define a Python function and use that:
## Define a Python function to launch a do-file
def dostata(dofile, *params):
## Launch a do-file, given the fullpath to the do-file
## and a list of parameters.
import subprocess
cmd = ["stata", "do", dofile]
for param in params:
cmd.append(param)
return subprocess.call(cmd)
## Do some processing in Python
## Run a do-file
dostata("/home/roberto/Desktop/pyexample3.do", "mpg", "weight", "foreign")
The complete call from a Terminal, with results:
roberto#roberto-mint ~/Desktop
$ python pydo.py
___ ____ ____ ____ ____ (R)
/__ / ____/ / ____/
___/ / /___/ / /___/ 12.1 Copyright 1985-2011 StataCorp LP
Statistics/Data Analysis StataCorp
4905 Lakeway Drive
College Station, Texas 77845 USA
800-STATA-PC http://www.stata.com
979-696-4600 stata#stata.com
979-696-4601 (fax)
Notes:
1. Command line editing enabled
. do /home/roberto/Desktop/pyexample3.do mpg weight foreign
. clear all
. set more off
.
. local y `1'
. local x1 `2'
. local x2 `3'
.
. display `"first parameter: `y'"'
first parameter: mpg
. display `"second parameter: `x1'"'
second parameter: weight
. display `"third parameter: `x2'"'
third parameter: foreign
.
. sysuse auto
(1978 Automobile Data)
. regress `y' `x1' `x2'
Source | SS df MS Number of obs = 74
-------------+------------------------------ F( 2, 71) = 69.75
Model | 1619.2877 2 809.643849 Prob > F = 0.0000
Residual | 824.171761 71 11.608053 R-squared = 0.6627
-------------+------------------------------ Adj R-squared = 0.6532
Total | 2443.45946 73 33.4720474 Root MSE = 3.4071
------------------------------------------------------------------------------
mpg | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
weight | -.0065879 .0006371 -10.34 0.000 -.0078583 -.0053175
foreign | -1.650029 1.075994 -1.53 0.130 -3.7955 .4954422
_cons | 41.6797 2.165547 19.25 0.000 37.36172 45.99768
------------------------------------------------------------------------------
.
. exit, STATA clear
Sources:
http://www.reddmetrics.com/2011/07/15/calling-stata-from-python.html
http://docs.python.org/2/library/subprocess.html
http://www.stata.com/support/faqs/unix/batch-mode/
A different route for using Python and Stata together can be found at
http://ideas.repec.org/c/boc/bocode/s457688.html
http://www.stata.com/statalist/archive/2013-08/msg01304.html
This answer extends #Roberto Ferrer's answer, solving a few issues I ran into.
Stata in system path
For stata to run code, it must be correctly set up in the system path (on Windows at least). At least for me, this was not automatically set up on installing Stata, and i found the simplest correction was to put in the full path (which for me was "C:\Program Files (x86)\Stata12\Stata-64) i.e.:
cmd = ["C:\Program Files (x86)\Stata12\Stata-64","do", dofile]`
How to quietly run the code in the background
It is possible to get the code to run quietly in the background (i.e. not opening up Stata each time), by adding the command /e i.e.
cmd = ["C:\Program Files (x86)\Stata12\Stata-64,"/e","do", dofile]
Log file storage location
Finally, if you are running quietly in the background, Stata will will want to save log files. It will do this in cmd's working directory. This must vary depending on where the code is being run from, but for me, since i was executing Python from Notepad++, it wanted to save the log files in C:\Program Files (x86)\Notepad++ , which Stata did not have write-access to. This can be changed by specifying the working directory when the sub-process is called.
These modifications to Roberto Ferrer's code lead to:
def dostata(dofile, *params):
cmd = ["C:\Program Files (x86)\Stata12\Stata-64","/e","do", dofile]
for param in params:
cmd.append(param)
return (subprocess.call(cmd, cwd=r'C:\location_to_save_log_files'))
If you're running this in a command-line setting, you should be able to call Stata from the command line from python (I don't know how to invoke a shell command from within Python, but it shouldn't be too hard, see here: Calling an external command in Python). To run Stata from the command line (aka batch mode), see here: http://www.stata.com/support/faqs/unix/batch-mode/
I am trying to embed Python 2.6 into MATLAB (7.12). I wanted to embed with a mex file written in C. This worked fine for small simple examples using scalars. However, if Numpy (1.6.1) is imported in anyway MATLAB crashes. I say anyway because I have tried a number of ways to load the numpy libraries including
In the python module (.py):
from numpy import *
With PyRun_SimpleString in the mex file:
PyRun_SimpleString(“from numpy import *”);
Calling numpy functions with Py_oBject_CallObject:
pOut = PyObject_CallObject(pFunc, pArgs);
Originally, I thought this may be a problem with embedding Numpy in C. However, Numpy works fine when embedded in simple C files that are compiled from the command line with /MD (multithread) switch with the Visual Studios 2005 C compiler. Next, I thought I will just change the make file in MATLAB to include the /MD switch. No such luck, mexopts.bat compiles with the /MD switch. I also manually commented out lines in the Numpy init module to find what was crashing MATLAB. It seems that loading any file with the extension pyd crashes MATLAB. The first of such files loaded in NumPy is multiarray.pyd. The MATLAB documentation describes how to debug mex files with visual studios which I did and placed the error message below. At this point I know the problem is a memory problem with the pyd’s and some conflict with MATLAB. Interestingly, I can use a system command in MATLAB to kick off a process in python that uses numpy and no error is generated. I will paste below the error message from MATLAB followed by the DEBUG output in visual studios of the processes that crash MATLAB. However, I am not pasting the whole thing because the list of first-chance exceptions is very long. Are there any suggestions for solving this integration problem?
MATLAB error
Matlab has encountered an internal problem and needs to close
MATLAB crash file:C:\Users\pml355\AppData\Local\Temp\matlab_crash_dump.3484-1:
------------------------------------------------------------------------
Segmentation violation detected at Tue Oct 18 12:19:03 2011
------------------------------------------------------------------------
Configuration:
Crash Decoding : Disabled
Default Encoding: windows-1252
MATLAB License : 163857
MATLAB Root : C:\Program Files\MATLAB\R2011a
MATLAB Version : 7.12.0.635 (R2011a)
Operating System: Microsoft Windows 7
Processor ID : x86 Family 6 Model 7 Stepping 10, GenuineIntel
Virtual Machine : Java 1.6.0_17-b04 with Sun Microsystems Inc. Java HotSpot(TM) Client VM mixed mode
Window System : Version 6.1 (Build 7600)
Fault Count: 1
Abnormal termination:
Segmentation violation
Register State (from fault):
EAX = 00000001 EBX = 69c38c20
ECX = 00000001 EDX = 24ae1da8
ESP = 0088af0c EBP = 0088af44
ESI = 69c38c20 EDI = 24ae1da0
EIP = 69b93d31 EFL = 00010202
CS = 0000001b DS = 00000023 SS = 00000023
ES = 00000023 FS = 0000003b GS = 00000000
Stack Trace (from fault):
[ 0] 0x69b93d31 C:/Python26/Lib/site-packages/numpy/core/multiarray.pyd+00081201 ( ???+000000 )
[ 1] 0x69bfead4 C:/Python26/Lib/site-packages/numpy/core/multiarray.pyd+00518868 ( ???+000000 )
[ 2] 0x69c08039 C:/Python26/Lib/site-packages/numpy/core/multiarray.pyd+00557113 ( ???+000000 )
[ 3] 0x08692b09 C:/Python26/python26.dll+00076553 ( PyEval_EvalFrameEx+007833 )
[ 4] 0x08690adf C:/Python26/python26.dll+00068319 ( PyEval_EvalCodeEx+002255 )
This error was detected while a MEX-file was running. If the MEX-file
is not an official MathWorks function, please examine its source code
for errors. Please consult the External Interfaces Guide for information
on debugging MEX-files.
If this problem is reproducible, please submit a Service Request via:
http://www.mathworks.com/support/contact_us/
A technical support engineer might contact you with further information.
Thank you for your help.
Output from Visual Studios DEBUGGER
First-chance exception at 0x0c12c128 in MATLAB.exe: 0xC0000005: Access violation reading location 0x00000004.
First-chance exception at 0x0c12c128 in MATLAB.exe: 0xC0000005: Access violation reading location 0x00000004.
First-chance exception at 0x0c12c128 in MATLAB.exe: 0xC0000005: Access violation reading location 0x00000004.
First-chance exception at 0x751d9673 in MATLAB.exe: Microsoft C++ exception: jitCgFailedException at memory location 0x00c3e210..
First-chance exception at 0x751d9673 in MATLAB.exe: Microsoft C++ exception: jitCgFailedException at memory location 0x00c3e400..
First-chance exception at 0x69b93d31 in MATLAB.exe: 0xC0000005: Access violation writing location 0x00000001.
> throw_segv_longjmp_seh_filter()
throw_segv_longjmp_seh_filter(): invoking THROW_SEGV_LONGJMP SEH filter
> mnUnhandledWindowsExceptionFilter()
MATLAB.exe has triggered a breakpoint
Try to approach the problem from the Python side: Python is a great glue language, I would suggest you to have Python run your Matlab and C programs. Python has:
Numpy
PyLab
Matplotlib
IPython
Thus, the combination is a good alternative for almost any existing Matlab module.
With matlab 2014b a possibility to call python functions directly in m code was added.