I have the following code, which writes an Input file and executes ANSYS Mechanical APDL using a Windows command. My issue is the execution time is much longer (15 minutes inside the software, upwards of 1 hour when calling from Python). I need it to be faster because I am varying as many as 'all' the input parameters.
def RunAPDL(E,t,w,p,aa,bb,lz,alpha,delta):
ansyspath = r'C:\Program Files\ANSYS.Inc\v181\ansys\bin\winx64\MAPDL.exe'
directory = r'C:\Users\Erik\Documents\ANSYS'
jobname = 'file'
memory = '4096'
reserve = '1024'
inputfile = r'C:\Users\Erik\Documents\ANSYS\ShellBucklingInput.inp'
outputfile = r'C:\Users\Erik\Documents\ANSYS\OutputFile.txt'
resultsfile = r'C:\Users\Erik\Documents\ANSYS\ShellBuckling.csv'
# Start time count
start = time.clock()
# Write input file
input_parameters = ('/NERR,200,10000,,OFF,0 \n'
'pi = acos(-1) \n'
'E = {:6.0f} ! N/mm2 Young\'s modulus\n'
't = {:4.2f} ! mm thickness\n'
'w = {:3.2f} ! Poisson\'s ratio\n'
'p = {:7.2f} ! N/mm2 external pressure\n'
'aa = {:6.2f} ! mm horizontal radius at u = 0\n'
'bb = {:6.2f} ! mm vertical radius\n'
'lz = {:6.2f} ! mm model height\n'
'lu = 2*asin(lz/2/bb) !mm \n'
'lv = 2*pi ! model perimeter at u = 0 \n'
'nu = 2*NINT(ABS(aa)/SQRT(ABS(aa)*t)) \n'
'nv = 3*nu ! number of elements along v-axis \n'
'alpha = {:4.2f} ! ratio of lu that is not loaded by p \n'
'delta = {:4.2f}*t ! prescribed imperfection magnitude \n'
'*ULIB,ShellBucklingLibrary,mac \n'
'*USE,ShellBuckling,pi,E,t,w,p,aa,bb,lz,lu,nu,nv,alpha,delta \n'
'/CLEAR'
).format(E,t,w,p,aa,bb,lz,alpha,delta)
with open(inputfile,'w') as f:
f.write(input_parameters)
# Call ANSYS
callstring = ('\"{}\" -p aa_t_a -dir \"{}\" -j \"{}\" -s read'
' -m {} -db {} -t -d win32 -b -i \"{}\" -o \"{}\"'
).format(ansyspath,directory,jobname,memory,reserve,inputfile,outputfile)
print('Invoking ANSYS with', callstring)
proc = subprocess.Popen(callstring).wait()
# Update pressure field for next analysis
with open(resultsfile,'r') as f:
lambdaS = float(list(csv.reader(f))[-1][16])
p = 1.2*lambdaS*p
print('Updated pressure is',p,' N/mm2.')
# Stop time count
stop = time.clock()
print('Elapsed time is ',stop-start,' seconds.')
return(p)
I execute it by pressing F5 on the Python shell, then messages appear in the shell:
Invoking ANSYS with "C:\Program Files\ANSYS
Inc\v181\ansys\bin\winx64\MAPDL.exe" -p aa_t_a -dir
"C:\Users\Erik\Documents\ANSYS" -j "file" -s read -m 4096 -db 1024 -t -d
win32 -b -i "C:\Users\Erik\Documents\ANSYS\ShellBucklingInput.inp" -o
"C:\Users\Erik\Documents\ANSYS\OutputFile.txt"
Updated pressure is -0.0046478399999999994 N/mm2.
Elapsed time is 4016.59094467131 seconds.
I don't know why your code is slower in Python but I suggest you to use PyAnsys package from PyPI. It offers a robust infrastructure to combine Python and APDL.
Related
On a CentOS 7.2 I have a file called cpuload, which contains the latest CPU load data in the following format:
last 30 sec:
average load: 0
cpu0 total load: 0
cpu1 total load: 0
cpu2 total load: 0
cpu3 total load: 1
cpu4 total load: 0
cpu5 total load: 0
cpu6 total load: 0
cpu7 total load: 0
last sec:
average load: 1
cpu0 total load: 5
cpu1 total load: 1
cpu2 total load: 1
cpu3 total load: 3
cpu4 total load: 2
cpu5 total load: 1
cpu6 total load: 0
cpu7 total load: 0
I want to get the number after the "average load:" of the "last sec" bit.
Two cli commands give me that information when I run them as shell commands on the terminal:
grep 'average load:' cpuload | sed -n 's/.*load: //p' | tail -n1
and
awk 'NR > 2 && /average load:/ {print $3}' cpuload
But when I run them in subprocess.Popen() with Shell=True I only get stderr:
for:
import subprocess
cmd = ["grep", "'average load:'", "cpuload", "|", "sed", "-n", "'s/.*load: //p'", "|", "tail", "-n1"]
test = subprocess.Popen(cmd, stdout = subprocess.PIPE, stderr = subprocess.PIPE, shell=True)
test.stderr.read()
I get:
b"Usage: grep [OPTION]... PATTERN [FILE]...\nTry 'grep --help' for more information.\n"
and for:
import subprocess
cmd = cmd = ["awk", "'NR > 2 && /average load:/ {print $3}'", "cpuload"]
test = subprocess.Popen(cmd, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
test.stderr.read()
I also get:
b"awk: cmd. line:1: 'NR > 2 && /average load:/ {print $3}'\nawk: cmd. line:1: ^ invalid char ''' in expression\n"
Even though I avoided using a |
or if shell=True I get:
b"Usage: awk [POSIX or GNU style options] -f progfile [--] file ...\nUsage: awk [POSIX or GNU style options] [--] 'program' file ...\nPOSIX options:\t\tGNU long options: (standard)\n\t-f progfile\t\t--file=progfile\n\t-F fs\t\t\t--field-separator=fs\n\t-v var=val\t\t--assign=var=val\nShort options:\t\tGNU long options: (extensions)\n\t-b\t\t\t--characters-as-bytes\n\t-c\t\t\t--traditional\n\t-C\t\t\t--copyright\n\t-d[file]\t\t--dump-variables[=file]\n\t-e 'program-text'\t--source='program-text'\n\t-E file\t\t\t--exec=file\n\t-g\t\t\t--gen-pot\n\t-h\t\t\t--help\n\t-L [fatal]\t\t--lint[=fatal]\n\t-n\t\t\t--non-decimal-data\n\t-N\t\t\t--use-lc-numeric\n\t-O\t\t\t--optimize\n\t-p[file]\t\t--profile[=file]\n\t-P\t\t\t--posix\n\t-r\t\t\t--re-interval\n\t-S\t\t\t--sandbox\n\t-t\t\t\t--lint-old\n\t-V\t\t\t--version\n\nTo report bugs, see node `Bugs' in `gawk.info', which is\nsection `Reporting Problems and Bugs' in the printed version.\n\ngawk is a pattern scanning and processing language.\nBy default it reads standard input and writes standard output.\n\nExamples:\n\tgawk '{ sum += $1 }; END { print sum }' file\n\tgawk -F: '{ print $1 }' /etc/passwd\n"
What am I doing wrong?
I have a file called cpuload, which contains the latest CPU load data ...I want to get the number after the "average load:" of the "last sec" bit
why not just to use simple python code in order to get the value you are looking for?
with open('cpuload') as f:
lines = [l.strip() for l in f.readlines()]
got_it = False
for line in lines:
if got_it:
parts = line.split(':')
result = parts[-1].strip()
print(result)
break
if line == 'last sec:':
got_it = True
output
1
First case with grep, sed, tail... and pipes.
You need to use shell = True parameter for Popen method and a single string for the command. We need to put cotes around parameters:
import subprocess
cmd = "grep 'average load:' cpuload | sed -n 's/.*load: //p' | tail -n1"
output = ""
test = subprocess.Popen(cmd, stdout = subprocess.PIPE, stderr = subprocess.PIPE, shell = True)
while True:
output += test.stdout.readline().decode("utf-8")
if test.poll() is not None:
break
print("output=<%s>" % (output))
Second case, without pipe:
You don't need to use shell = True parameter for Popen method and a single string for the command. We don't put cotes around parameters:
import subprocess
cmd = ["/usr/bin/awk", "NR > 2 && /^average load:/ {print $3}", "cpuload"]
output = ""
test = subprocess.Popen(cmd, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
while True:
output += test.stdout.readline().decode("utf-8")
if test.poll() is not None:
break
print("output=<%s>" % (output))
The issue was with passing the awk parameter to subprocess with ' around them, as detailed here
Did not accept Ed Morton's comment as it did not specify what should have been done.
Hi I'm currently working on a python script that generates shell scripts to install agents on a linux server. The .sh files that the python scripts output keeps ending up with a "syntax error: unexpected end of file" but when i manually type in the exact output in vi, there seems to be no issue. Is there any issue with how I'm writing it in python or is it feasible to do it through python?
python script
import csv
def menu():
print("type of scripts")
print("1. Install + Generation")
print("2. Unregister + Reregister")
print("3. Unregister + Uninstall")
#Converts numeral choice into type for script naming
def choicename(choice):
choice = int(choice)
if choice==1:
return "install"
elif choice == 2 :
return "rereg"
else:
return "uninstall"
#Generates the install agent scripts
def installScript(agentname,agentfile,mgrfile,prigw,secgw,ostype):
#Generates the script for Linux agents (.sh)
if ostype=="Linux":
agentpath = 'agent="/opt/test/ragent/bin/racli"'
installerpath = '\ninstaller="/opt/test/installer/bin/racli"'
checkAgent = '\nif [ ! -e "$agent" ]; then' +"\n" + "./" + agentfile + " -n -d /opt/test" + '\nelse\necho "Agent is already installed"\nfi'
checkInstaller = '\nif [ ! -e "$installer" ]; then' + "\n" +"./" + mgrfile + " -n -d /opt/test"+ '\nelse\necho "Manager is already installed"\nfi'
regAgent = "\n/opt/test/ragent/bin/cli registration advanced-register registration-type=Primary ragent-name="+ agentname+ " gw-ip="+ prigw+ " gw-port=443 manual-settings-activation=Automatic monitor-networkchannels=Both password=$1"
if secgw!="":
regAgent+="\n/opt/test/ragent/bin/cli registration advanced-register registration-type=Secondary ragent-name="+ agentname+ " gw-ip="+ secgw+ " gw-port=443 manual-settings-activation=Automatic monitor-networkchannels=Both password=$1"
startAgent="\n/opt/test/ragent/bin/rainit start"
regInstaller="\n/opt/test/installer/bin/cliinstaller registration register-use-existing package-folder-path=\".\" package-folder-size=1024"
startInstaller="\n/opt/test/installer/bin/rainstallerinit start"
sf = open(agentname+ "_install.sh","w")
sf.write(agentpath+installerpath+checkAgent+checkInstaller+regAgent+startAgent+regInstaller+startInstaller)
sf.close()
def scriptSplit(option,agentname,agentfile,mgrfile,prigw,secgw,ostype):
if option=="install":
installScript(agentname,agentfile,mgrfile,prigw,secgw,ostype)
elif option =="rereg":
reregScript(agentname,agentfile,mgrfile,prigw,secgw,ostype)
elif option =="uninstall":
uninstallScript()
#Collects user input for function type
def main():
menu()
choice = input("Please choose the type of script you would like to generate: ")
option = choicename(choice)
filename = input("Please enter the name of the csv file: ")
with open(filename) as csv_file:
creader = csv.reader(csv_file, delimiter=",")
line_count = 0
for row in creader:
if line_count!=0:
agentname=row[0]
agentfile=row[1]
mgrfile=row[2]
prigw=row[3]
secgw=row[4]
installtype=row[5]
scriptSplit(option,agentname,agentfile,mgrfile,prigw,secgw,installtype)
line_count += 1
#### END OF FUNCTIONS ###
main()
output from above script
agent="/opt/test/ragent/bin/racli"
installer="/opt/test/installer/bin/racli"
if [ ! -e "$agent" ]; then
./agent1.bsx -n -d /opt/test
else
echo "Agent is already installed"
fi
if [ ! -e "$installer" ]; then
./installer1.bsx -n -d /opt/test
else
echo "Manager is already installed"
fi
/opt/test/ragent/bin/cli registration advanced-register registration-type=Primary ragent-name=agent1 gw-ip=10.255.0.80 gw-port=443 manual-settings-activation=Automatic monitor-networkchannels=Both password=$1
/opt/test/ragent/bin/cli registration advanced-register registration-type=Secondary ragent-name=agent1 gw-ip=10.255.0.81 gw-port=443 manual-settings-activation=Automatic monitor-networkchannels=Both password=$1
/opt/test/ragent/bin/rainit start
/opt/test/installer/bin/cliinstaller registration register-use-existing package-folder-path="." package-folder-size=1024
/opt/test/installer/bin/rainstallerinit start
csv file it reads from
Agent Name,Agent File,Installer File,Primary IP,Secondary IP,Type
agent1,agent1.bsx,installer1.bsx,10.255.0.80,10.255.0.81,Linux
agent2,agent2.bsx,installer2.bsx,10.255.0.81,,Linux
The newline convention is \r\n on windows. Bash interprets newline character as... newline character, and the \r is a normal character for bash.
Fix:
sf = open(agentname+ "_install.sh", "w", newline='\n')
I created a script to download a youtube video and extract an images from it for each period
screenshotvideo
def screenshotvideo(url, interval, id, fullduration, title, quality):
interval = int(interval)
parsed_t = isodate.parse_duration(fullduration)
durationseconds=parsed_t.total_seconds()
iterat=int(durationseconds/int(interval))
for i in range(0, iterat):
print(str(id))
print(str(i))
print(str(i*interval))
part(url, time.strftime('%H:%M:%S', time.gmtime(int(i*interval))), "00:00:01", title+"-"+str(id), quality)
part
def part(url, starttime, duration, name, quality):
f = os.popen("ffmpeg $(youtube-dl -f "+quality+" -g '"+url+"' | sed 's/.*/-ss "+starttime+" -i &/') -t "+duration+" -c copy "+name+".mp4")
now = f.read()
print(now)
f = os.popen("ffmpeg -i "+name+".mp4 -ss 00:00:00 -vframes 1 "+name+".jpg")
now = f.read()
print(now)
f = os.popen("rm -rf "+name+".mp4")
now = f.read()
print(now)
so i got the folowing error in the first ffmpeg command
[mp4 # 0x55d537f64240] Could not find tag for codec vp8 in stream #0, codec not currently supported in container.
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
The error is pretty clear:
codec not currently supported in container.
Use a different container that supports vp8, like webm or mkv.
The answer is close to szatmary answer but i need to download video in mp4 format, so all i neet to do is to edit the first command
ffmpeg $(youtube-dl -f "+quality+" -g '"+url+"' | sed 's/.*/-ss "+starttime+" -i &/') -t "+duration+" -c:v libx264 "+name+".mp4
after that i got other error for the images file name that i want to saved each time from the video.
So this is the final solution i found:
def screenshotvideo(url, interval, id, fullduration, title, quality):
interval = int(interval)
parsed_t = isodate.parse_duration(fullduration)
durationseconds=parsed_t.total_seconds()
iterat=int(durationseconds/int(interval))
for i in range(0, iterat):
print(str(id))
print(str(i))
print(str(i*interval))
part(url, time.strftime('%H:%M:%S', time.gmtime(int(i*interval))), "00:00:01", title+"-"+str(id), quality, i)
def part(url, starttime, duration, name, quality, i):
f = os.popen("ffmpeg $(youtube-dl -f "+quality+" -g '"+url+"' | sed 's/.*/-ss "+starttime+" -i &/') -t "+duration+" -c:v libx264 "+name+".mp4")
now = f.read()
print(now)
f = os.popen("ffmpeg -i "+name+".mp4 -ss 00:00:00 -vframes 1 "+name+"_"+str(i)+".jpg")
now = f.read()
print(now)
f = os.popen("rm -rf "+name+".mp4")
now = f.read()
print(now)
I have a python script conve_cal.py which take 7 arguments as follow:
enter code here
import numpy as np
import sys
def read_file(filename1,filename2):
Input1 = np.loadtxt(filename1)
Input2 = np.loadtxt(filename2)
#print type(Input1)
#print Input1
return Input1,Input2
def NU_calc(T_avg,Trght,Ttop,rmv,argv1,argv2,arg3):
....
if __name__ == "__main__":
T_avg = sys.argv[1]
Trght = sys.argv[2]
Ttop = sys.argv[3]
rmv = sys.argv[4]
arg3 = sys.argv[5]
file1 = sys.argv[6]
file2 = sys.argv[7]
print (sys.argv)
In1,In2 = read_file(file1,file2)
#print type(arg3)
NU_calc(T_avg,Trght,Ttop,rmv,In1,In2,arg3)
If I run the code stand alone in the Terminal as
python2 conve_calc.py 285 282 300 0 noclip Input1.dat Input2.dat
the code give the output as desired. But If i use the same syntax with bash variables in the bash script as follows;
#!/bin/bash
...
...
t_avg=`grep "Average" test.avg |cut -f6 -d" "`
t_right=`grep "Tright" ./0/Input_conditions|cut -f 3 |cut -f 1 -d';'`
t_top=`grep "Ttop" ./0/Input_conditions|cut -f 3 |cut -f 1 -d';'`
echo "$t_right $t_top $Hr $Ht" >> $start_path/post_process_data/'HT_stat.dat'
rm test.avg tmp.dat test_fl.txt
# Call
# Arg-1; Average Temp
## Arg-2; Temp of right wall
## Arg-3; Temp of top wall
## Arg-4; # of faces to remove (0:None ,1...upto max of no. of faces)
## Arg-5; Right GradT file
## Arg-6; Top GradT file
rv0="0" # Change this to Remove the faces
rv1="1"
c1="noclip"
c2="middle"
i1="Input1.dat"
i2="Input2.dat"
python2 $start_path/conve_calc.py $t_avg $t_right $t_top $rv1 $c1 $i1 $i2 >> $start_path/post_process_data/\
'Common_out.dat'
But with this bash script input I am getting Following error I am unable to find why these inputs are getting wrong.
Traceback (most recent call last):
File "/home/meisu/OpenFOAM/meisu-2.4.0/Cyence_data/foam_adagio/conve_calc.py", line 69, in <module>
file2 = sys.argv[7]
IndexError: list index out of range
I have looked into various stack solutions but none of them worked.
Your problem is that one of the variables is empty. The shell expands the variables and then parses the command so it thinks there are only 6 parameters for the command. You could add two types of defensive coding. First, for the derived variables likely to fail, test them before you use them. Second, enclose your parameters in quotes so that empty parameters, or parameters with characters such as spaces that will mess up the command parsing, will be handed down properly.
#!/bin/bash
...
...
t_avg=`grep "Average" test.avg |cut -f6 -d" "`
if [ -z "$t_avg" ]; then
echo t_avg failed >&2
exit 2
fi
t_right=`grep "Tright" ./0/Input_conditions|cut -f 3 |cut -f 1 -d';'`
if [ -z "$t_right" ]; then
echo t_right failed >&2
exit 2
fi
t_top=`grep "Ttop" ./0/Input_conditions|cut -f 3 |cut -f 1 -d';'`
if [ -z "$t_top" ]; then
echo t_top failed >&2
exit 2
fi
echo "$t_right $t_top $Hr $Ht" >> $start_path/post_process_data/'HT_stat.dat'
rm test.avg tmp.dat test_fl.txt
# Call
# Arg-1; Average Temp
## Arg-2; Temp of right wall
## Arg-3; Temp of top wall
## Arg-4; # of faces to remove (0:None ,1...upto max of no. of faces)
## Arg-5; Right GradT file
## Arg-6; Top GradT file
rv0="0" # Change this to Remove the faces
rv1="1"
c1="noclip"
c2="middle"
i1="Input1.dat"
i2="Input2.dat"
python2 $start_path/conve_calc.py "$t_avg" "$t_right" "$t_top" "$rv1" "$c1" "$i1" "$i2" >> "$start_path/post_process_data/Common_out.dat"
You can see exactly what is sent to your python script with a toy script that just prints its arguments and exits
!#/usr/bin/env python
import sys
for i, arg in enumerate(sys.argv):
print i, arg
print '---- end ----'
I have a perl script that can be executed from the console as follows:
perl perlscript.pl -i input.txt -o output.txt --append
I want to execute this script from my python code. I figured out that subprocess.Popen can be used to connect to perl and I can pass my arguments with it. But, I also want to pass a variable (made by splitting up a text file) in place of input.txt.
I have tried the following but it doesn't seem to work and gives an obvious TypeError in line 8:
import re, shlex, subprocess, StringIO
f=open('fulltext.txt','rb')
text= f.read()
l = re.split('\n\n',str(text))
intxt = StringIO.StringIO()
for i in range(len(l)):
intxt.write(l[i])
command_line='perl cnv_ltrfinder2gff.pl -i '+intxt+' -o output.gff --append'
args=shlex.split(command_line)
p = subprocess.Popen(args)
Is there any other work around for this?
EDIT: Here is a sample of the file fulltext.txt. Entries are separated by a line.
Predict protein Domains 0.021 second
>Sequence: seq1 Len:13143 [1] seq1 Len:13143 Location : 9 - 13124 Len: 13116 Strand:+ Score : 6 [LTR region similarity:0.959] Status : 11110110000 5'-LTR : 9 - 501 Len: 493 3'-LTR : 12633 - 13124 Len: 492 5'-TG : TG , TG 3'-CA : CA , CA TSR : NOT FOUND Sharpness: 1,1 Strand + : PBS : [14/20] 524 - 543 (LysTTT) PPT : [12/15] 12553 - 12567
Predict protein Domains 0.019 second
>Sequence: seq5 Len:11539 [1] seq5 Len:11539 Location : 7 - 11535 Len: 11529 Strand:+ Score : 6 [LTR region similarity:0.984] Status : 11110110000 5'-LTR : 7 - 506 Len: 500 3'-LTR : 11036 - 11535 Len: 500 5'-TG : TG , TG 3'-CA : CA , CA TSR : NOT FOUND Sharpness: 1,1 Strand + : PBS : [15/22] 515 - 536 (LysTTT) PPT : [11/15] 11020 - 11034
I want to separate them and pass each entry block to the perl script. All the files are in the same directory.
you might be interested in the os module
and string formatting
Edit
I think I uderstand what you want now. correct me if I am wrong, but I think:
You want to split your fulltext.txt into blocks.
Every block contains a seq(number)
You want to run your perl script once for every block with as input file your seq(number)
if this is what you want, you could use the following code.
import os
in_file = 'fulltext.txt'
seq = []
with open(in_file,'r') as handle:
lines = handle.readlines()
for i in range(0,len(lines)):
if lines[i].startswith(">"):
seq.append(lines[i].rstrip().split(" ")[1])
for x in seq:
command = "perl perl cnv_ltrfinder2gff.pl -i %s.txt -o output.txt --append"%x
os.system(command)
The docs for --infile option:
Path of the input file. If an input file is not provided, the program
will expect input from STDIN.
You could omit --infile and pass input via a pipe (stdin) instead:
#!/usr/bin/env python
from subprocess import Popen, PIPE
with open('fulltext.txt') as file: # read input data
blocks = file.read().split('\n\n')
# run a separate perl process for each block
args = 'perl cnv_ltrfinder2gff.pl -o output.gff --append'.split()
for block in blocks:
p = Popen(args, stdin=PIPE, universal_newlines=True)
p.communicate(block)
if p.returncode != 0:
print('non-zero exit status: %s on block: %r' % (p.returncode, block))
You can run several perl scripts concurrently:
from multiprocessing.dummy import Pool # use threads
def run((i, block)):
filename = 'out%03d.gff' % i
args = ['perl', 'cnv_ltrfinder2gff.pl', '-o', filename]
p = Popen(args, stdin=PIPE, universal_newlines=True, close_fds=True)
p.communicate(block)
return p.returncode, filename
exit_statuses, filenames = zip(*Pool().map(run, enumerate(blocks, start=1)))
It runs several (equal to the number of CPUs on your system) child processes in parallel. You could specify a different number of worker threads (pass to Pool()).