I have a file called main.py and based on the user input I want to run the code either from file export.py, either from file import.py. This is how my code from main.py looks:
print("Hi, " + os.getlogin() + "!")
print("Extracting filenames...")
path = input('Tell me the path!\n')
filenames = [x for x in os.listdir(path)]
ID = [f for f in filenames if re.search("(?<=ID)(\d+)", f)]
filestxt = "Files.txt"
idtxt = "ID.txt"
PathFiles = os.path.join(path, filestxt)
PathID = os.path.join(path, idtxt)
file = open(PathFiles, "w+")
file.write('\n'.join(ID))
file.close()
with open(PathID, 'w') as f:
for item in ID:
list = re.findall("(?<=ID)(\d+)", item)
string = ('\n'.join(list))
f.write("%s\n" % string)
key = input("Export or Import(e/i)?:")
if key == 'e':
os.system('python export.py')
When I am hitting the 'e' button Python is running the code from export.py, but when it gets to the line
from main import PathID
instead of importing the variable which I need for the following function
with open(PathID) as f:
for line in f:
...
the code from main.py is running again and again from the beginning and I get the following lines in the console:
"Hi, " + os.getlogin() + "!"
"Extracting filenames..."
'Tell me the path!\n'
"Export or Import(e/i)?:"
All I want in export.py is to tell Python to read the ID.txt file from the path I have specified in the main.py file.
How can I call the function from main.py in export.py without getting this endless loop?
Try to use
if __name__ == '__main__':
before
key = input ("Export or Import (e / i) ?:")
if key == 'e':
os.system ('python export.py')
or before any code that should always be executed at startup.
To expand on Jossnix's answer:
When you execute os.system('python export.py'), you're launching a separate Python interpreter process.
When you execute from main import PathID within export.py, all of the code in main.py is run, and then, once it's finished running, the control flow is returned to export.py and it has access to PathID. The problem is that, as it stands, your main.py asks for user input. So, your main.py is stuck waiting for user input - you have to provide the input again to this new Python interpreter session! Hence, export.py is stuck while trying to import main.
Jossnix's solution works because it ensures that the user input component of main.py does not get run if main.py is being imported from another module, but will be run if main.py is executed as the main script.
I think you should get rid of the os.system('python export.py') line entirely. It's wasteful: you're launching a completely separate Python interpreter session and the print messages in main.py get run again (this is pretty confusing for the end-user!). I'd say you're better off having whatever code you want to run if the user enters the key 'e' wrapped in a function, and then run this function directly from main.py (if the user has indeed entered 'e'). You could do this: Create such a function (f) in export.py, taking a PathID argument. Then, within main.py, from export import f. Finally, if the user entered 'e', run f(PathID, ...).
Related
I am trying to run a python script within another python script. Which will run 10 times and produce 10 outputs.
I want to run program1.py inside program2.py. Now my program1.py was initially taking a C executable inside it and it takes 1 command line argument.
The program1.py looks like below:
import os
import sys
dataset = sys.argv[1]
os.system(f"/home/Dev/c4.5 -u -f {dataset}")
os.system(f"/home/Dev/c4.5rules -u -f {dataset}")
os.system(f"/home/Dev/c4.5rules -u -f {dataset} > Temp")
f = open('Temp')
# Some code
Where c4.5 and c4.5rules are the name of the executable files. To run this I was using python3 program1.py dataset_name
Now I am trying to put this program1.py inside program2.py and I am trying this below approach:
import os
import subprocess
# Some code
for track in range(0, 10):
with open(f'Train_{track}', 'r') as firstfile, open(f'DF_{track}.data', 'w') as secondfile:
for line in firstfile:
secondfile.write(line)
os.system("/home/Dev/program1.py DF_track")
#subprocess.Popen("/home/Dev/program1.py DF_track", shell=True)
Where I simply want to get the output of program1.py 10 times and want to use DF_track as the command line input for each output generation.
Using above approach I am getting lots of error. Please help.
Edit_1 :
Actually whenever I am trying to run, my cursor is not working, it is freezing, so unable to copy the errors.
Here are some of them :
1. attempt to perform an operation not allowed by security policy.
2. syntax error : word expected (expecting ")")
Imagine I have 2 files, the first file is a.py and the other is b.py and I want to call the a.py from b.py.
The content of a.py is:
print('this is the a.py file')
and the content of b.py is:
import os
stream = os.popen('python3 a.py')
output = stream.read()
print(output)
Now when I call b.py from terminal I get the output I expect which is a.py print statment
user#mos ~ % python3 b.py
this is the a.py file
You can do this with subprocess too instead of os module.
Here is a nice blog I found online where I got the code from: https://janakiev.com/blog/python-shell-commands/
See the example below.
a.py
def do_something():
pass
b.py
from a import do_something
do_something()
I have python script like below. In this script, I am collecting stdout and stderr of the script in a file and storing in Linux.
In this script, I am running the function path_finder in a loop over input_file
In this script, I am using subprocess to move data in Linux to a different location.
I want this subprocess call to run after finishing the loop but instead, it runs when the loop runs for the first time and when the second time the loop runs it throws an error which is expected. As the file is present it throws an error.
#!/usr/bin/python
import os
import sys
import traceback
import subprocess
def path_finder(
table,
mysql_user,
user,
dir,
):
day = datetime.now().strftime('%Y-%m-%d')
month = datetime.now().strftime('%Y-%m')
Linux_path = '/data/logging/{}'.format(input_file)
New_path = '/user/{}/{}/logging/{}/{}/{}'.format(user,dir,mysql_user,month,day)
subprocess.call(["rm", Linux_path])
so = se = open('/data/logging/{}'.format(input_file), 'a',
0)
#re-open stdout without buffering
sys.stdout = os.fdopen(sys.stdout.fileno(), 'a', 0)
# redirect stdout and stderr to the log file opened above
os.dup2(so.fileno(), sys.stdout.fileno())
os.dup2(se.fileno(), sys.stderr.fileno())
### CODE:
Do something
### if errors the print traceback
### repeat the same for every table in input file
## Execute below statement after for loop completed
subprocess.call(["cp", Linux_path, New_path])
if len(sys.argv) != 5:
print 'Invalid number of args......'
exit()
input_file = sys.argv[1]
mysql_user = sys.argv[2]
user = sys.argv[3]
dir = sys.argv[4]
input = open("{}.format(input_file)", "r")
for table in input:
path_finder(
table,
mysql_user,
user,
dir,
)
sc.stop()
print
How can I change my script so that the sub process call will run after the for loop is done?
I don't see what the problem is. The statement you want to execute last is currently present in the function 'path_finder' which is why it is running every time.
To make this run only once and after the for loop is finished, put the statement after it.
for table in input:
path_finder(
table,
mysql_user,
user,
dir,
)
subprocess.call(["cp", Linux_path, New_path])
This should do it.
I've been trying to run a Java program and capture it's STDOUT output to a file from the Python script. The idea is to run test files through my program and check if it matches the answers.
Per this and this SO questions, using subprocess.call is the way to go. In the code below, I am doing subprocess.call(command, stdout=f) where f is the file I opened.
The resulted file is empty and I can't quite understand why.
import glob
test_path = '/path/to/my/testfiles/'
class_path = '/path/to/classfiles/'
jar_path = '/path/to/external_jar/'
test_pattern = 'test_case*'
temp_file = 'res'
tests = glob.glob(test_path + test_pattern) # find all test files
for i, tc in enumerate(tests):
with open(test_path+temp_file, 'w') as f:
# cd into directory where the class files are and run the program
command = 'cd {p} ; java -cp {cp} package.MyProgram {tc_p}'
.format(p=class_path,
cp=jar_path,
tc_p=test_path + tc)
# execute the command and direct all STDOUT to file
subprocess.call(command.split(), stdout=f, stderr=subprocess.STDOUT)
# diff is just a lambda func that uses os.system('diff')
exec_code = diff(answers[i], test_path + temp_file)
if exec_code == BAD:
scream(':(')
I checked the docs for subprocess and they recommended using subprocess.run (added in Python 3.5). The run method returns the instance of CompletedProcess, which has a stdout field. I inspected it and the stdout was an empty string. This explained why the file f I tried to create was empty.
Even though the exit code was 0 (success) from the subprocess.call, it didn't mean that my Java program actually got executed. I ended up fixing this bug by breaking down command into two parts.
If you notice, I initially tried to cd into correct directory and then execute the Java file -- all in one command. I ended up removing cd from command and did the os.chdir(class_path) instead. The command now contained only the string to run the Java program. This did the trick.
So, the code looked like this:
good_code = 0
# Assume the same variables defined as in the original question
os.chdir(class_path) # get into the class files directory first
for i, tc in enumerate(tests):
with open(test_path+temp_file, 'w') as f:
# run the program
command = 'java -cp {cp} package.MyProgram {tc_p}'
.format(cp=jar_path,
tc_p=test_path + tc)
# runs the command and redirects it into the file f
# stores the instance of CompletedProcess
out = subprocess.run(command.split(), stdout=f)
# you can access useful info now
assert out.returncode == good_code
I'm working with Django and I'd created two database. Everything seems to work fine, but then I had to edit one of the two and add a column.. From that moment the db wouldn't work anymore, so I exported in a text file the first database and thinking "now I recreate the two db and run a python script to refill the first one". The problem is that whene I try to run the script I get errors, because I can't run the command like bash using os.system, and I don't really know any other way... So, here's my code:
import os
def func ():
try:
FILE=open ("./languagedb.txt", "r")
except IOError:
print 'Can\'t open db file'
exit (1)
for line in FILE:
if (line.startswith('INSERT')):
values=line[43:-1]
language=values[1:3]
values=values[6:]
field=""
fieldBool=True
i=0
while fieldBool:
try:
c=values[i]
except:
print ''
if c != '\'':
field=field+str(c)
i=i+1
else:
fieldBool=False
values=values [(i+3):]
text=""
textBool=True
i=0
while textBool:
try:
c=values[i]
except:
print ''
if c != '\'':
text=text+str(c)
i=i+1
else:
textBool=False
comand="Language.objects.create(language=\""+language+"\", text=\""+text+"\", campo=\""+field+"\")"
os.system(comand)
This is the way I call the shell:
python manage.py shell
and the commands I give it:
import django
from languageMods.models import *
import mymigration #The name fo the file containing the above code
mymigration.func()
And I get the following error, for example
sh: -c: line 0: syntax error near unexpected token `language="en",'
Which is shell's error.
Does someone know how to execute a command from a python script in a python shell?
If you start your script the way you describe it you can just call the django DB API directly in your code:
Language.objects.create(language=language, text=text, campo=field)
I am trying to write a python CLI program using module python cmd. When I try to execute another python script in my CLI program my objective is I have some python script in other folder and CLI program in other folder. I am trying to execute those python script using CLI program.
Below is the os.popen method used to execute other script there is CLI program:
import cmd
import os
import sys
class demo(cmd.Cmd):
def do_shell(self,line,args):
"""hare is function to execute the other script"""
output = os.popen('xterm -hold -e python %s' % args).read()
output(sys.argv[1])
def do_quit(self,line):
return True
if __name__ == '__main__':
demo().cmdloop()
and hare is error:
(Cmd) shell demo-test.py
Traceback (most recent call last):
File "bemo.py", line 18, in <module>
demo().cmdloop()
File "/usr/lib/python2.7/cmd.py", line 142, in cmdloop
stop = self.onecmd(line)
File "/usr/lib/python2.7/cmd.py", line 221, in onecmd
return func(arg)
TypeError: do_shell() takes exactly 3 arguments (2 given)
there is some link to other cmd CLI program
1 = cmd – Create line-oriented command processors
2 = Console built with Cmd object (Python recipe)
and some screen shot's for more information:
Please run above code in your system.
As specified in the doc:
https://pymotw.com/2/cmd/index.html
do_shell is defined as such:
do_shell(self, args):
But you are defining it as
do_shell(self, line, args):
I think the intended use is define it as specified from the documentation.
I ran your code and followed your example. I replicated your error. I then, as specified in the documentation for do_shell, I changed the method to the as expected:
do_shell(self, args):
From there, the sys module was missing, so you need to import that as well (unless you did not copy it from your source). After that, I got an error for index out of range, probably because of the expectation of extra parameters needing to be passed.
Furthermore, because you are talking about Python scripts, I don't see the need for the extra commands you are adding, I simply changed the line to this:
output = os.popen('python %s' % args).read()
However, if there is a particular reason you need the xterm command, then you can probably put that back and it will work for your particular case.
I also, did not see the use case for this:
output(sys.argv[1])
I commented that out. I ran your code, and everything worked. I created a test file that just did a simple print and it ran successfully.
So, the code actually looks like this:
def do_shell(self, args):
"""hare is function to execute the other script"""
output = os.popen('python %s' % args).read()
print output
The full code should look like this:
import cmd
import os
import sys
class demo(cmd.Cmd):
def do_shell(self, args):
"""hare is function to execute the other script"""
output = os.popen('python %s' % args).read()
print output
def do_quit(self,line):
return True
if __name__ == '__main__':
demo().cmdloop()