I'm trying to do everything I would have previously used Bash for in Python, in order to finally learn the language. However, this problem has me stumped, and I haven't been able to find any solutions fitting my use-case.
There is an element of trying to run before I can walk with though, so I'm looking for some direction.
Here's the issue:
I have a Python script that starts a separate program that creates and writes to a log file.
I want to watch that log file, and print out "Successful Run" if the script detects the "Success" string in the log, and "Failed Run" if the "Failed" string is found instead. The underlying process generally takes about 10 seconds to get to the stage where it'll write "Success" or "Failure" to the log file. Neither string will appear in the log at the same. It's either a success, or failure. It can't be both.
I've been attempting to do this with a while loop. So I can continue to watch the log file, until the string appears, and then exit when it does. I have got it working for just one string, but I'm unsure how to accomodate the other string.
Here's the code I'm running.
log_path = "test.log"
success = "Success"
failure = "Failed"
with open(log_path) as log:
while success != True:
if success in log.read():
print("Process Successfully Completed")
sys.exit()
Thanks to the pointers above from alaniwi and David, I've actually managed to get it to work, using the following code. So I must have been quite close originally.
I've wrapped it all in a while True, put the log.read() into a variable, and added an elif. Definitely interested in any pointers on whether this is the most Pythonic way to do it though? So please critique if need be.
while True:
with open(log_path) as log:
read_log = log.read()
if success in read_log:
print("Process Successfully Completed")
sys.exit()
elif fail in read_log:
print("Failed")
sys.exit()
Related
I guess I'm not the first asking this question, but I haven't found a solution that I could use/understand yet. And the issue is probably not as simple as i first expected.
I think it can be boiled down to two general questions:
1) Is there a way to avoid Python to stop when an error occur and just jump on to the next line of code in the script?
2) Is there a way to make Python execute a line of code if an error occurs? Like, if error then...
My concrete problem:
I have a very large program with a lot of functions and other stuff, which would take forever to adjust individually by using "try" for example (if i understand it correctly)
My program run as a large loop that gather information and keeps running. This means that it does not really matter to me, that my program fails multiple time as long as it keeps running. I can easily handle that some of the information is with error and would just like my program to take a note of it and keep going.
Is there a solution to this?
As you rightly pointed out, the try/catch block in Python is by far your best ally:
for i in range(N):
try: do_foo() ; except: do_other_foo()
try: do_bar() ; except: do_other_bar()
Alternatively, you could also use, in case you didn't need the Exception:
from contextlib import suppress
for i in range(N):
with suppress(Exception):
do_foo()
with suppress(Exception):
do_bar()
Your only possibility is to rely on the try/except clause. Keep in mind that the try/except may use also finally and else (see documentation:
try:
print("problematic code - error NOT raised")
except:
print("code that gets executed only if an error occurs")
else:
print("code that gets executed only if an error does not occur")
finally:
print("code that gets ALWAYS executed")
# OUTPUT:
# problematic code - error NOT raised
# code that gets executed only if an error does not occur
# code that gets ALWAYS executed
or, when an error is raised:
try:
print("problematic code - error raised!")
raise "Terrible, terrible error"
except:
print("code that gets executed only if an error occurs")
else:
print("code that gets executed only if an error does not occur")
finally:
print("code that gets ALWAYS executed")
# OUTPUT:
# problematic code - error raised!
# code that gets executed only if an error occurs
# code that gets ALWAYS executed
I urge to point out, by the way, that ignoring everything makes me shiver:
you really should (at least, more or less) identify which exception can be raised, catch them (except ArithmeticError: ..., check built-in exceptions) and handle them individually. What you're trying to do will probably snowball into an endless chain of problems, and ignoring them will probably create more problems!
I think that this question helps to understand what a robust software is, meanwhile on this one you can see how SO community thinks python exceptions should be handled
Here is the goal: a parser that reunites some information from some domains and organize them into one place.
I am a newbie with Python, having chosen to do this job with this language because of learning curve and things.
For the matter, I am doing the parsing with BeautifulSoup lib and that works like a charm. The routine is triggered via crontab in a CentOS 6, Python 2.7.
However, one of my parsing scripts sent me a log with memory error, what was causing the py file to quit without complete its job. Google here and there and found out that some very long html Python parsing would be doing my server ran out of memory. It would be better close, decompose and even garbage collect everything script would not be using there anymore.
Implemented the three things, no more memory errors in the crontab task. However, every time the script runs, I receive an email from crontab with the log of the parsing, what means that something went wrong there. Checking the databank, all the information was recorded alright, script also completed the entire task, still some error occurred, or crontab would not email me with a log.
In fact, when I run the script directly at the terminal on server, same occurs: the script won’t conclude, unless I ctrl+c it, it will be frozen in the screen. However, again, looking at the bank, all the tasks where completed without a error.
I tried work only with gc, tried only close() and only release(). Any of these three resources would freeze the screen/generate a log error (however without a error explicitaly in it).
Here is a simple version of what I am doing to better understanding:\
class GrabCategories():
def __init__(self):
target = 'http://provider-site.com/info.html'
try:
page = urllib2.urlopen(target)
if page.getcode() == 404:
print 'Page not found', target
return False
soup = BeautifulSoup(page.read())
page.close() #not using this anymore, may I close it?
except:
print 'Could not open', target
return
content = soup.find('div', {'id': 'box-content'})
soup.decompose() #not using this anymore, may I decompose it?
c=0
for link in content.findAll('a'):
#define some vars
try:
catPage = urllib2.urlopen(link['a'])
if catPage.getcode() == 404:
print 'Page not found', catPage
return False
catSoup = BeautifulSoup(catPage.read())
catPage.close() #not using this anymore, may I close it?
except:
print 'Could no open', target
continue
#do some things with the page content etc
catSoup.decompose() #not using this anymore, may I decompose it?
if(c%10):
gc.collect()
c=c+1
I have a script runReports.py that is executed every night. Suppose for some reason the script takes too long to execute, I want to be able to stop it from terminal by issuing a command like ./runReports.py stop.
I tried to implement this by having the script to create a temporary file when the stop command is issued.
The script checks for existence of this file before running each report.
If the file is there the script stops executing, else it continues.
But I am not able to find a way to make the issuer of the stop command aware that the script has stopped successfully. Something along the following lines:
$ ./runReports.py stop
Stopping runReports...
runReports.py stopped successfully.
How to achieve this?
For example if your script runs in loop, you can catch signal http://en.wikipedia.org/wiki/Unix_signal and terminate process:
import signal
class SimpleReport(BaseReport):
def __init__(self):
...
is_running = True
def _signal_handler(self, signum, frame):
is_running = False
def run(self):
signal.signal(signal.SIGUSR1, self._signal_handler) # set signal handler
...
while is_running:
print("Preparing report")
print("Exiting ...")
To terminate process just call kill -SIGUSR1 procId
You want to achieve inter process communication. You should first explore the different ways to do that : system V IPC (memory, very versatile, possibly baffling API), sockets (including unix domain sockets)(memory, more limited, clean API), file system (persistent on disk, almost architecture independent), and choose yours.
As you are asking about files, there are still two ways to communicate using files : either using file content (feature rich, harder to implement), or simply file presence. But the problem using files, is that is a program terminates because of an error, it may not be able to write its ended status on the disk.
IMHO, you should clearly define what are your requirements before choosing file system based communication (testing the end of a program is not really what it is best at) unless you also need architecture independence.
To directly answer your question, the only reliable way to know if a program has ended if you use file system communication is to browse the list of currently active processes, and the simplest way is IMHO to use ps -e in a subprocess.
Instead of having a temporary file, you could have a permanent file(config.txt) that has some tags in it and check if the tag 'running = True'.
To achieve this is quiet simple, if your code has a loop in it (I imagine it does), just make a function/method that branches a check condition on this file.
def continue_running():
with open("config.txt") as f:
for line in f:
tag, condition = line.split(" = ")
if tag == "running" and condition == "True":
return True
return False
In your script you will do this:
while True: # or your terminal condition
if continue_running():
# your regular code goes here
else:
break
So all you have to do to stop the loop in the script is change the 'running' to anything but "True".
I am trying to use pyserial to talk interact with a board and I am sending some commands and waiting for their reply am using this piece of code for waiting for => prompt
# Waits till it finds the => prompt
def s_uboot_prompt(ser):
value = ""
matcher = re.compile("\=\>")
trymatcher = re.compile("try")
while (not matcher.search(value)):
if( trymatcher.search(value.strip()) ):
print "******* Command failure. Exiting ..... **********"
ser.write("boot\r")
ser.close()
sys.exit(0)
value = ser.readline()
print value
time.sleep(2)
It works for all of the command but for one of the tftp commands where the output is really huge it fails to read the last couple of lines and the above code loops forever. is there any reason this is happening ? I am not able to think of any logical reason. Please note it's not able to read couple of last lines.
thnks
I have a python script that invokes the following command:
# make
After make, it also invokes three other programs. Is there a standard way of telling whether the make command was successful or not? Right now, if make is successful or unsuccessful, the program still continues to run. I want to raise an error that the make was not possible.
Can anyone give me direction with this?
The return value of the poll() and wait() methods is the return code of the process. Check to see if it's non-zero.
Look at the exit code of make. If you are using the python module commands, then you can get the status code easily. 0 means success, non-zero means some problem.
import os
if os.system("make"):
print "True"
else:
print "False"
Use subprocess.check_call(). That way you don't have to check the return code yourself - an Exception will be thrown if the return code was non-zero.