I am facing this problem while I try to loop tweet_id using the API and write it to tweet_json.txt, the output for all data is Failed which I know is wrong
Before it was working good but when I try to Run all the code again it starts to show failed
for tweet_id in df['tweet_id']:
try:
tweet = api.get_status(tweet_id, tweet_mode = 'extended')
with open('tweet_json.txt', 'a+') as file:
json.dump(tweet._json, file)
file.write('\n')
print (tweet_id, 'success')
except:
print (tweet_id, 'Failed')
Your except is swallowing whatever exception is causing your code to die. Until you comment out the except or make it more specific you won't know if your problem is the Twitter API or file I/O or something else. Good luck!
A quick step forward would be to adjust your exception handler so that it writes the exception. I like to use the format_exc function to get my stack traces so i can write it with a logger, or however i want to handle it.
from traceback import format_exc
try:
a = "" + 1
except Exception as ex:
print("Exception encountered! \n %s " % format_exc())
Related
try:
msg_json = json.loads(message_string)
if "task" in msg_json:
job_type = msg_json["task"]
return (job_type, msg_json)
logger.error(
"Could not parse message: must provide 'task' property",
extra={"message_string": message_string},
)
return empty
except Exception:
logger.exception(
"Error parsing JSON message. Did you accidentally double-escape it?",
extra={"message_string": message_string},
)
return empty
I have this code where i am trying to load some JSON formatted message string. After looking back at this piece of code i feel i maybe using try and catch in the wrong way and i was looking for suggestions as i am new to python and there may be cleaner approaches. There is no bug here but this is more for me to learn the "Cleaner" approach. As such i am open to all suggestions that explain the cleaner and more correct approach.
You could handle both of your error cases in catch blocks, which makes your "happy path" code a bit cleaner and neatly groups all the error handling in one place:
try:
msg_json = json.loads(message_string)
return (msg_json["task"], msg_json)
except KeyError:
logger.error(
"Could not parse message: must provide 'task' property",
extra={"message_string": message_string},
)
return empty
except Exception:
logger.exception(
"Error parsing JSON message. Did you accidentally double-escape it?",
extra={"message_string": message_string},
)
return empty
hi guys its my first time to ask help here i hope you can help me
i have this code that i write it
def my_function():
try :
with open('file.csv', 'r') as f:
data = list(csv.reader(f, delimiter=','))
i = 1
while i <= 10:
i += 1
fname = data[i][0]
lname = data[i][1]
options = Options()
driver = webdriver.Chrome(options=options)
driver.get("https://www.test.net/")
#Do staff
except Exception as e:
print(e)
driver.quit()
time.sleep(1)
print('******RESTART******')
my_function()
my_function()
well i'm trying to make this script run without stop .. the problem that i'm facing is when it stops for example in line number 8 (i = 8) and restart again ,it starts from first line (i = 1) .
i want the script to restart from line 8 and continue to 9 , 10 ...
can you please guide me to the right solution .. thank you
Your code is making this far more difficult than it needs to be.
First, you almost certainly don't want to wrap this entire block of code in a "catch all" exception handler. You want your exception handling to be sufficiently specific (limited) that you can do something meaningful with the exception. For example:
#!python
# Assumes Python version 3 or later
import sys, csv
filename='myfile.csv'
with open(filename as f:
try:
reader = csv.reader(f)
for record in reader:
if len(record) != 2:
# log error and continue
print('Malformed records in {}: {}'.format(filename, reader.line_num), file=sys.stderr)
continue
# do stuff with this record, knowing it has exactly two fields:
fname = record[0]
lname = record[1]
# etc ...
except csv.Error as e:
print('Error handling {} at line {}: {}'.format(filename, reader.line_num, e), file=sys.stderr)
Note that your errors probably weren't specifically in the csv module. It's pretty tolerant of malformed lines. But I'm showing how to wrap the reader and processing code within exception handling just for that. Your error was probably an IndexError (trying to access an item past the number of items in a list ... outside of its valid indexing range. It's better to just check the length of each record rather than use exception handling for that ... though it's possible either way.
There's a quite reasonable example (very similar code) in the documentation for the standard libraries: https://docs.python.org/3/library/csv.html
Also, stylistically, I'd suggest that a named tuple or a lightweight class (using __slots__) for managing these records. This would allow you to use dot notation to access the .fname and .lname of each rather than using [x] and numeric indexing. (Numeric indexing gets progressively more cumbersome and error prone as your code complexity increases).
You can set i to a key word argument with a default of 1 then on each exception pass the current i when you restart your function so it picks up from there.
This is a simplified example of what I'm recommending following the same general method you are using in your question (but with fake data so I can run it without having your CSV file).
def my_function(i=1):
try:
if i == 4: # to prevent forever recursion
return
else:
print(i) # keep track of loops
i += 1
x = int('te') # causes an error
except ValueError:
my_function(i) # send current i back through
my_function(i=0)
thank you for your quick response .. i tried the solution provided by –Kevin Welch and –Selcuk it works fine for me thx
here is the solution
def my_function():
try :
with open('file.csv', 'r') as f:
data = list(csv.reader(f, delimiter=','))
i = 1
while i <= 10:
i += 1
try :
fname = data[i][0]
lname = data[i][1]
options = Options()
driver = webdriver.Chrome(options=options)
driver.get("https://www.test.net/")
# Do staff
except Exception as e:
print(e)
driver.quit()
time.sleep(1)
print('******RESTAR******')
continue
my_function()
am developing a python application . I have validated customer id from database. Means if the entered custid is present in database, i am raising an exception. In exception class i am printing the message . So far it is printing the message. But i am not sure how to get control back to the statement where i am taking the input.
main app
Custid=input("enter custid)
Validate_custid(Custid)
Print(Custid)
validate_custid module
From connections import cursor
From customExceptions import invalidcustidException
Def validate_custid(custid):
Cursor.execute("select count(custid) from customer where custid=:custid",{"custid":custid})
For row in cursor:
Count=row[0]
If Count==0:
Raise invalidcustidException
So far its printing the message in exception.now i want my program to take custid as input whenever this exception occurs. The process should iterate until user enters valid custid.
You should use a try-except block with else statement:
while True:
custid = input('Input custom Id: ')
try:
# Put your code that may be throw an exception here
validate_custid(custid)
except InvalidcustidException as err:
# Handle the exception here
print(err.strerror)
continue # start a new loop
else:
# The part of code that will execute when no exceptions thrown
print('Your custom id {} is valid.'.format(custid))
break # escape the while loop
Take a look at here: https://docs.python.org/3.4/tutorial/errors.html#handling-exceptions
You'll want a try except block.
try:
# portion of code that may throw exception
except invalidcuspidError:
# stuff you want to do when exception thrown
See https://docs.python.org/2/tutorial/errors.html for more.
What you are trying to do is called exception handling. I think the Python docs explain this better than me, so here you go: https://docs.python.org/2/tutorial/errors.html#handling-exceptions
I am writing a code to parse through a bunch of xml files. It basically looks like this:
for i in range(0, 20855):
urlb = str(i)
url = urla + urlb
trys=0
t=0
while (trys < 3):
try:
cfile = UR.urlopen(url)
trys = 3
except urllib.error.HTTPError as e:
t=t+1
print('error at '+str(time.time()-tstart)+' seconds')
print('typeID = '+str(i))
print(e.code)
print(e.read())
time.sleep (0.1)
trys=0+t
tree = ET.parse(cfile) ##parse xml file
root = tree.getroot()
...do a bunch of stuff with i and the file data
I'm having a problem with some of the urls I'm calling not actually containing an xml file which breaks my code. I have a list of all the actual numbers that I use instead of the range shown but i really don't want to go through all 21000 and remove each number that fails. Is there an easier way to get around this? I get an error from the while loop (which i have to deal with timeouts really) that looks like this:
b'A non-marketable type was given'
error at 4.321678161621094 seconds
typeID = 31
400
So I was thinking there has to be a good way to bail out of that iteration of the for-loop if my while-loop returns three errors but i can't use break. Maybe an if/else-loop under the while-loop that just passes if the t variable is 3?
You might try this:
for i in range(0, 20855):
url = '%s%d' % (urla, i)
for trys in range(3):
try:
cfile = UR.urlopen(url)
break
except urllib.error.HTTPError as e:
print('error at %s seconds' % (time.time()-tstart))
print('typeID = %i'%i)
print(e.code)
print(e.read())
time.sleep(0.1)
else:
print "retry failed 3 times"
continue
try:
tree = ET.parse(cfile) ##parse xml file
except Exception, e:
print "cannot read xml"
print e
continue
root = tree.getroot()
...do a bunch of stuff with i and the file data
Regarding your "algorithmic" problem: You can always set an error state (as simple as e.g. last_iteration_successful = False) in the while body, then break out of the while body, then check the error state in the for body, and conditionally break out of the for body, too.
Regarding architecture: Prepare your code for all relevant errors that might occur, via proper exception handling with try/except blocks. It might also make sense to define custom Exception types, and then raise them manually. Raising an exception immediately interrupts the current control flow, it could save many breaks.
I am new to Python and working on a utility that changes an XML file into an HTML. The XML comes from a call to request = urllib2.Request(url), where I generate the custom url earlier in the code, and then set response = urllib2.urlopen(request) and, finally, xml_response = response.read(). This works okay, as far as I can tell.
My trouble is with parsing the response. For starters, here is a partial example of the XML structure I get back:
I tried adapting the slideshow example in the minidom tutorial here to parse my XML (which is ebay search results, by the way): http://docs.python.org/2/library/xml.dom.minidom.html
My code so far looks like this, with try blocks as an attempt to diagnose issues:
doc = minidom.parseString(xml_response)
#Extract relevant information and prepare it for HTML formatting.
try:
handleDocument(doc)
except:
print "Failed to handle document!"
def getText(nodelist): #taken straight from slideshow example
rc = []
for node in nodelist:
if node.nodeType == node.TEXT_NODE:
print "A TEXT NODE!"
rc.append(node.data)
return ''.join(rc) #this is a string, right?
def handleDocument(doc):
outputFile = open("EbaySearchResults.html", "w")
outputFile.write("<html>\n")
outputFile.write("<body>\n")
try:
items = doc.getElementsByTagName("item")
except:
"Failed to get elements by tag name."
handleItems(items)
outputFile.write("</html>\n")
outputFile.write("</body>\n")
def handleItems(items):
for item in items:
title = item.getElementsByTagName("title")[0] #there should be only one title
print "<h2>%s</h2>" % getText(title.childNodes) #this works fine!
try: #none of these things work!
outputFile.write("<h2>%s</h2>" % getText(title.childNodes))
#outputFile.write("<h2>" + getText(title.childNodes) + "</h2>")
#str = getText(title.childNodes)
#outputFIle.write(string(str))
#outputFile.write(getText(title.childNodes))
except:
print "FAIL"
I do not understand why the correct title text does print to the console but throws an exception and does not work for the output file. Writing plain strings like this works fine: outputFile.write("<html>\n") What is going on with my string construction? As far as I can tell, the getText method I am using from the minidom example returns a string--which is just the sort of thing you can write to a file..?
If I print the actual stack trace...
...
except:
print "Exception when trying to write to file:"
print '-'*60
traceback.print_exc(file=sys.stdout)
print '-'*60
traceback.print_tb(sys.last_traceback)
...
...I will instantly see the problem:
------------------------------------------------------------
Traceback (most recent call last):
File "tohtml.py", line 85, in handleItems
outputFile.write(getText(title.childNodes))
NameError: global name 'outputFile' is not defined
------------------------------------------------------------
Looks like something has gone out of scope!
Fellow beginners, take note.