I've got a loop processing sockets, and I've set a pdb.set_trace() breakpoint to stop and inspect the results of the call to select.select() every time through the loop. However, there are also bugs elsewhere in my code, and it seems that the standard traceback is being overwritten by pdb.set_trace. As a result, when the program quits, the traceback points to the line immediately following the set_trace(), not the line that contained the error.
Is there a way to access the actual traceback, or does pdb.set_trace() clobber it?
Here's the relevant code snippet:
while True:
read_socks, write_socks, _ = select.select(all_sockets, all_sockets, '')
pdb.set_trace()
if listen_socket.fileno() in read_socks:
new_socket, address = listen_socket.accept()
id_num = new_socket.fileno()
all_sockets[id_num] = new_socket
for id_num in write_socks:
# do something that triggers an Assertion error
and then I get the traceback as follows:
Traceback (most recent call last):
File "socktactoe_server.py", line 62, in <module>
if listen_sock.fileno() in read_socks:
AssertionError
Here is a short reproducible test case; run it, hit c every time there is a breakpoint, after the second continue the assert raises an exception and it's reported for the wrong line:
import pdb
x = 0
while True:
pdb.set_trace()
y = "line of code not triggering an error"
x += 1
assert x != 3
Output:
Traceback (most recent call last):
File "minimal_pdb_traceback.py", line 7, in <module>
y = "line of code not triggering an error"
AssertionError
It looks like you found a bug in the Python pdb module! I can reproduce your simple case in Python versions 2.7, 3.2 and 3.3, while the problem is not present in Python 2.4, 2.5, 2.6 or 3.1.
At first glance, I don't see a preexisting bug in the python bug tracker. Please report your problem there, with your minimal test case, as well as what python versions it can be reproduced on.
Related
When I run the code below in jupyter, it is excellent, I get the results I want.
real = fr.load_image_file("image.jpg")
unknown = fr.load_image_file("image2.jpg")
# encodings
real_encoding = fr.face_encodings(real)[0]
unk = fr.face_encodings(unknown)[0]
result = fr.compare_faces([real_encoding], unk)
print(result)
but when I copy and paste the same code to an editor(atom the one I am using), and then run it using command prompt, I get an error as shown below
Traceback (most recent call last):
File "face_recognizer.py", line 13, in <module>
real_encoding = fr.face_encodings(real)[0]
IndexError: list index out of range
I activated the environment in which I was running the cmd.
Can someone advise me where I maybe going wrong?
There was no face captured in the photo.
That is why it throws an exception.
I can't seem to generate this exception with python code with a genuine error.
I used the code from this question to check my work. Here it is, modified only slightly:
import py_compile
def check(python_file):
try:
file = open(python_file, 'r')
py_compile.compile(python_file, doraise=True)
except py_compile.PyCompileError:
print("<"+python_file+"> does not contain syntactically correct Python code")
else:
print("Compiled " + python_file + " with no issues.")
check("example.py")
The file example.py contains just:
print ("This is fine.")
prant ("This should be an error.")
'prant' instead of 'print' would be a simple syntax error, and if I run 'python example.py' then I see:
This is fine.
Traceback (most recent call last):
File "example.py", line 2, in <module>
prant ("This should be an error.")
NameError: name 'prant' is not defined
If I call the script at the top 'compiler.py' and then run 'python compiler.py' it will say there are no issues.
I have verified that compiler.py will complain about syntactic correctness if there are unmatched parentheses or quotes, so it does catch some problems. But I would like to be able to detect when a file has errors in the same way that running 'python example.py' or whatever would do. Basically, if it has an error when running it with 'python', I'd like to be able to detect that.
Is there a way to do this? And why is PyCompileError not being thrown when there is a syntax error?
A package that I'm using in my python program is throwing a warning that I'd like to understand the exact cause of. I've set logging.captureWarning(True) and am capturing the warning in my logging, but still have no idea where it is coming from. How do I also log the stack trace so I can see where in my code the warning is coming from? Do I use traceback?
I've ended up going with the below:
import warnings
import traceback
_formatwarning = warnings.formatwarning
def formatwarning_tb(*args, **kwargs):
s = _formatwarning(*args, **kwargs)
tb = traceback.format_stack()
s += ''.join(tb[:-1])
return s
warnings.formatwarning = formatwarning_tb
logging.captureWarnings(True)
It's a little hackish, but you can monkeypatch the warnings.warn method to this:
import traceback
import warnings
def g():
warnings.warn("foo", Warning)
def f():
g()
warnings.warn("bar", Warning)
_old_warn = warnings.warn
def warn(*args, **kwargs):
tb = traceback.extract_stack()
_old_warn(*args, **kwargs)
print("".join(traceback.format_list(tb)[:-1]))
warnings.warn = warn
f()
print("DONE")
This is the output:
/tmp/test.py:14: Warning: foo
_old_warn(*args, **kwargs)
File "/tmp/test.py", line 17, in <module>
f()
File "/tmp/test.py", line 8, in f
g()
File "/tmp/test.py", line 5, in g
warnings.warn("foo", Warning)
/tmp/test.py:14: Warning: bar
_old_warn(*args, **kwargs)
File "/tmp/test.py", line 17, in <module>
f()
File "/tmp/test.py", line 9, in f
warnings.warn("bar", Warning)
DONE
See that calling the original warnings.warn function does not report the line you'd want, bu the stack trace is indeed correct (you could print the warning message yourself).
If you do not know what data/instruction is causing the warning throw, you can use tools like the standard Python Debugger.
The documentation is really good and detailed, but some quickly examples that may help should be:
Without modifying source code: invoking the debbugger as script:
$ python -m pdb myscript.py
Modifying source code: you can make use of calls to pdb.set_trace(), that work like breakpoints; For example, consider I have the following example code:
x = 2
x = x * 10 * 100
y = x + 3 + y
return y
And I would like to know what value does x and y have before the return, or what does the stack contains, I would add the following line between those statements:
pdb.set_trace()
And I will be promted to the (Pdb) prompt, that will allow you to go through the code line by line. Useful commands for the (Pdb) prompt are:
n: executes the next statement.
q: quits the whole program.
c: quits the (Pdb) prompt and stops debugging.
p varname: prints the value of varname
As you do not provide more information, I do not know if that should be enough, but I think that at least, it may be a good start.
BONUS EDIT
Based on this answer, I have found there is a nice and friendly GUI debugging tool, that you can simply install by:
$ pip install pudb
And run the debugger with your script with:
$ python -m pudb.run myscript.py
EDIT: Adding the postmortem debugging
If we do not even know if the code is going to crash or not, we can enter in postmortem debugging if there has been a crash. From the Pbd documentation:
The typical usage to inspect a crashed program is:
>>> import pdb
>>> import mymodule
>>> mymodule.test()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "./mymodule.py", line 4, in test
test2()
File "./mymodule.py", line 3, in test2
print spam
NameError: spam
>>> pdb.pm()
> ./mymodule.py(3)test2()
-> print spam
(Pdb)
As postmortem looks at sys.last_traceback, to enter only if there is a traceback (and so on, a warning or crash):
if sys.last_traceback:
pdb.pm()
You can turn warnings into exceptions, which means you will get a stack trace automatically:
warnings.filterwarnings("error")
See https://docs.python.org/3.4/library/warnings.html#the-warnings-filter
If it was me, I'd go with #Lluís Vilanova's quick & dirty hack, just to find something. But if that's not an option...
If you really want a "logging" solution, you could try something like this (fully working source).
Basic steps are:
Create a custom logging.Formatter subclass that includes the current stack where the logging record is formatted
Use that formatter on the class of the warning
The meat of the code is the custom formatter:
class Formatter(logging.Formatter):
def format(self, record):
record.stack_info = ''.join(traceback.format_stack())
return super().format(record)
Per the docs:
New in version 3.2: The stack_info parameter was added.
For python 3.2 and above, using the optional stack_info keyword argument is the easiest way to get stack trace info along with the log message.
In the example below, "Server.py" is using "lib2.py", which is in turn using "lib.py".
On enabling the stack_info argument the complete trace back is logged along with every logging.log() call. This works the same with logging.info() and other convenience methods as well.
Usage :-
logging.log(DEBUG, "RWL [{}] : acquire_read()".format(self._ownerName), stack_info=True)
Output :-
2018-10-06 10:59:55,726|DEBUG|MainThread|lib.py|acquire_read|RWL [Cache] : acquire_read()
Stack (most recent call last):
File "./Server.py", line 41, in <module>
logging.info("Found {} requests for simulation".format(simdata.count()))
File "<Path>\lib2.py", line 199, in count
with basics.ReadRWLock(self.cacheLock):
File "<Path>\lib.py", line 89, in __enter__
self.rwLock.acquire_read()
File "<Path>\lib.py", line 34, in acquire_read
logging.log(DEBUG, "RWL [{}] : acquire_read()".format(self._ownerName), stack_info=True)
I'm running PyCharm Community Edition 4.0.4
Does anyone know why the error messages don't display after the console output?
Thanks
C:\Python27\python.exe "F:/Google Drive/code/python_scripts/leetcode/lc_127_word_ladder.py"
Traceback (most recent call last):
START
File "F:/Google Drive/code/python_scripts/leetcode/lc_127_word_ladder.py", line 68, in <module>
print sol.ladderLength('talk', 'tail', set)
Graph:
File "F:/Google Drive/code/python_scripts/leetcode/lc_127_word_ladder.py", line 54, in ladderLength
hall ['fall']
for item in graph[node[0]]:
fall ['hall']
KeyError: 'talk'
End Graph:
Visited = {'talk': 0}
Node = ['talk', 0]
Queue Before = deque([])
Process finished with exit code 1
If you'll notice, print statements such as START, Graph:, hall ['fall'], up to Queue Before = deque([]) all happen within the functioning part of my code. The Error messages should appear after all this.
This is caused by PyCharm mixing print statements from stdout and stderr. There's a fix if you add the following line to your idea.properties file:
output.reader.blocking.mode=true
Get to idea.properties via Help | Edit Custom Properties.
might just be an issue with the stdout.
a workaround would be to use sys.flush.stdout() after your print statements.
import sys
do_something()
print("Your print statement")
sys.stdout.flush()
I'm new to pycharm, so not sure if there's a clean way to do this. But as a workaround, you could replace your print function with a custom one that sleeps quickly after printing, then your traceback should appear after your outputs.
import time
print = (lambda p: lambda *args,**kwargs: [p(*args,**kwargs), time.sleep(.01)])(print)
'''
# the above is just a one liner equivalent to this decorator
def add_sleep(p):
def new_p(*args, **kwargs):
p(*args,**kwargs)
time.sleep(.01)
return new_p
print = add_sleep(print)
'''
I have been stuck with this error for a couple of hours now. Not sure what is wrong. Below is the piece of code
NameError: global name 'GetText' is not defined
class BaseScreen(object):
def GetTextFromScreen(self, x, y, a, b, noofrows = 0):
count = 0
message = ""
while (count < noofrows):
line = Region(self.screen.x + x, self.screen.y + y + (count * 20), a, b)
message = message + "\n" + line.text()
count += 1
return message
class HomeScreen(BaseScreen):
def GetSearchResults(self):
if self.screen.exists("Noitemsfound.png"):
return 'No Items Found'
else:
return self.GetTextFromScreen(36, 274, 680, 20, 16)
class HomeTests(unittest.TestCase):
def test_001S(self):
Home = HomeScreen()
Home.ResetSearchCriteria()
Home.Search("0009", "Key")
self.assertTrue("0009" in Home.GetSearchResults(), "Key was not returned")
Basescreen class has all the reusable methods applicable across different screens.
Homescreen inherits Basescreen.
In HomeTests test case class, the last step is to Home.GetSearchResults() which in turn calls a base class method and the error.
Note:
I have another screenclass and testcaseclass doing the same which works without issues.
I have checked all the importing statements and is ok
'GetText' in the error message is the name of method initially after which i changed it to GetTextFromScreen
Error message is still pointing to a line 88 in code which is not there any more. Module import/reloading issue?
Try clearing out your *.pyc files (or __pycache__ if using 3+).
You asked:
Error message is still pointing to a line 88 in code which is not there any more. Module import/reloading issue?
Yes. The traceback (error messages) will show the current (newest saved) file, even if you haven't run it yet. You must reload/reimport to get the new file.
The discrepancy comes from the fact that traceback printouts read from the script file (scriptname.py) saved on your drive. However, the program is run either from the module saved in memory, or sometimes from the .pyc file. If you fix an error by changing your script, and save it to your drive, then the same error will still occur if you don't reload it.
If you're running interactively for testing, you can use the reload function:
>>> import mymodule
>>> mymodule.somefunction()
Traceback (most recent call last):
File "mymodule.py", line 3, in somefunction
Here is a broken line
OhNoError: Problem with your file
Now, you fix the error and save mymodule.py, return to your interactive session, but you still get the error, but the traceback shows the fixed line
>>> mymodule.somefunction()
Traceback (most recent call last):
File "mymodule.py", line 3, in somefunction
Here is the fixed line
OhNoError: Problem with your file
So you have to reload the module:
>>> reload(mymodule)
<module 'mymodule' from '/path/to/mymodule.py'>
>>> mymodule.somefunction()
Success!