I have developed a python framework that is being used by others. In order to print any data to the output, the developer should use a Log class (Log.print(...)) and should not use the print() method directly. Is there any ways to force this rule throughout the code? For example, by throwing an error when a developer uses the print method directly like this:
Error: print method cannot be called directly. Please use Log.print().
Suppressing print (as discussed here) is not a good idea as the developer might get confused.
Actullay, below two line code are the same:
sys.stdout.write('hello'+'\n')
print('hello')
so, you can redirect sys.stdout to a class which raise a exception at calling print.
import sys
class BlockPrint():
call_print_exception = Exception('Error: print method cannot be called directly. Please use Log.print().')
def write(self, str):
raise self.call_print_exception
bp = BlockPrint()
sys.stdout=bp
print('aaa')
Output:
Traceback (most recent call last):
File "p.py", line 12, in <module>
print('aaa')
File "p.py", line 7, in write
raise self.call_print_exception
Exception: Error: print method cannot be called directly. Please use Log.print().
I can't seem to generate this exception with python code with a genuine error.
I used the code from this question to check my work. Here it is, modified only slightly:
import py_compile
def check(python_file):
try:
file = open(python_file, 'r')
py_compile.compile(python_file, doraise=True)
except py_compile.PyCompileError:
print("<"+python_file+"> does not contain syntactically correct Python code")
else:
print("Compiled " + python_file + " with no issues.")
check("example.py")
The file example.py contains just:
print ("This is fine.")
prant ("This should be an error.")
'prant' instead of 'print' would be a simple syntax error, and if I run 'python example.py' then I see:
This is fine.
Traceback (most recent call last):
File "example.py", line 2, in <module>
prant ("This should be an error.")
NameError: name 'prant' is not defined
If I call the script at the top 'compiler.py' and then run 'python compiler.py' it will say there are no issues.
I have verified that compiler.py will complain about syntactic correctness if there are unmatched parentheses or quotes, so it does catch some problems. But I would like to be able to detect when a file has errors in the same way that running 'python example.py' or whatever would do. Basically, if it has an error when running it with 'python', I'd like to be able to detect that.
Is there a way to do this? And why is PyCompileError not being thrown when there is a syntax error?
I have a fully working programme that I wish to run. It executes on my friend's laptop, but not mine, (I've saved it to my documents folder) the following is the program:
def DigitCount(n):
#how many decimal digits in integer 'n'
if n<0:
n=-n
digitCount=1
powerOfTen=10
while powerOfTen<=n:
digitCount+=1
powerOfTen*=10
return digitCount
But I keep getting the following error:
>>> DigitCount(100)
Traceback (most recent call last):
File "<pyshell#9>", line 1, in <module>
DigitCount(100)
NameError: name 'DigitCount' is not defined
Wait, are you saying you do the following from the command line?
$ python DigitCount.py
$ python
>>> DigitCount(100)
That won't work. You have to do this:
$ python
>>> import DigitCount
>>> DigitCount.DigitCount(100)
A package that I'm using in my python program is throwing a warning that I'd like to understand the exact cause of. I've set logging.captureWarning(True) and am capturing the warning in my logging, but still have no idea where it is coming from. How do I also log the stack trace so I can see where in my code the warning is coming from? Do I use traceback?
I've ended up going with the below:
import warnings
import traceback
_formatwarning = warnings.formatwarning
def formatwarning_tb(*args, **kwargs):
s = _formatwarning(*args, **kwargs)
tb = traceback.format_stack()
s += ''.join(tb[:-1])
return s
warnings.formatwarning = formatwarning_tb
logging.captureWarnings(True)
It's a little hackish, but you can monkeypatch the warnings.warn method to this:
import traceback
import warnings
def g():
warnings.warn("foo", Warning)
def f():
g()
warnings.warn("bar", Warning)
_old_warn = warnings.warn
def warn(*args, **kwargs):
tb = traceback.extract_stack()
_old_warn(*args, **kwargs)
print("".join(traceback.format_list(tb)[:-1]))
warnings.warn = warn
f()
print("DONE")
This is the output:
/tmp/test.py:14: Warning: foo
_old_warn(*args, **kwargs)
File "/tmp/test.py", line 17, in <module>
f()
File "/tmp/test.py", line 8, in f
g()
File "/tmp/test.py", line 5, in g
warnings.warn("foo", Warning)
/tmp/test.py:14: Warning: bar
_old_warn(*args, **kwargs)
File "/tmp/test.py", line 17, in <module>
f()
File "/tmp/test.py", line 9, in f
warnings.warn("bar", Warning)
DONE
See that calling the original warnings.warn function does not report the line you'd want, bu the stack trace is indeed correct (you could print the warning message yourself).
If you do not know what data/instruction is causing the warning throw, you can use tools like the standard Python Debugger.
The documentation is really good and detailed, but some quickly examples that may help should be:
Without modifying source code: invoking the debbugger as script:
$ python -m pdb myscript.py
Modifying source code: you can make use of calls to pdb.set_trace(), that work like breakpoints; For example, consider I have the following example code:
x = 2
x = x * 10 * 100
y = x + 3 + y
return y
And I would like to know what value does x and y have before the return, or what does the stack contains, I would add the following line between those statements:
pdb.set_trace()
And I will be promted to the (Pdb) prompt, that will allow you to go through the code line by line. Useful commands for the (Pdb) prompt are:
n: executes the next statement.
q: quits the whole program.
c: quits the (Pdb) prompt and stops debugging.
p varname: prints the value of varname
As you do not provide more information, I do not know if that should be enough, but I think that at least, it may be a good start.
BONUS EDIT
Based on this answer, I have found there is a nice and friendly GUI debugging tool, that you can simply install by:
$ pip install pudb
And run the debugger with your script with:
$ python -m pudb.run myscript.py
EDIT: Adding the postmortem debugging
If we do not even know if the code is going to crash or not, we can enter in postmortem debugging if there has been a crash. From the Pbd documentation:
The typical usage to inspect a crashed program is:
>>> import pdb
>>> import mymodule
>>> mymodule.test()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "./mymodule.py", line 4, in test
test2()
File "./mymodule.py", line 3, in test2
print spam
NameError: spam
>>> pdb.pm()
> ./mymodule.py(3)test2()
-> print spam
(Pdb)
As postmortem looks at sys.last_traceback, to enter only if there is a traceback (and so on, a warning or crash):
if sys.last_traceback:
pdb.pm()
You can turn warnings into exceptions, which means you will get a stack trace automatically:
warnings.filterwarnings("error")
See https://docs.python.org/3.4/library/warnings.html#the-warnings-filter
If it was me, I'd go with #Lluís Vilanova's quick & dirty hack, just to find something. But if that's not an option...
If you really want a "logging" solution, you could try something like this (fully working source).
Basic steps are:
Create a custom logging.Formatter subclass that includes the current stack where the logging record is formatted
Use that formatter on the class of the warning
The meat of the code is the custom formatter:
class Formatter(logging.Formatter):
def format(self, record):
record.stack_info = ''.join(traceback.format_stack())
return super().format(record)
Per the docs:
New in version 3.2: The stack_info parameter was added.
For python 3.2 and above, using the optional stack_info keyword argument is the easiest way to get stack trace info along with the log message.
In the example below, "Server.py" is using "lib2.py", which is in turn using "lib.py".
On enabling the stack_info argument the complete trace back is logged along with every logging.log() call. This works the same with logging.info() and other convenience methods as well.
Usage :-
logging.log(DEBUG, "RWL [{}] : acquire_read()".format(self._ownerName), stack_info=True)
Output :-
2018-10-06 10:59:55,726|DEBUG|MainThread|lib.py|acquire_read|RWL [Cache] : acquire_read()
Stack (most recent call last):
File "./Server.py", line 41, in <module>
logging.info("Found {} requests for simulation".format(simdata.count()))
File "<Path>\lib2.py", line 199, in count
with basics.ReadRWLock(self.cacheLock):
File "<Path>\lib.py", line 89, in __enter__
self.rwLock.acquire_read()
File "<Path>\lib.py", line 34, in acquire_read
logging.log(DEBUG, "RWL [{}] : acquire_read()".format(self._ownerName), stack_info=True)
I'm sure that the answer for this is out there, but I've read the site info, I've watched the video they made and I've tried to find a really basic tutorial but I can't. I've been messing about with this for most of the day and It's not really making sense to me.
Here's my error:
vco#geoHP:~$ python3 a_blah.py "don't scare the cats" magic
Traceback (most recent call last):
File "a_blah.py", line 20, in <module>
arguments = docopt.docopt(__doc__)
File "/usr/lib/python3/dist-packages/docopt.py", line 579, in docopt
raise DocoptExit()
docopt.DocoptExit: Usage:
a_blah.py <start>... <end>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "a_blah.py", line 33, in <module>
except DocoptExit:
NameError: name 'DocoptExit' is not defined
line 20 - I don't see why that line is creating an error, it worked before and I've seen that exact line in others programmes?
I don't know why the line 570 of docopt is creating an error - I've seen others use DocoptExit(), isn't this something that's just part of Docopt? Do I have to write my own exit function for this? (I've not seen anyone else do that)
here's the code
import docopt
if __name__ == '__main__':
try:
arguments = docopt.docopt(__doc__)
print(arguments['<start>'])
print("that was that")
print(arguments['<end>'])
except docopt.DocoptExit:
print("this hasn't worked")
What I'm trying to make this for is a script that I've written that moves files from one place to another based on their extension.
So the arguments at the command line will be file type, start directory, destination directory, and an option to delete them from the start directory after they've been moved.
I'm trying (and failing) to get docopt working on it's own prior to including it in the other script though.
The exception you want is in docopt's namespace. You never import it into your global namespace, so you can't refer to it simply with it's name. You need to import it separately or refer to it through the module. You also shouldn't use parenthesis after the exception.
import docopt
try:
# stuff
except docopt.DocoptExit:
# other stuff
or
import docopt
from docopt import DocoptExit
try:
# stuff
except DocoptExit:
# other stuff