How do I detect an error when compiling a directory of python files using compile_dir?
Currently I get something on stderr, but no way to detect it in my app.
py_compile.compile() takes a doraise argument, but nothing here.
Or is there a better way to do this from a python script?
Edit:
I fixed it with os.walk and calling py_compile.compile for each file. But the question remains.
I don't see a better way. The code is designed to support the command-line program, and the API doesn't seem fully meant to be used as a library.
If you really had to use the compileall then you could fake it out with this hack, which notices that "quiet" is tested for boolean-ness while in the caught exception handler. I can override that with nonzero, check the exception state to see if it came from py_compile (quiet is tested in other contexts) and do something with that information:
import sys
import py_compile
import compileall
class ReportProblem:
def __nonzero__(self):
type, value, traceback = sys.exc_info()
if type is not None and issubclass(type, py_compile.PyCompileError):
print "Problem with", repr(value)
raise type, value, traceback
return 1
report_problem = ReportProblem()
compileall.compile_dir(".", quiet=report_problem)
Förresten, det finns GothPy på första måndagen varje månad, om du skulle ta sällskap med andra Python-användare i Gbg.
works fine for me. Could it be that you're not setting doraise to True somehow?
Related
Situation
The xlwings package provides a convenient way to call python functions from an excel VBA module. The xlwings documentation gives the following basic example:
Write the code below into a VBA module.
Sub HelloWorld()
RunPython ("import hello; hello.world()")
End Sub
This calls the following code in hello.py:
# hello.py
import numpy as np
import xlwings as xw
def world():
wb = xw.Book.caller()
wb.sheets[0].range('A1').value = 'Hello World!'
Trying to run the python function world() directly (instead of calling it from excel VBA) gives the following error message:
Exception: Book.caller() must not be called directly. Call through
Excel or set a mock caller first with Book.set_mock_caller().
Question
I would like to modify the world() function such that it raises a custom exception instead when being run directly. In order to achieve this I first need to determine programmatically whether the world() function is being run directly or being called from excel VBA (at least that's what I'd think). How could I do this?
You can catch the exception and then raise your own:
def world():
try:
wb = xw.Book.caller()
except Exception:
raise CustomException(custom_message)
wb.sheets[0].range('A1').value = 'Hello World!'
You are worried that Exception is too generic, and rightly so. But that's xlwings' fault, not yours. If it is raising a generic Exception that's all you are left with to catch. You could check the exception message to make sure that you are not catching the wrong exception, but that would be brittle. Error messages are usually undocumented and not to be regarded as public, stable API.
Alternatively you can fix the problem where the problem is, xlwings' source code, and make it do what looks to me like the right thing to do: raising a more specific exception.
class NotFromExcelError(Exception):
pass
And at the end of caller:
raise NotFromExcelError('Book.caller() must not be called directly. Call through Excel '
'or set a mock caller first with Book.set_mock_caller().')
I hope a pull request like this would be accepted because raising a bare Exception like it currently does looks really wrong.
Im writing a private online Python interpreter for VK, which would closely simulate IDLE console. Only me and some people in whitelist would be able to use this feature, no unsafe code which can harm my server. But I have a little problem. For example, I send the string with code def foo():, and I dont want to get SyntaxError but continue defining function line by line without writing long strings with use of \n. exec() and eval() doesn't suit me in that case. What should I use to get desired effect? Sorry if duplicate, still dont get it from similar questions.
The Python standard library provides the code and codeop modules to help you with this. The code module just straight-up simulates the standard interactive interpreter:
import code
code.interact()
It also provides a few facilities for more detailed control and customization of how it works.
If you want to build things up from more basic components, the codeop module provides a command compiler that remembers __future__ statements and recognizes incomplete commands:
import codeop
compiler = codeop.CommandCompiler()
try:
codeobject = compiler(some_source_string)
# codeobject is an exec-utable code object if some_source_string was a
# complete command, or None if the command is incomplete.
except (SyntaxError, OverflowError, ValueError):
# If some_source_string is invalid, we end up here.
# OverflowError and ValueError can occur in some cases involving invalid literals.
It boils down to reading input, then
exec <code> in globals,locals
in an infinite loop.
See e.g. IPython.frontend.terminal.console.interactiveshell.TerminalInteractiveSh
ell.mainloop().
Continuation detection is done in inputsplitter.push_accepts_more() by trying ast.parse().
Actually, IPython already has an interactive web console called Jupyter Notebook, so your best bet should be to reuse it.
I'm trying to save myself just a few keystrokes for a command I type fairly regularly in Python.
In my python startup script, I define a function called load which is similar to import, but adds some functionality. It takes a single string:
def load(s):
# Do some stuff
return something
In order to call this function I have to type
>>> load('something')
I would rather be able to simply type:
>>> load something
I am running Python with readline support, so I know there exists some programmability there, but I don't know if this sort of thing is possible using it.
I attempted to get around this by using the InteractivConsole and creating an instance of it in my startup file, like so:
import code, re, traceback
class LoadingInteractiveConsole(code.InteractiveConsole):
def raw_input(self, prompt = ""):
s = raw_input(prompt)
match = re.match('^load\s+(.+)', s)
if match:
module = match.group(1)
try:
load(module)
print "Loaded " + module
except ImportError:
traceback.print_exc()
return ''
else:
return s
console = LoadingInteractiveConsole()
console.interact("")
This works with the caveat that I have to hit Ctrl-D twice to exit the python interpreter: once to get out of my custom console, once to get out of the real one.
Is there a way to do this without writing a custom C program and embedding the interpreter into it?
Edit
Out of channel, I had the suggestion of appending this to the end of my startup file:
import sys
sys.exit()
It works well enough, but I'm still interested in alternative solutions.
You could try ipython - which gives a python shell which does allow many things including automatic parentheses which gives you the function call as you requested.
I think you want the cmd module.
See a tutorial here:
http://wiki.python.org/moin/CmdModule
Hate to answer my own question, but there hasn't been an answer that works for all the versions of Python I use. Aside from the solution I posted in my question edit (which is what I'm now using), here's another:
Edit .bashrc to contain the following lines:
alias python3='python3 ~/py/shellreplace.py'
alias python='python ~/py/shellreplace.py'
alias python27='python27 ~/py/shellreplace.py'
Then simply move all of the LoadingInteractiveConsole code into the file ~/py/shellreplace.py Once the script finishes executing, python will cease executing, and the improved interactive session will be seamless.
Is there a convenient way to get a more detailed stack trace on a Python exception? I'm hoping to find a wrapper utility/module or some other way to get a bit more info from the stack trace without having to actually modify the Python script that generates it. I'd like to be able to use this when running unit tests, or doctests, or when running utilities or inline scripts from the shell.
Specifically I think I'd like to have the values of local variables, or maybe just the values of the arguments passed to the innermost function in the stack trace. Some options to set the detail level would be nifty.
Not specifically related to your problem, but you might find this code useful -- automatically starts up the python debugger when a fatal exception occurs. Good for working with interactive code. It's originally from ActiveState
# code snippet, to be included in 'sitecustomize.py'
import sys
def info(type, value, tb):
if hasattr(sys, 'ps1') or not sys.stderr.isatty():
# we are in interactive mode or we don't have a tty-like
# device, so we call the default hook
sys.__excepthook__(type, value, tb)
else:
import traceback, pdb
# we are NOT in interactive mode, print the exception...
traceback.print_exception(type, value, tb)
print
# ...then start the debugger in post-mortem mode.
pdb.pm()
sys.excepthook = info
Did you have a look at traceback module?
http://docs.python.org/library/traceback.html
Also on SO:
Showing the stack trace from a running Python application
As mentionned by pyfunc, you can use the function in the traceback module but you only get a stacktrace.
If you want to inspect the stack you have to use the sys.exc_info() function and walk the traceback member and dump information from its frame (tb_frame). See the python Reference Manual for further information on these types.
Here is an example:
def killit(a):
a[10000000000000] = 1
def test(a):
killit(a)
def iterate_traceback(tb):
while tb is not None:
yield tb
tb = tb.tb_next
try:
test(tuple())
except Exception as e:
import sys
exception_info = sys.exc_info()
traceback = exception_info[2]
for tb in iterate_traceback(traceback):
print "-" * 10
print tb.tb_frame.f_code
print tb.tb_frame.f_locals
print tb.tb_frame.f_globals
Is there a simple way to detect, within Python code, that this code is being executed through the Python debugger?
I have a small Python application that uses Java code (thanks to JPype). When I'm debugging the Python part, I'd like the embedded JVM to be passed debug options too.
Python debuggers (as well as profilers and coverage tools) use the sys.settrace function (in the sys module) to register a callback that gets called when interesting events happen.
If you're using Python 2.6, you can call sys.gettrace() to get the current trace callback function. If it's not None then you can assume you should be passing debug parameters to the JVM.
It's not clear how you could do this pre 2.6.
Other alternative if you're using Pydev that also works in a multithreading is:
try:
import pydevd
DEBUGGING = True
except ImportError:
DEBUGGING = False
A solution working with Python 2.4 (it should work with any version superior to 2.1) and Pydev:
import inspect
def isdebugging():
for frame in inspect.stack():
if frame[1].endswith("pydevd.py"):
return True
return False
The same should work with pdb by simply replacing pydevd.py with pdb.py. As do3cc suggested, it tries to find the debugger within the stack of the caller.
Useful links:
The Python Debugger
The interpreter stack
Another way to do it hinges on how your python interpreter is started. It requires you start Python using -O for production and with no -O for debugging. So it does require an external discipline that might be hard to maintain .. but then again it might fit your processes perfectly.
From the python docs (see "Built-in Constants" here or here):
__debug__
This constant is true if Python was not started with an -O option.
Usage would be something like:
if __debug__:
print 'Python started without optimization'
If you're using Pydev, you can detect it in such way:
import sys
if 'pydevd' in sys.modules:
print "Debugger"
else:
print "commandline"
From taking a quick look at the pdb docs and source code, it doesn't look like there is a built in way to do this. I suggest that you set an environment variable that indicates debugging is in progress and have your application respond to that.
$ USING_PDB=1 pdb yourprog.py
Then in yourprog.py:
import os
if os.environ.get('USING_PDB'):
# debugging actions
pass
You can try to peek into your stacktrace.
https://docs.python.org/library/inspect.html#the-interpreter-stack
when you try this in a debugger session:
import inspect
inspect.getouterframes(inspect.currentframe()
you will get a list of framerecords and can peek for any frames that refer to the pdb file.
I found a cleaner way to do it,
Just add the following line in your manage.py
#!/usr/bin/env python
import os
import sys
if __debug__:
sys.path.append('/path/to/views.py')
if __name__ == "__main__":
....
Then it would automatically add it when you are debugging.
Since the original question doesn't specifically call out Python2 - This is to confirm #babbageclunk's suggested usage of sys also works in python3:
from sys import gettrace as sys_gettrace
DEBUG = sys_gettrace() is not None
print("debugger? %s" % DEBUG)
In my perllib, I use this check:
if 'pdb' in sys.modules:
# We are being debugged
It assumes the user doesn't otherwise import pdb