Develop Reference

Python change Exception printable output, eg overload __builtins__

I am searching for a way to change the printable output of an Exception to a silly message in order to learn more about python internals (and mess with a friend ;), so far without success.
Consider the following code
try:
x # is not defined
except NameError as exc:
print(exc)
The code shall output name 'x' is not defined
I would like the change that output to the name 'x' you suggested is not yet defined, my lord. Improve your coding skills.
So far, I understood that you can't change __builtins__ because they're "baked in" as C code, unless:
You use forbiddenfruit.curse method which adds / changes properties of any object
You manually override the dictionnaries of an object
I've tried both solutions, but without success:
forbiddenfruit solution:
from forbiddenfruit import curse
curse(BaseException, 'repr', lambda self: print("Test message for repr"))
curse(BaseException, 'str', lambda self: print("Test message for str"))
try:
x
except NameError as exc:
print(exc.str()) # Works, shows test message
print(exc.repr()) # Works, shows test message
print(repr(exc)) # Does not work, shows real message
print(str(exc)) # Does not work, shows real message
print(exc) # Does not work, shows real message
Dictionnary overriding solution:
import gc
underlying_dict = gc.get_referents(BaseException.__dict__)[0]
underlying_dict["__repr__"] = lambda self: print("test message for repr")
underlying_dict["__str__"] = lambda self: print("test message for str")
underlying_dict["args"] = 'I am an argument list'
try:
x
except NameError as exc:
print(exc.__str__()) # Works, shows test message
print(exc.__repr__()) # Works, shows test message
print(repr(exc)) # Does not work, shows real message
print(str(exc)) # Does not work, shows real message
print(exc) # Does not work, shows real message
AFAIK, using print(exc) should rely on either __repr__ or __str__, but it seems like the print function uses something else, which I cannot find even when reading all properties of BaseException via print(dir(BaseException)).
Could anyone give me an insight of what print uses in this case please ?
[EDIT]
To add a bit more context:
The problem I'm trying to solve began as a joke to mess with a programmer friend, but now became a challenge for me to understand more of python's internals.
There's no real business problem I'm trying to solve, I just want to get deeper understanding of things in Python.
I'm quite puzzled that print(exc) won't make use of BaseException.__repr__ or __str__ actually.
[/EDIT]

Intro
I'd go with a more critical approach on why you'd even want to do what you want to do.
Python provides you with an ability to handle specific exceptions. That means if you had a business problem, you'd use a particular exception class and provide a custom message for that specific case. Now, remember this paragraph and let's move on, I'll refer to this later.
TL;DR
Now, let's go top-down:
Catching all kinds of errors with except Exception is generally not a good idea if want you catch let's say a variable name error. You'd use except NameError instead. There's really not much you'd add to it that's why it had a default message that perfectly described the issue. So it's assumed you'd use it as it's given. These are called concrete exceptions.
Now, with your specific case notice the alias as exc. By using the alias you can access arguments passed to the exception object, including the default message.
try:
x # is not defined
except NameError as exc:
print(exc.args)
Run that code (I put it in app.py) and you'll see:
$ python app.py
("name 'x' is not defined",)
These args are passed to the exception as a series (list, or in this case immutable list that is a tuple).
This leads to the idea of the possibility of easily passing arguments to exceptions' constructors (__init__). In your case "name 'x' is not defined" was passed as an argument.
You can use this to your advantage to solve your problem without much effort by just providing a custom message, like:
try:
x # is not defined
except NameError as exc:
your_custom_message = "the name 'x' you suggested is not yet defined, my lord. Improve your coding skills"
# Now, you can handle it based on your requirement:
# print(your_custom_message)
# print(NameError(your_custom_message))
# raise NameError(your_custom_message)
# raise NameError(your_custom_message) from exc
The output is now what you wanted to achieve.
$ python app.py
the name 'x' you suggested is not yet defined, my lord. Improve your coding skills
Remember the first paragraph when I said I'd refer to it later? I mentioned providing a custom message for a specific case. If you build your own library when you want to handle name errors to specific variables relevant to your product, you assume your users will use your code that might raise that NameError exception. They will most likely catch it with except Exception as exc or except NameError as exc. And when they do print(exc), they will see your message now.
Summary
I hope that makes sense to you, just provide a custom message and pass it as an argument to NameError or simply just print it. IMO, it's better to learn it right together with why you'd use what you use.

Errors like this are hard-coded into the interpreter (in the case of CPython, anyway, which is most likely what you are using). You will not be able to change the message printed from within Python itself.
The C source code that is executed when the CPython interpreter tries to look up a name can be found here: https://github.com/python/cpython/blob/master/Python/ceval.c#L2602. If you would want to change the error message printed when a name lookup fails, you would need to change this line in the same file:
#define NAME_ERROR_MSG \
"name '%.200s' is not defined"
Compiling the modified source code would yield a Python interpreter that prints your custom error message when encountering a name that is not defined.

I'll just explain the behaviour you described:
exc.__repr__()
This will just call your lambda function and return the expected string. Btw you should return the string, not print it in your lambda functions.
print(repr(exc))
Now, this is going a different route in CPython and you can see this in a GDB session, it's something like this:
Python/bltinmodule.c:builtin_repr will call Objects/object.c:PyObject_Repr - this function gets the PyObject *v as the only parameter that it will use to get and call a function that implements the built-in function repr(), BaseException_repr in this case. This function will format the error message based on a value from args structure field:
(gdb) p ((PyBaseExceptionObject *) self)->args
$188 = ("name 'x' is not defined",)
The args value is set in Python/ceval.c:format_exc_check_arg based on a NAME_ERROR_MSG macro set in the same file.
Update: Sun 8 Nov 20:19:26 UTC 2020
test.py:
import sys
import dis
def main():
try:
x
except NameError as exc:
tb = sys.exc_info()[2]
frame, i = tb.tb_frame, tb.tb_lasti
code = frame.f_code
arg = code.co_code[i + 1]
name = code.co_names[arg]
print(name)
if __name__ == '__main__':
main()
Test:
# python test.py
x
Note:
I would also recommend to watch this video from PyCon 2016.

Django: "referenced before assignment" but only for some variables

I'm writing a small app in Django and I'm keeping the state saved in a few variables I declare out of the methods in views.py. Here is the important part of this file:
from app.playerlist import fullList
auc_unsold = fullList[:]
auc_teams = []
auc_in_progress = []
auc_current_turn = -1
print(auc_in_progress)
def auc_action(request):
data = json.loads(request.GET["data"])
# ...
elif data[0] == "start":
random.shuffle(auc_teams)
print(auc_unsold)
print(auc_in_progress)
auc_in_progress = [None, 0, None]
print(auc_in_progress)
The auc_unsold and auc_teams variables work fine; the auc_in_progress variable is not seen by this method, though, giving the error in the title. If I take out the print statement and let this code assign a value to it, the exception will be thrown somewhere else in the code as soon as I use that variable again.
I have tried making another variable and this new one seems to suffer from this problem as well.
What is happening?
Edit: I found a solution: if I write global auc_in_progress just before the print statements, then everything works fine. If I try writing that as I declare the variable above it doesn't work, though, for some reason.
I am unsatisfied with this, because I don't know why this happens and because I dislike using global like that, but eh. Someone has an explanation?

You should absolutely not be doing this, either your original code or your proposed solution with global.
Anything at module level will be shared across requests, not only for the current user but for all users for that process. So everyone will see the same auction, etc.
The reason for your error is because you assign to that variable within your function, which automatically makes it a local variable: see this question for more details. But the solution recommended there, which is the same as your workaround - ie use global - is not appropriate here; you should store the data somewhere specifically associated with the user, eg the session.

Python - Pass variable handle to evaluate

I am writing some program using python and the z3py module.
What I am trying to do is the following: I extract a constraint of an if or a while statement from a function which is located in some other file. Additionally I extract the used variables in the statement as well as their types.
As I do not want to parse the constraint by hand into a z3py friendly form, I tried to use evaluate to do this for me. Therefore I used the tip of the following page: Z3 with string expressions
Now the problem is: I do not know how the variables in the constraint are called. But it seems as I have to name the handle of each variable like the actual variable. Otherwise evaluate won't find it. My code looks like this:
solver = Solver()
# Look up the constraint:
branch = bd.getBranchNum(0)
constr = branch.code
# Create handle for each variable, depending on its type:
for k in mapper.getVariables():
var = mapper.getVariables()[k]
if k in constr:
if var.type == "intNum":
Int(k)
else:
Real(k)
# Evaluate constraint, insert the result and solve it:
f = eval(constr)
solver.insert(f)
solve(f)
As you can see I saved the variables and constraints in classes. When executing this code I get the following error:
NameError: name 'real_x' is not defined
If I do not use the looping over the variables, but instead the following code, everything works fine:
solver = Solver()
branch = bd.getBranchNum(0)
constr = branch.code
print(constr)
real_x = Real('real_x')
int_y = Int('int_y')
f = eval(constr)
print(f)
solver.insert(f)
solve(f)
The problem is: I do not know, that the variables are called "real_x" or "int_y". Furthermore I do not know how many variables there are used, which means I have to use some dynamic thing like a loop.
Now my question is: Is there a way around this? What can I do to tell python that the handles already exist, but have a different name? Or is my approach completely wrong and I have to do something totally different?

This kind of thing is almost always a bad idea (see Why eval/exec is bad for more details), but "almost always" isn't "always", and it looks like you're using a library that was specifically designed to be used this way, in which case you've found one of the exceptions.
And at first glance, it seems like you've also hit one of the rare exceptions to the Keep data out of your variable names guideline (also see Why you don't want to dynamically create variables). But you haven't.
The only reason you need these variables like real_x to exist is so that eval can see them, right? But the eval function already knows how to look for variables in a dictionary instead of in your global namespace. And it looks like what you're getting back from mapper.getVariables() is a dictionary.
So, skip that whole messy loop, and just do this:
variables = mapper.getVariables()
f = eval(constr, globals=variables)
(In earlier versions of Python, globals is a positional-only argument, so just drop the globals= if you get an error about that.)
As the documentation explains, this gives the eval function access to your actual variables, plus the ones the mapper wants to generate, and it can do all kinds of unsafe things. If you want to prevent unsafe things, do this:
variables = dict(mapper.getVariables())
variables['__builtins__'] = {}
f = eval(constr, globals=variables)

Way in Python to make vars visible in calling method scope?

I find myself doing something like this constantly to pull GET args into vars:
some_var = self.request.get('some_var', None)
other_var = self.request.get('other_var', None)
if None in [some_var, other_var]:
logging.error("some arg was missing in " + self.request.path)
exit()
What I would really want to do is:
pull_args('some_var', 'other_var')
And that would somehow pull these variables to be available in current scope, or log an error and exit if not (or return to calling method if possible). Is this possible in Python?

First, a disclaimer: "pulling" variables into the local scope in any way other than var = something is really really really not recommended. It tends to make your code really confusing for someone who isn't intimately familiar with what you're doing (i.e. anyone who isn't you, or who is you 6 months in the future, etc.)
That being said, for educational purposes only, there is a way. Your pull_args function could be implemented like this:
def pull_args(request, *args):
pulled = {}
try:
for a in args:
pulled[a] = request[a]
except AttributeError:
logging.error("some arg was missing in " + self.request.path)
exit()
else:
caller = inspect.stack()[1][0]
caller.f_locals.update(pulled)
At least, something to that effect worked when I came up with it probably about a year ago. I wouldn't necessarily count on it continuing to work in future Python versions. (Yet another reason not to do it) I personally have never found a good reason to use this code snippet.

No it's not and also pointless. Writing to outer namespaces completely destroys the purpose of namespaces, which is having only the things around that you explicitly set. Use lists!
def pull_args(*names):
return [self.request.get(name, None) for name in names]
print None in pull_args('some_var', 'other_var')
Probably this works too, to check if all _var are set:
print all(name in self.request for name in ('some_var', 'other_var'))

Debug history of variable changing in python

I need magic tool, that helps me to understand where one my problem variable is changed in the code.
I know about perfect tool:
pdb.set_trace()
and I need something similar format, but about only one variable changing history.
For example, my current problem is strange value of context['request'] variable inside Django's tag template definition method. The value is string '<<request>>' and I don't understand where it modified from Django's Request object. I can't debug it, because problem is appearing not so often, but permanently. I only see it in error emails and I can't call it specially. The perfect solution will be to create a log with variable's assignment and any modifications.

I'm not really familiar with django, so your mileage may vary. In general, you can override the __setitem__ method on objects to capture item assignment. However, this doesn't work on dictionaries, only on user-created classes, so first of all it depends on what this context object is.
As I get from a short look at the Django docs, it's indeed not a regular dict, so you can try something like this:
def log_setitem(obj):
class Logged(obj.__class__):
def __setitem__(self, item, val):
print "setting", item, "to", val, "on", self
super(Logged, self).__setitem__(item, val)
obj.__class__ = Logged
d = {}
try:
log_setitem(d) # throws an error
except:
print "doesn't work"
class Dict2(dict):
pass
d2 = Dict2()
log_setitem(d2) # this works
d2["hello"] = "world" # prints the log message before assigning
Even if this works, it of course only works if the assignment actually happens through the "standard" way, i.e. somewhere in the code there's a call like context['request'] = "something".
Might be worth a try, but I can't promise you anything.