I am trying to solve a interview question: how to convert a BST to a double linked list.
Below is my code. I am really confused about how python recursion works. Why after the convert2DoubleLst_recursive , the prev and head remain to be None, I have assigned a new value to these two variables, why after this method call, I cannot save the change. I remembered that python pass arguments by reference, but why here I cannot save the changes. Many thanks in advance.
def convert2_recursive(node):
prev,head=None,None
convert2DoubleLst_recursive(node,prev,head)
end=head.lChild
printList(head,end)
def convert2DoubleLst_recursive(node,prev,head):
if node:
convert2DoubleLst_recursive(node.lChild,prev,head)
node.lChild=prev
if prev:
prev.rChild=node
else:
head=node
nextNode=node.rChild
head.lChild=node
node.rChild=head
prev=node
convert2DoubleLst_recursive(nextNode,prev,head)
Naively put, because there is no pass-by-reference in Python. Recursion is not your problem.
It may become a little clearer, when you think "There are no variables in Python, only names and values".
When the statement convert2DoubleLst_recursive(node.lChild, prev, head) is executed, actually the function convert2DoubleLst_recursive is called with the values that node.lChild, prev and head point to at that time.
Then, within convert2DoubleLst_recursive, those values are given the (local) names node, etc. (which are different names from those before!).
Later you assign new values to those names, recurse (which is not actually relevant in your code) and essentially Python forgets the names and their values when exiting the function.
Basically, between line 8 and 10 prev does not change it's value (as you experienced), because the name used in the caller is never seen inside of the callee. The value assigned to that name is not changed in line 3 and it is not relevant what happens inside the callee.
If you want to pass values back to your caller, you have to return them.
Alternatively you could replace None by a guardian object that behaves like your node class and represents None.
Usually, though, there are better data structures you would use in Python than linked lists.
(Python lists for example are already linked lists internally and can be used instead.)
A much deeper analysis can be found here: https://stackoverflow.com/a/986145/110707 .
Related
I am new to Python from R. I have recently spent a lot of time reading up on how everything in Python is an object, objects can call methods on themselves, methods are functions within a class, yada yada yada.
Here's what I don't understand. Take the following simple code:
mylist = [3, 1, 7]
If I want to know how many times the number 7 occurs, I can do:
mylist.count(7)
That, of course, returns 1. And if I want to save the count number to another variable:
seven_counts = mylist.count(7)
So far, so good. Other than the syntax, the behavior is similar to R. However, let's say I am thinking about adding a number to my list:
mylist.append(9)
Wait a minute, that method actually changed the variable itself! (i.e., "mylist" has been altered and now includes the number 9 as the fourth digit in the list.) Assigning the code to a new variable (like I did with seven_counts) produces garbage:
newlist = mylist.append(9)
I find the inconsistency in this behavior a bit odd, and frankly undesirable. (Let's say I wanted to see what the result of the append looked like first and then have the option to decide whether or not I want to assign it to a new variable.)
My question is simple:
Is there a way to know in advance if calling a particular method will actually alter your variable (object)?
Aside from reading the documentation (which for some methods will include type annotations specifying the return value) or playing with the method in the interactive interpreter (including using help() to check the docstring for a type annotation), no, you can't know up front just by looking at the method.
That said, the behavior you're seeing is intentional. Python methods either return a new modified copy of the object or modify the object in place; at least among built-ins, they never do both (some methods mutate the object and return a non-None value, but it's never the object just mutated; the pop method of dict and list is an example of this case).
This either/or behavior is intentional; if they didn't obey this rule, you'd have had an even more confusing and hard to identify problem, namely, determining whether append mutated the value it was called on, or returned a new object. You definitely got back a list, but is it a new list or the same list? If it mutated the value it was called on, then
newlist = mylist.append(9)
is a little strange; newlist and mylist would be aliases to the same list (so why have both names?). You might not even notice for a while; you'd continue using newlist, thinking it was independent of mylist, only to look at mylist and discover it was all messed up. By having all such "modify in place" methods return None (or at least, not the original object), the error is discovered more quickly/easily; if you try and use newlist, mistakenly believing it to be a list, you'll immediately get TypeErrors or AttributeErrors.
Basically, the only way to know in advance is to read the documentation. For methods whose name indicates a modifying operation, you can check the return value and often get an idea as to whether they're mutating. It helps to know what types are mutable in the first place; list, dict, set and bytearray are all mutable, and the methods they have that their immutable counterparts (aside from dict, which has no immutable counterpart) lack tend to mutate the object in place.
The default tends to be to mutate the object in place simply because that's more efficient; if you have a 100,000 element list, a default behavior for append that made a new 100,001 element list and returned it would be extremely inefficient (and there would be no obvious way to avoid it). For immutable types (e.g. str, tuple, frozenset) this is unavoidable, and you can use those types if you want a guarantee that the object is never mutate in place, but it comes at a cost of unnecessary creation and destruction of objects that will slow down your code in most cases.
Just checkout the doc:
>>> list.count.__doc__
'L.count(value) -> integer -- return number of occurrences of value'
>>> list.append.__doc__
'L.append(object) -> None -- append object to end'
There isn't really an easy way to tell, but:
immutable object --> no way of changing through method calls
So, for example, tuple has no methods which affect the tuple as it is unchangeable so methods can only return new instances.
And if you "wanted to see what the result of the append looked like first and then have the option to decide whether or not I want to assign it to a new variable" then you can concatenate the list with a new list with one element.
i.e.
>>> l = [1,2,3]
>>> k = l + [4]
>>> l
[1, 2, 3]
>>> k
[1, 2, 3, 4]
Not from merely your invocation (your method call). You can guarantee that the method won't change the object if you pass in only immutable objects, but some methods are defined to change the object -- and will either not be defined for the one you use, or will fault in execution.
I Real Life, you look at the method's documentation: that will tell you exactly what happens.
[I was about to include what Joe Iddon's answer covers ...]
ie we have the global declaration, but no local.
"Normally" arguments are local, I think, or they certainly behave that way.
However if an argument is, say, a list and a method is applied which modifies the list, some surprising (to me) results can ensue.
I have 2 questions: what is the proper way to ensure that a variable is truly local?
I wound up using the following, which works, but it can hardly be the proper way of doing it:
def AexclB(a,b):
z = a+[] # yuk
for k in range(0, len(b)):
try: z.remove(b[k])
except: continue
return z
Absent the +[], "a" in the calling scope gets modified, which is not desired.
(The issue here is using a list method,
The supplementary question is, why is there no "local" declaration?
Finally, in trying to pin this down, I made various mickey mouse functions which all behaved as expected except the last one:
def fun4(a):
z = a
z = z.append(["!!"])
return z
a = ["hello"]
print "a=",a
print "fun4(a)=",fun4(a)
print "a=",a
which produced the following on the console:
a= ['hello']
fun4(a)= None
a= ['hello', ['!!']]
...
>>>
The 'None' result was not expected (by me).
Python 2.7 btw in case that matters.
PS: I've tried searching here and elsewhere but not succeeded in finding anything corresponding exactly - there's lots about making variables global, sadly.
It's not that z isn't a local variable in your function. Rather when you have the line z = a, you are making z refer to the same list in memory that a already points to. If you want z to be a copy of a, then you should write z = a[:] or z = list(a).
See this link for some illustrations and a bit more explanation http://henry.precheur.org/python/copy_list
Python will not copy objects unless you explicitly ask it to. Integers and strings are not modifiable, so every operation on them returns a new instance of the type. Lists, dictionaries, and basically every other object in Python are mutable, so operations like list.append happen in-place (and therefore return None).
If you want the variable to be a copy, you must explicitly copy it. In the case of lists, you slice them:
z = a[:]
There is a great answer than will cover most of your question in here which explains mutable and immutable types and how they are kept in memory and how they are referenced. First section of the answer is for you. (Before How do we get around this? header)
In the following line
z = z.append(["!!"])
Lists are mutable objects, so when you call append, it will update referenced object, it will not create a new one and return it. If a method or function do not retun anything, it means it returns None.
Above link also gives an immutable examle so you can see the real difference.
You can not make a mutable object act like it is immutable. But you can create a new one instead of passing the reference when you create a new object from an existing mutable one.
a = [1,2,3]
b = a[:]
For more options you can check here
What you're missing is that all variable assignment in python is by reference (or by pointer, if you like). Passing arguments to a function literally assigns values from the caller to the arguments of the function, by reference. If you dig into the reference, and change something inside it, the caller will see that change.
If you want to ensure that callers will not have their values changed, you can either try to use immutable values more often (tuple, frozenset, str, int, bool, NoneType), or be certain to take copies of your data before mutating it in place.
In summary, scoping isn't involved in your problem here. Mutability is.
Is that clear now?
Still not sure whats the 'correct' way to force the copy, there are
various suggestions here.
It differs by data type, but generally <type>(obj) will do the trick. For example list([1, 2]) and dict({1:2}) both return (shallow!) copies of their argument.
If, however, you have a tree of mutable objects and also you don't know a-priori which level of the tree you might modify, you need the copy module. That said, I've only needed this a handful of times (in 8 years of full-time python), and most of those ended up causing bugs. If you need this, it's a code smell, in my opinion.
The complexity of maintaining copies of mutable objects is the reason why there is a growing trend of using immutable objects by default. In the clojure language, all data types are immutable by default and mutability is treated as a special cases to be minimized.
If you need to work on a list or other object in a truly local context you need to explicitly make a copy or a deep copy of it.
from copy import copy
def fn(x):
y = copy(x)
I am very new to Python and as an exercise I tried solving a basic finance exercise using code. My objective is to get a dictionary of spot rates and then a dictionary of discount rates calculated from those. I had thought to something like this:
discountrates={}
def discountrates(n):
spotrates={}
for x in range(1,n+1):
spotrates['s'+str(x)]=float(input('What is s'+str(x)+'? (not in percentage)'))
for y in range(1,n+1):
discountrates['d(0,'+str(y)+')']= 1/((1+float(spotrates['s'+str(y)]))**y)
for key, value in discountrates.items():
print (key, value)
Now the problem is that dictionary items cannot be accessed in a function. When I looked in your forum, I found solutions for unpacking the dictionary but that does not work in my case because I need to access a specific element of the dictionary, whose name cannot be fully specified (as I have seen in the Python manual) because it's part of a loop, in order for the formula to work without having to manually insert anything else. I used a dictionary in the first place to create names that were automatically generated but now I can't seem to get the information out of it.
What is the best solution?
Thanks in advance for the help. It's been driving me crazy.
It's because you called your global variable discountratesdict not discountrates (which is the name of your function).
I suggest you don"t name your dictionary like your function since the later will overwrite the former. In line 1 you say discountrates is an empty dict, in line 2 you say discountrates is a function object. You need to give them different names in python if they are on the same scope.
Furthermore why do you need discountrates to be global? would you like to keep old rates if n is smaller than a previous n? For performance I suggest you combine the two loops. Besides that there is no reason why the second loop can't read for x ... as well since zou don't use x anymore anyway. As a further hint, if you come to the conclusion, that a global is the only way it might help to add global discountratesdict, so it is easier to spot that a global is intended here, even though this is not necessary in your particular case since the []-operator needs an object and thus it already refers to your global.
Putting all this together yields:
discountratedict={}
def discountrates(n):
global discountratedict
spotrates={}
for x in range(1,n+1):
spotrates['s'+str(x)]=float(input('What is s'+str(x)+'? (not in percentage)'))
discountratedict['d(0,'+str(x)+')']= 1/((1+float(spotrates['s'+str(x)]))**x)
for key, value in discountratedict.items():
print (key, value)
I saw in a book about language description that says
On the other hand, a name can be bound to no object (a dangling pointer),
one object (the usual case), or several objects (a parameter name in a
recursive function).
How can we bind a name to several objects? Isnt that what we call an array for example where all elements have the same name but with index? For a recursive function like the example here:
x = 0
def f(y):
global x
x += 1
if x < 4 :
y +=100
f(y)
else: return
f(100)
Is the name y binded with multiple values that are created recursively since the nametable has already the y name binded to an initial value which is being reproduced with recursion?
EDITED Just press here Visualizer and see what it generates. :)
No.
A name is bound to one single object . When we are talking about Python - it is either bound to a single object in a given context, or do not exist at all.
What happens, is that the inner workings may have the name defined in several "layers" - but your code will only see one of those.
If a name is a variable in a recursive function, you will only see whatver is bound to it in the current running context - each time there is a function call in Python, the execution frame, which is an object which holds several attributes of the running code, including a reference to the local variables, is frozen. On the called function, a new execuciton frame is created, and there, the variable names are bound again to whatever new values they have in the called context. Your code just "see" this instance.
Then, there is the issue of global variables and builtin objects in Python: if a name is not a local variable in the function execution context, it is searched in the globals variables for the module (again, just one of those will be visible).ANd if the name is not defiend in the globals, them, Python looks for it in globals().__builtins__ that is your last call.
If I understand you correctly, you're asking about what rules Python has for creating variables in different scopes. Python uses lexical scoping on the function level.
It's hard to tell exactly what you're getting at with the code you've written, but, while there may be a different value associated with y in different scopes (with a value of y defined at each level of recursion), your code will only ever be able to see one at a time (the value defined at the scope in which you're operating).
To really understand scoping rules in Python, I would have a look at PEP 227. Also, have a look at this Stack Overflow question.
Finally, to be able to speak intelligently about what a "name" is in Python, I suggest you read about how Python is a "Call-By-Object" language.
At this point, we are capable of understanding that, instead of a "nametable", python uses a dictionary to hold what is accessible in a given scope. See this answer for a little more detail. The implication of this is that you can never have two of the same name in a single scope (for the same reason you can't have two of the same key in a python dictionary). So, while y may exist in a dictionary for a different scope, you have no way of accessing it, since you can only access the variables in the current scope's dictionary.
The key is:
several objects (a parameter name in a recursive function).
The passage is almost certainly not referring to arrays, but simply to the fact that in a recursive function (or any function, but a recursive function is likely to have multiple activations at one time), a parameter may be bound to a different value in each recursive call.
This does not mean that you can access each such object in every stack frame; indeed the point of the technique is to ensure that only one such value is accessible in each stack frame.
Firstly, you should mention in the question that the sentence from the book is not related explicitly to Python (as jsbueno wrote, one name is bound to exactly one object in Python).
Anyway, name bound to no object is a bit inaccurate. Generally, names are related to variables, and name related to a dangling pointer is the name of that pointer variable.
When speaking about the variable scope (i.e. the part of code where the variable is used), one variable name can be used only for a single value at a time. However, there may be other parts of code, independent on the one where we think about that variable. In the other part of code, the same name can be used; however, the two variables with the same name are totally isolated. This is the case of local variables also in the case of function bodies. If the language allows recursion, it must be capable to create another isolated space of local variable even for another call of the same function.
In Python, each function can also access outer variables, but it is more usual to use the inner, local variables. Whenever you assign a name some value, it is created in the local space.
I have a Python extension module which creates a tuple as an attribute of another object, and sets items in the tuple. Whenever I execute this module in Python, I keep getting the error SystemError: bad argument to internal function
After reading over the docs for PyTuple, and debugging my program for a few hours, I still couldn't figure out what the hell was going on. Running my program through a debugger indicated the problem was occurring within a library call inside the Python interpreter. So, finally, I looked at the Python source code, and at long last I realized the problem. The PyTuple_SetItem function has an interesting restriction which I didn't know about, and can't find explicitly documented.
Here is the important function in the Python source (edited for clarity):
int PyTuple_SetItem(register PyObject *op, register Py_ssize_t i, PyObject *newitem)
{
.....
if (!PyTuple_Check(op) || op->ob_refcnt != 1) {
Py_XDECREF(newitem);
PyErr_BadInternalCall();
return -1;
}
.....
}
The important line here is the condition op->ob_refcnt != 1. So here's the problem: you can't even call PyTuple_SetItem unless the Tuple has a ref-count of 1. It looks like the idea here is that you're never supposed to use PyTuple_SetItem except right after you create a tuple using PyTuple_New(). I guess this makes sense, since Tuples are supposed to be immutable, after all, so this restriction helps keep your C code more in line with the abstractions of the Python type system.
However, I can't find this restriction documented anywhere. Relevant docs seem to be here and here, neither of which specify this restriction. The docs basically say that when you call PyTuple_New(X), all items in the tuple are initialized to NULL. Since NULL is not a valid Python value, it's up to the extension-module programmer to make sure that all slots in the Tuple are filled in with proper Python values before returning the Tuple to the interpreter. But it doesn't say anywhere that this must be done while the Tuple object has a ref count of 1.
So now, the problem is that I've basically coded myself into a corner because I wasn't aware of this (undocumented?) restriction on PyTuple_SetItem. My code is structured in such a way that it's very inconvenient to insert items into the tuple until after the tuple itself has become an attribute of another object. So, when it comes time to fill in the items in the tuple, the tuple already has a higher ref count.
I'll probably have to restructure my code, but I was seriously considering just temporarily setting the ref count on the Tuple to 1, inserting the items, and then restoring the original ref count. Of course, that's a horrible hack, I know, and not any kind of permanent solution. Regardless, I'd like to know if the requirement regarding the ref count on the Tuple is documented anywhere. Is it just an implementation detail of CPython, or is it something that API users can rely on as expected behavior?
I'm quite sure that you can get around the restrictions by using PyTuple_SET_ITEM instead of PyTuple_SetItem. PyTuple_SET_ITEM is a macro defined in tupleobject.h as follows:
#define PyTuple_SET_ITEM(op, i, v) (((PyTupleObject*)(op))->ob_item[i] = v
So, if you are absolutely, definitely and utterly sure that:
op is a tuple object
you haven't initialized slot i in the tuple so far
you own a reference to v and you want to let the tuple steal it and
there is no chance of another Python object using the tuple for anything before you call PyTuple_SET_ITEM
then I guess you are safe to use PyTuple_SET_ITEM.
The Python C API is very underdocumented and I would not be surprised if this restriction wasn't mentioned anywhere.
Of course, you should never be modifying tuples once something has gotten a hold of them regardless; either pass in the elements you need to put in the tuple, or use a list instead.