When I assign a value into an array the scope of the variable remain local (see loc()).
However if I access the element of an array the scope becomes global ( see glob())
import numpy as np
M = np.array([1])
def loc():
M = 2
return 0
def glob():
M[0] = 3
return 0
loc()
print M
>>> [1]
glob()
print M
>>> [3]
Why does this happen ? How can i locally modify the elements of an array without modifying the array globally? I need to have a loop inside my function changing one element at a time.
You're mixing several things here.
First of all, M = 2 creates a local variable named M (you can see it in locals()) and prevents you from accessing the original M later on (although you're not doing it... But just to make a point). That's sometimes referred to as "shadowing".
Second of all, the np.array is a mutable object (the opposite of an immutable object), and changes to it will reflect in any reference to it. What you have in your glob function is a reference to M.
You can look at an np.array as a piece of memory that has many names, and if you changed it, the changes will be evident no matter what name you're using to access it. M[0] is simply a reference to a specific part of this memory. This reflects the object's "state".
If you'd do something like:
M = np.array([1])
def example()
another_name_for_M = M
M = 2
another_name_for_M[0] = 2
you would still see the global M changing, but you're using a new name to access it.
If you would use a string, a tuple, a frozenset and the likes, which are all immutable objects that can not be (easily) changed, you wouldn't be able to actually change their state.
Now to your question, if you don't want the function to mutate the array just send a copy of it using np.copy, and not the actual one:
import numpy as np
my_array = np.array([1])
def array_mutating_function(some_array):
some_array[0] = 1337
print some_array # prints [1337]
# send copy to prevent mutating the original array
array_mutating_function(np.copy(my_array))
print my_array # prints [1]
This will effectively make it immutable on the outer scope, since the function will not have a reference to it unless it's using it's name on the outer scope, which is probably not a good idea regardless.
If the function should never change any array, move the copy to be made on inside the function no matter what array is sent, preventing it from changing any array that was sent to it:
def array_mutating_function(some_array):
some_array = np.copy(some_array)
some_array[0] = 1337
SImply explaining.:
cannot update a global varaible inside a funcion unless access it as global inside function.
But it can modify
Check:
import numpy as np
M = np.array([1])
def loc():
global M
M = 2
return 0
def glob():
M[0] = 3
return 0
loc()
print M
>>>2
Related
I'm trying to dynamically add function calls to fill in array columns. I will be accessing the array millions of times so it needs to be quick.
I'm thinking to add the call of a function into a dictionary by using a string variable
numpy_array[row,column] = dict[key[index containing function call]]
The full scope of the code I'm working with is too large to post here is an equivalent simplistic example I've tried.
def hello(input):
return input
dict1 = {}
#another function returns the name and ID values
name = 'hello'
ID = 0
dict1["hi"] = globals()[name](ID)
print (dict1)
but it literally activates the function when using
globals()[name](ID)
instead of copy pasting hello(0) as a variable into the dictionary.
I'm a bit out of my depth here.
What is the proper way to implement this?
Is there a more efficient way to do this than reading into a dictionary on every call of
numpy_array[row,column] = dict[key[index containing function call]]
as I will be accessing and updating it millions of times.
I don't know if the dictionary is called every time the array is written to or if the location of the column is already saved into cache.
Would appreciate the help.
edit
Ultimately what I'm trying to do is initialize some arrays, dictionaries, and values with a function
def initialize(*args):
create arrays and dictionaries
assign values to global and local variables, arrays, dictionaries
Each time the initialize() function is used it creates a new set of variables (names, values, ect) that direct to a different function with a different set of variables.
I have an numpy array which I want to store information from the function and associated values created from the initialize() function.
So in other words, in the above example hello(0), the name of the function, it's value, and some other things as set up within initialize()
What I'm trying to do is add the function with these settings to the numpy array as a new column before I run the main program.
So as another example. If I was setting up hello() (and hello() was a complex function) and when I used initialize() it might give me a value of 1 for hello(1).
Then if I use initialize again it might give me a value of 2 for hello(2).
If I used it one more time it might give the value 0 for the function goodbye(0).
So in this scenaro let's say I have an array
array[row,0] = stuff()
array[row,1] = things()
array[row,2] = more_stuff()
array[row,3] = more_things()
Now I want it to look like
array[row,0] = stuff()
array[row,1] = things()
array[row,2] = more_stuff()
array[row,3] = more_things()
array[row,4] = hello(1)
array[row,5] = hello(2)
array[row,6] = goodbye(0)
As a third, example.
def function1():
do something
def function2():
do something
def function3():
do something
numpy_array(size)
initialize():
do some stuff
then add function1(23) to the next column in numpy_array
initialize():
do some stuff
then add function2(5) to the next column in numpy_array
initialize():
do some stuff
then add function3(50) to the next column in numpy_array
So as you can see. I need to permanently append new columns to the array and feed the new columns with the function/value as directed by the initialize() function without manual intervention.
So fundamentally I need to figure out how to assign syntax to an array column based upon a string value without activating the syntax on assignment.
edit #2
I guess my explanations weren't clear enough.
Here is another way to look at it.
I'm trying to dynamically assign functions to an additional column in a numpy array based upon the output of a function.
The functions added to the array column will be used to fill the array millions of times with data.
The functions added to the array can be various different function with various different input values and the amount of functions added can vary.
I've tried assigning the functions to a dictionary using exec(), eval(), and globals() but when using these during assignment it just instantly activates the functions instead of assigning them.
numpy_array = np.array((1,5))
def some_function():
do some stuff
return ('other_function(15)')
#somehow add 'other_function(15)' to the array column.
numpy_array([1,6] = other_function(15)
The functions returned by some_function() may or may not exist each time the program is run so the functions added to the array are also dynamic.
I'm not sure this is what the OP is after, but here is a way to make an indirection of functions by name:
def make_fun_dict():
magic = 17
def foo(x):
return x + magic
def bar(x):
return 2 * x + 1
def hello(x):
return x**2
return {k: f for k, f in locals().items() if hasattr(f, '__name__')}
mydict = make_fun_dict()
>>> mydict
{'foo': <function __main__.make_fun_dict.<locals>.foo(x)>,
'bar': <function __main__.make_fun_dict.<locals>.bar(x)>,
'hello': <function __main__.make_fun_dict.<locals>.hello(x)>}
>>> mydict['foo'](0)
17
Example usage:
x = np.arange(5, dtype=int)
names = ['foo', 'bar', 'hello', 'foo', 'hello']
>>> np.array([mydict[name](v) for name, v in zip(names, x)])
array([17, 3, 4, 20, 16])
I have a question on a fairly simple task in python, however, I didn't manage to find a solution. I would like to assign a new value to an already existing variable in python. However, the changes I am doing to the variable within the function don't stick to the variable.
Here is a simplified example of my problem:
y = 1
x = None
def test(var):
var = y
return var
test(x)
print(x)
The print simply returns none. So the changes I have done to the variable within the function are non permanent.
How can I make the changes on the input-variable of the function permanent?
Thanks in advance!
Variables in Python are just names which refer to objects. In an expression, the name is a stand-in for the actual object. Saying test(x) means "pass the object referred to by x into test". It does not mean "pass the symbol x into test".
In addition, re-assigning a name only changes what object that name refers to. It affects neither the object nor any of its aliases.
In short, the name var you modify inside test has no relation to x at all.
The preferred way to have a function change something is by reassigning the result:
x = 2
def change(var):
return var * 2
x = change(x) # x now refers to 4 instead of 2
print(x)
If you want to change a name outside a function, you can use the nonlocal and global keywords:
x = 2
def change_x():
global x
x = x * 2
change_x() # x now refers to 4 instead of 2
print(x)
While this can make some trivial problems easy to solve, it is generally a bad idea for larger programs. Using global variables means one can no longer use the function in isolation; results may depend on how often and in what order such a function is called.
If you have some self-contained group of values and means to modify them, a class can be used to describe this:
class XY:
def __init__(self, x, y):
self.x, self.y = x, y
def swap(self):
self.x, self.y = self.y, self.x
my_values = XY(None, 1)
print(my_values.x, my_values.y)
my_values.swap()
print(my_values.x, my_values.y)
In contrast to global variables, you can create as many isolated instances of classes as needed. Each instance can be worked on in isolation, without affecting the others.
You can also use mutable values to make changes visible to the outside. Instead of changing the name, you modify the value.
x = [2] # x is a mutable list, containing the value 2 at the moment
def change(var):
var[0] = 4 # change leading element of argument
change(x) # x now contains 4 instead of 2
print(x)
This is an example of passing variables to functions by value. By default, when you pass a variable to a function in Python, it is passed by value.
What it means is, that a new variable with a new scope is created with the same value of x. So, any change that happens to the new x is not reflected to the x outside the function's scope.
If you want to get the value from the function back, you can use the return statement (as you have used). return returns the value back from the function. However, in your example there is no variable to receive it. Hence, it is lost.
You would have to call the function as x = test(x). This ensures that x receives the value back from the function.
I am trying to understand what does this class
class undo:
def __init__(self, ss):
self.ss = ss
In my head it should simply put the value of ss uniquely in the variables i decide to use,but when I'm using it it rewrites all the variables, as if it was shared.
sum_expenses[100][0] = 100
sum_expenses[99][2] = 99
s = 1
a = [0 for i in range(100)]
a[s] = undo(sum_expenses)
output(a[1].ss)
sum_expenses[100][0] = 0
b = undo(sum_expenses)
print " "
print b.ss
print " "
sum_expenses[99][2] = 1
a[2] = undo(sum_expenses)
print a[2].ss
I do not understand why it overwrites all the variables with the current values of sum_expense, when I try to put it individually so that I can use the past values of sum_expenses.
Thank you and have a good day!
It happens because you're giving __init__ a reference to the list. If you change the list somewhere else, the reference in .ss still points to the same list, so the changes are visible there, too.
You could copy the list, since it's 2D try deepcopy.
Everytime you call this function it overrides reference to thr whole array ss, changing it to the new one you just passed to the constructor.
You can also create this array within the class and pass indexes and value to it like so:
undo.add(index1,index2,value)
Or you can make another variable ss_old and have make the function return that variable before you set it to a new value.
I am trying to build some code and I have defined a function as this to test how a counter works inside of the function:
def errorPrinting(x):
x += 1
return x
I then use the function in some conditional logic where I want the counter to increase if the conditions are met.
x = 1
for row in arcpy.SearchCursor(fc):
if not row.INCLUSION_TYPE or len(row.TYPE.strip()) == 0:
errorPrinting(x)
print x
elif len(row.TYPE) not in range(2,5):
errorPrinting(x)
print x
elif row.INCLUSION_TYPE.upper() not in [y.upper() for y in TableList]:
errorPrinting(x)
print x
I'm still fairly new with using functions, so maybe I am not understanding how to return the value back outside of the function to be used in the next iteration of the for loop. It keeps returning 1 on me. Can anyone show me how to return the x outside of the function after it has been increased by one x+= 1?
Thanks,
Mike
You're not incrementing your global x, you're incrementing the local paramater that also happens to be named x! (Your parameter to errorPrinting could have been named anything. I'm calling it xLocal.)
As you can see here, x isn't incremented by the function.
>>> def inc(xLocal):
... xLocal += 1
... return xLocal
...
>>> x = 4
>>> inc(x)
5
>>> x
4
You need to reassign the value of x to the return value of the function each time. Like this
x = 1
for row in arcpy.SearchCursor(fc):
if not row.INCLUSION_TYPE or len(row.TYPE.strip()) == 0:
x = errorPrinting(x) # <=== here
print x
elif len(row.TYPE) not in range(2,5):
x = errorPrinting(x) # <=== here
print x
elif row.INCLUSION_TYPE.upper() not in [y.upper() for y in TableList]:
x = errorPrinting(x) # <=== here
print x
Integral parameters and other primitives aren't normally passed by reference in Python. (Lists, dicts, etc. are. Modifying lists unintentionally is actually a very common mistake in Python.)
Edit: passing by "reference" and "value" isn't really correct to talk about in Python. See this nice question for more details.
So, using my previous example:
>>> x = 4
>>> x = inc(x)
>>> x
5
Note that if this had been parameter that is passed by reference, like a list, this strategy would have worked.
>>> def incList(xList):
... for i in range(len(xList)):
... xList[i] += 1
...
>>> xList
[1]
>>> incList(xList)
>>> xList
[2]
Note that normal, Pythonic syntax:
for i in xList:
i += 1
would not increment the global value.
Note: If you're looking to keep tabs on a lot of things, I also recommend the logging module that #SB. mentioned. It's super useful and makes debugging large programs a lot easier. You can get time, type of message, etc.
You're bit by scope. You may want to check out this link for a quick primer.
You can do something simple and say x = errorPrinting(x) in all cases you call errorPrinting and get what you want. But I think there are better solutions where you'll learn more.
Consider implementing an error logger object that maintains a count for you. Then you can do logger.errorPrinting() and your instance of logger will manage the counter. You may also want to look into python's built in logging facilities.
Edited for the OP's benefit, since if functions are a new concept, my earlier comments may be a little hard to follow.
I personally think the nicest way to address this issue is to wrap your related code in an object.
Python is heavily based on the concept of objects, which you can think of as grouping data with functions that operate on that data. An object might represent a thing, or in some cases might just be a convenient way to let a few related functions share some data.
Objects are defined as "classes," which define the type of the object, and then you make "instances," each of which are a separate copy of the grouping of data defined in the class.
class MyPrint(object):
def __init__(self):
self.x = 1
def errorPrinting(self):
self.x += 1
return self.x
def myPrint(self):
for row in arcpy.SearchCursor(fc):
if not row.INCLUSION_TYPE or len(row.TYPE.strip()) == 0:
self.errorPrinting()
print self.x
elif len(row.TYPE) not in range(2,5):
self.errorPrinting()
print self.x
elif row.INCLUSION_TYPE.upper() not in [y.upper() for y in TableList]:
self.errorPrinting()
print self.x
p = MyPrint()
p.myPrint()
The functions __init__(self), errorPrinting(self), and myPrint(self), are all called "methods," and they're the operations defined for any object in the class. Calling those functions for one of the class's instance objects automatically sticks a self argument in front of any arguments that contains a reference to the particular instance that the function is called for. self.x refers to a variable that's stored by that instance object, so the functions can share that variable.
What looks like a function call to the class's name:
p = MyPrint()
actually makes a new instance object of class MyPrint, calls MyPrint.__init__(<instance>), where <instance> is the new object, and then assigns the instance to p. Then, calling
p.myprint()
calls MyPrint.myprint(p).
This has a few benefits, in that variables you use this way only last as long as the object is needed, you can have multiple counters for different tasks that are doing the same thing, and scope is all taken care of, plus you're not cluttering up the global namespace or having to pass the value around between your functions.
The simplest fix, though perhaps not the best style:
def errorPrinting():
global x
x += 1
Then convert x=errorPrinting(x) to errorPrinting ()
"global x" makes the function use the x defined globally instead of creating one in the scope of the function.
The other examples are good though. Study all of them.
I'm trying to create a Python script that opens several databases and compares their contents. In the process of creating that script, I've run into a problem in creating a list whose contents are objects that I've created.
I've simplified the program to its bare bones for this posting. First I create a new class, create a new instance of it, assign it an attribute and then write it to a list. Then I assign a new value to the instance and again write it to a list... and again and again...
Problem is, it's always the same object so I'm really just changing the base object. When I read the list, I get a repeat of the same object over and over.
So how do you write objects to a list within a loop?
Here's my simplified code
class SimpleClass(object):
pass
x = SimpleClass
# Then create an empty list
simpleList = []
#Then loop through from 0 to 3 adding an attribute to the instance 'x' of SimpleClass
for count in range(0,4):
# each iteration creates a slightly different attribute value, and then prints it to
# prove that step is working
# but the problem is, I'm always updating a reference to 'x' and what I want to add to
# simplelist is a new instance of x that contains the updated attribute
x.attr1= '*Bob* '* count
print "Loop Count: %s Attribute Value %s" % (count, x.attr1)
simpleList.append(x)
print '-'*20
# And here I print out each instance of the object stored in the list 'simpleList'
# and the problem surfaces. Every element of 'simpleList' contains the same attribute value
y = SimpleClass
print "Reading the attributes from the objects in the list"
for count in range(0,4):
y = simpleList[count]
print y.attr1
So how do I (append, extend, copy or whatever) the elements of simpleList so that each entry contains a different instance of the object instead of all pointing to the same one?
You demonstrate a fundamental misunderstanding.
You never created an instance of SimpleClass at all, because you didn't call it.
for count in xrange(4):
x = SimpleClass()
x.attr = count
simplelist.append(x)
Or, if you let the class take parameters, instead, you can use a list comprehension.
simplelist = [SimpleClass(count) for count in xrange(4)]
A list comprehension can be used to fill a list with separate instances of a class, like so:
instancelist = [MyClass() for i in range(29)]
This avoids the problem with multiplying a list of one element with *, which re-uses the same object.
It shouldn't be necessary to recreate the SimpleClass object each time, as some are suggesting, if you're simply using it to output data based on its attributes. However, you're not actually creating an instance of the class; you're simply creating a reference to the class object itself. Therefore, you're adding a reference to the same class attribute to the list (instead of instance attribute), over and over.
Instead of:
x = SimpleClass
you need:
x = SimpleClass()
Create a new instance each time, where each new instance has the correct state, rather than continually modifying the state of the same instance.
Alternately, store an explicitly-made copy of the object (using the hint at this page) at each step, rather than the original.
If I understand correctly your question, you ask a way to execute a deep copy of an object.
What about using copy.deepcopy?
import copy
x = SimpleClass()
for count in range(0,4):
y = copy.deepcopy(x)
(...)
y.attr1= '*Bob* '* count
A deepcopy is a recursive copy of the entire object. For more reference, you can have a look at the python documentation: https://docs.python.org/2/library/copy.html
I think this simply demonstrates what you are trying to achieve:
# coding: utf-8
class Class():
count = 0
names = []
def __init__(self,name):
self.number = Class.count
self.name = name
Class.count += 1
Class.names.append(name)
l=[]
l.append(Class("uno"))
l.append(Class("duo"))
print l
print l[0].number, l[0].name
print l[1].number, l[1].name
print Class.count, Class.names
Run the code above and you get:-
[<__main__.Class instance at 0x6311b2c>,
<__main__.Class instance at 0x63117ec>]
0 uno
1 duo
2 ['uno', 'duo']