I am trying to modify a numpy array "in-place". I am interested in re-arranging the array in-place (instead of return:ing a re-arranged version of the array).
Here is an example code:
from numpy import *
def modar(arr):
arr=arr[[1,0]] # comment & uncomment this line to get different behaviour
arr[:,:]=0
print "greetings inside modar:"
print arr
def test2():
arr=array([[4,5,6],[1,2,3]])
print "array before modding"
print arr
print
modar(arr)
print
print "array now"
print arr
test2()
The assignment ar=arr[[1,0]] breaks the correspondence of "arr" to the original array passed to the function "modar". You can confirm this by commenting/uncommenting that line.. this happens, of course, as a new array has to be created.
How can I tell python that the new array still corresponds to "arr"?
Simply, how can I make "modar" to rearrange the array "in-place"?
Ok.. I modified that code and replaced "modarr" by:
def modar(arr):
# arr=arr[[1,0]] # comment & uncomment this line to get different behaviour
# arr[:,:]=0
arr2=arr[[1,0]]
arr=arr2
print "greetings inside modar:"
print arr
The routine "test2" still gets an unmodified array from "modar".
"For all cases of index arrays, what is returned is a copy of the
original data, not a view as one gets for slices."
http://docs.scipy.org/doc/numpy/user/basics.indexing.html
In this case you could do:
arr2 = arr[[1, 0]]
arr[...] = arr2[...]
where the temporary array arr2 is used to store the fancy indexing result. The last line copies the data from arr2 to the original array, keeping the reference.
Note: be sure in your operations that arr2 has the same shape of arr in order to avoid strange results...
Here is a solution with additional playing around. Basically the same as Saullo's.
from numpy import *
def modar1(arr):
# arr=arr[[1,0]] # (a)
arr[:,:]=arr[[1,0]][:,:] # (b)
print "greetings inside modar:"
print arr
# (a) arr is now referring to a new array .. python does not know if it
# has the same type / size as the original parameter array
# and therefore "arr" does not point to the original parameter array anymore. DOES NOT WORK.
#
# (b) explicit copy of each element. WORKS.
def modar2(arr):
arr2=arr.copy()
arr2=arr2[[1,0]]
# arr=arr2 # (a)
arr[:,:]=arr2[:,:] # (b)
print "greetings inside modar:"
print arr
# (a) same problem as in modar1
# .. it seems that *any* reference "arr=.." will point "arr" to something else as than original parameter array
# and "in-place" modification does not work. DOES NOT WORK
#
# (b) does an explicit copying of each array element. WORKS
#
def modar3(arr):
arr2=arr.copy()
arr2=arr2[[1,0]]
for i in range(arr.shape[0]):
arr[i]=arr2[i]
print "greetings inside modar:"
print arr
# this works, as there is no reference "arr=", i.e. to the whole array
def test2():
#
# the goal:
# give an array "arr" to a routine "modar"
# After calling that routine, "arr" should appear re-arranged
#
arr=array([[4,5,6],[1,2,3]])
print "array before modding"
print arr
print
modar1(arr) # OK
# modar2(arr) # OK
# modar3(arr) # OK
print
print "array now"
print arr
test2()
Related
I'm trying to dynamically add function calls to fill in array columns. I will be accessing the array millions of times so it needs to be quick.
I'm thinking to add the call of a function into a dictionary by using a string variable
numpy_array[row,column] = dict[key[index containing function call]]
The full scope of the code I'm working with is too large to post here is an equivalent simplistic example I've tried.
def hello(input):
return input
dict1 = {}
#another function returns the name and ID values
name = 'hello'
ID = 0
dict1["hi"] = globals()[name](ID)
print (dict1)
but it literally activates the function when using
globals()[name](ID)
instead of copy pasting hello(0) as a variable into the dictionary.
I'm a bit out of my depth here.
What is the proper way to implement this?
Is there a more efficient way to do this than reading into a dictionary on every call of
numpy_array[row,column] = dict[key[index containing function call]]
as I will be accessing and updating it millions of times.
I don't know if the dictionary is called every time the array is written to or if the location of the column is already saved into cache.
Would appreciate the help.
edit
Ultimately what I'm trying to do is initialize some arrays, dictionaries, and values with a function
def initialize(*args):
create arrays and dictionaries
assign values to global and local variables, arrays, dictionaries
Each time the initialize() function is used it creates a new set of variables (names, values, ect) that direct to a different function with a different set of variables.
I have an numpy array which I want to store information from the function and associated values created from the initialize() function.
So in other words, in the above example hello(0), the name of the function, it's value, and some other things as set up within initialize()
What I'm trying to do is add the function with these settings to the numpy array as a new column before I run the main program.
So as another example. If I was setting up hello() (and hello() was a complex function) and when I used initialize() it might give me a value of 1 for hello(1).
Then if I use initialize again it might give me a value of 2 for hello(2).
If I used it one more time it might give the value 0 for the function goodbye(0).
So in this scenaro let's say I have an array
array[row,0] = stuff()
array[row,1] = things()
array[row,2] = more_stuff()
array[row,3] = more_things()
Now I want it to look like
array[row,0] = stuff()
array[row,1] = things()
array[row,2] = more_stuff()
array[row,3] = more_things()
array[row,4] = hello(1)
array[row,5] = hello(2)
array[row,6] = goodbye(0)
As a third, example.
def function1():
do something
def function2():
do something
def function3():
do something
numpy_array(size)
initialize():
do some stuff
then add function1(23) to the next column in numpy_array
initialize():
do some stuff
then add function2(5) to the next column in numpy_array
initialize():
do some stuff
then add function3(50) to the next column in numpy_array
So as you can see. I need to permanently append new columns to the array and feed the new columns with the function/value as directed by the initialize() function without manual intervention.
So fundamentally I need to figure out how to assign syntax to an array column based upon a string value without activating the syntax on assignment.
edit #2
I guess my explanations weren't clear enough.
Here is another way to look at it.
I'm trying to dynamically assign functions to an additional column in a numpy array based upon the output of a function.
The functions added to the array column will be used to fill the array millions of times with data.
The functions added to the array can be various different function with various different input values and the amount of functions added can vary.
I've tried assigning the functions to a dictionary using exec(), eval(), and globals() but when using these during assignment it just instantly activates the functions instead of assigning them.
numpy_array = np.array((1,5))
def some_function():
do some stuff
return ('other_function(15)')
#somehow add 'other_function(15)' to the array column.
numpy_array([1,6] = other_function(15)
The functions returned by some_function() may or may not exist each time the program is run so the functions added to the array are also dynamic.
I'm not sure this is what the OP is after, but here is a way to make an indirection of functions by name:
def make_fun_dict():
magic = 17
def foo(x):
return x + magic
def bar(x):
return 2 * x + 1
def hello(x):
return x**2
return {k: f for k, f in locals().items() if hasattr(f, '__name__')}
mydict = make_fun_dict()
>>> mydict
{'foo': <function __main__.make_fun_dict.<locals>.foo(x)>,
'bar': <function __main__.make_fun_dict.<locals>.bar(x)>,
'hello': <function __main__.make_fun_dict.<locals>.hello(x)>}
>>> mydict['foo'](0)
17
Example usage:
x = np.arange(5, dtype=int)
names = ['foo', 'bar', 'hello', 'foo', 'hello']
>>> np.array([mydict[name](v) for name, v in zip(names, x)])
array([17, 3, 4, 20, 16])
I need to run a loop where I use the iterator only for the bound and access the result of the loop afterwards.
import tensorflow as tf
def add(h_tl):
res = tf.add(h_tl, tf.constant(1, shape=[2,1]))
return res
x = tf.constant(5) # iterator for while loop
h_tl = tf.constant(0, shape=[2,1])
H_l = tf.constant(0, shape=[2,1])
def body(x): # Body of loop
global H_l
global h_tl # I need to use H_l and h_tl in the body of the loop
h_tl = add(h_tl) # Some simple modifications to h_tl which will be added to H_l
H_l = tf.concat([H_l, h_tl], 1)
# I want the result of H_l after the loop and use it for further calculations
return (x+1)
def condition(x): # Stop condition
return x < tf.constant(10)
y = tf.while_loop(condition, body, [x])
sess = tf.Session()
print(sess.run(y))
print(sess.run(H_l)) # Result of the loop
sess.close()
So, I want to do calculations on H_l which is done for every x between 5 and 10. How do I access the end result of H_l after the loop is completed? When I run x I get the correct value of 10, but when I try to run H_l I get the following error: "Operation 'while/concat' has been marked as not fetchable."
TensorFlow while loops are designed to allow different iterations to be run in parallel. This causes them to behave rather differently from what you'd expect coming from other programming languages.
One such oddity is that if you create a tensor inside the loop, you cannot access it from outside. In your case, you are creating a new tensor with every tf.concat op. Yes, even though that tensor is stored under the same Python variable name as the previous version, it is still a new tensor: indeed, the original H_l is a constant, so its value cannot be changed; you can only create a new tensor in its place. The same is true for h_tl.
To solve this problem for H_l, what you want is to store an array of tensors, and add another entry to this array at each iteration. The data structure for doing this is tf.TensorArray, which you should initialize outside the loop and then write into at each iteration.
For h_tl, in which you only want to examine the final value, you don't need a tf.TensorArray; you could use a tf.Variable instead, and use tf.assign to update its value.
In both cases, take care to ensure that the mutating ops are executed, by including them in a tf.control_dependencies block.
When I assign a value into an array the scope of the variable remain local (see loc()).
However if I access the element of an array the scope becomes global ( see glob())
import numpy as np
M = np.array([1])
def loc():
M = 2
return 0
def glob():
M[0] = 3
return 0
loc()
print M
>>> [1]
glob()
print M
>>> [3]
Why does this happen ? How can i locally modify the elements of an array without modifying the array globally? I need to have a loop inside my function changing one element at a time.
You're mixing several things here.
First of all, M = 2 creates a local variable named M (you can see it in locals()) and prevents you from accessing the original M later on (although you're not doing it... But just to make a point). That's sometimes referred to as "shadowing".
Second of all, the np.array is a mutable object (the opposite of an immutable object), and changes to it will reflect in any reference to it. What you have in your glob function is a reference to M.
You can look at an np.array as a piece of memory that has many names, and if you changed it, the changes will be evident no matter what name you're using to access it. M[0] is simply a reference to a specific part of this memory. This reflects the object's "state".
If you'd do something like:
M = np.array([1])
def example()
another_name_for_M = M
M = 2
another_name_for_M[0] = 2
you would still see the global M changing, but you're using a new name to access it.
If you would use a string, a tuple, a frozenset and the likes, which are all immutable objects that can not be (easily) changed, you wouldn't be able to actually change their state.
Now to your question, if you don't want the function to mutate the array just send a copy of it using np.copy, and not the actual one:
import numpy as np
my_array = np.array([1])
def array_mutating_function(some_array):
some_array[0] = 1337
print some_array # prints [1337]
# send copy to prevent mutating the original array
array_mutating_function(np.copy(my_array))
print my_array # prints [1]
This will effectively make it immutable on the outer scope, since the function will not have a reference to it unless it's using it's name on the outer scope, which is probably not a good idea regardless.
If the function should never change any array, move the copy to be made on inside the function no matter what array is sent, preventing it from changing any array that was sent to it:
def array_mutating_function(some_array):
some_array = np.copy(some_array)
some_array[0] = 1337
SImply explaining.:
cannot update a global varaible inside a funcion unless access it as global inside function.
But it can modify
Check:
import numpy as np
M = np.array([1])
def loc():
global M
M = 2
return 0
def glob():
M[0] = 3
return 0
loc()
print M
>>>2
This is an issue specific to the use of python scripts in booggie 2.
I want to return multiple strings to the sequence and store them there in variables.
The script should look like this:
def getConfiguration(config_id):
""" Signature: getConfiguration(int): string, string"""
return "string_1", "string_2"
In the sequence I wanna have this:
(param_1, param_2) = getConfiguration(1)
Please note: The booggie-project does not exist anymore but led to the development of Soley Studio which covers the same functionality.
Scripts in booggie 2 are restricted to a single return value.
But you can return an array which then contains your strings.
Sadly Python arrays are different from GrGen arrays so we need to convert them first.
So your example would look like this:
def getConfiguration(config_id):
""" Signature: getConfiguration(int): array<string>"""
#TypeHelper in booggie 2 contains conversion methods from Python to GrGen types
return TypeHelper.ToSeqArray(["string_1", "string_2"])
return a tuple
return ("string_1", "string_2")
See this example
In [124]: def f():
.....: return (1,2)
.....:
In [125]: a, b = f()
In [126]: a
Out[126]: 1
In [127]: b
Out[127]: 2
Still, it's not possible to return multiple values but a python list is now converted into a C#-array that works in the sequence.
The python script itself should look like this
def getConfiguration(config_id):
""" Signature: getConfiguration(int): array<string>"""
return ["feature_1", "feature_2"]
In the sequence, you can then use this list as if it was an array:
config_list:array<string> # initialize array of string
(config_list) = getConfigurationList(1) # assign script output to that array
{first_item = config_list[0]} # get the first string("feature_1")
{second_item = config_list[1]} # get the second string("feature_2")
For the example above I recommend using the following code to access the entries in the array (in the sequence):
config_list:array<string> # initialize array of string
(config_list) = getConfigurationList(1) # assign script output to that array
{first_item = config_list[0]} # get the first string("feature_1")
{second_item = config_list[1]} # get the second string("feature_2")
I have a multi-dimensional array of objects. I want to interate over the objects using the nditer iterator.
Here is a code example:
import numpy as np
class Test:
def __init__(self,a):
self.a = a
def get_a(self):
return self.a
b = np.empty((2,3),dtype = object)
t_00 = Test(0)
t_01 = Test(1)
t_11 = Test (11)
b[0,0] = t_00
b[0,1] = t_01
b[1,1] = t_11
for item in np.nditer(b,flags = ["refs_ok"]):
if item:
print item.get_a()
I would expect the "item" to contain the object reference that I can use to access data.
However I am getting the following error:AttributeError: 'numpy.ndarray' object has no attribute 'get_a'
My question is how can I go thru the array to access the object in the array?
The array.flat method of iteration will work, and can confirm that this works as you'd expect
for item in b.flat:
if item:
print item.get_a()
Iterating over an array with nditer gives you views of the original array's cells as 0-dimensional arrays. For non-object arrays, this is almost equivalent to producing scalars, since 0-dimensional arrays usually behave like scalars, but that doesn't work for object arrays.
If you were determined to go through nditer for this, you could extract the elements from the 0-dimensional views with the item() method:
for element in np.nditer(b,flags = ["refs_ok"]):
element = element.item()
if element:
print(element.get_a())