Using numpy and matplotlib it seems quite common that functions allow both a number (float or int) or a numpy array as argument like this:
import numpy as np
print np.sin(0)
# 0
x = np.arange(0,4,0.1)
y = np.sin(x)
In this example I call np.sin once with an integer argument, and once with a numpy array x. I now want to write a function that allows similar treatment, but I don't know how. For example:
def fun(foo, n):
a = np.zeros(n)
for i in range(n):
a[i] = foo
return a
would allow me to call fun like fun(1, 5) but not like fun(x, 5). My actual calculation is much more complicated, of course.
How can I initialize a such that I can have both simple numbers or a whole array of numbers as elements?
Thanks a lot for your help!
Builtin numpy functions often start with a
def foo(a, ...):
a = np.asarray(a)
...
That is, they transform the input argument(s) to array (no copy if it is already an array). The allows them to work with scalars and lists.
Once the argument is an array it has a shape and can be broadcasted against other arguments.
In your example, it's unclear what is supposed to happen when foo is an array
def fun(foo, n):
a = np.zeros(n)
for i in range(n):
a[i] = foo
return a
a is initialized as a dtype float array. That means a[i]=foo works only if foo is a single element number (scalar, possibly a single element array). If foo is an array with more than one value you probably get an error about attempting to set an element with a sequence.
a[i] is short for a[i,...]. That is it indexes on the 1st dimension. So if a was initialed correctly it could accept arrays as inputs (subject to broadcasting rules).
If a was initialed as np.zeros(n, dtype=object), then a[i]=foo will work with anything, since it a just contains pointers to Python objects.
np.frompyfunc is a way of generating an array from a function. But it returns an array of dtype=object. np.vectorize uses that but gives you more control over the output type. But both work with scalars. An array, if given as argument, is passed element by element to the function.
You need a to inherit the dimensions of foo:
def fun(foo, n):
a = np.zeros((n,) + np.shape(foo))
for i in range(n):
a[i] = foo
return a
You can use type identification :
import numpy as np
def double(a):
if type(a)==int:
return 2*a
elif type(a)==float:
return 2.0*a
elif type(a)==list:
return [double(x) for x in a]
elif type(a)==np.ndarray:
return 2*a
else:
print "bad type"
print double(7)
print double(7.2)
print double([2,9,7])
print double(np.array([[9,8],[2,3]]))
result :
>14
>14.4
>[4, 18, 14]
>[[18 16]
[ 4 6]]
with eventually a recursive treatement as I did on list
Related
Updated Question
Following from my original post, with the use of #Attack68 's code, I have created a program that successfully evolved the function with a choice of multiplicative functions based on a random variable. However, now I am receiving an error saying the list indices must be integers (even though I'm fairly sure they are), I'm not sure what has happened, The code is as follows:
import numpy as np
import scipy.integrate as integrate
x=np.linspace(0.0,1.0,100)
n=10 #iterations
d=700.0
def f(x):
return np.sin(x)
def g(x,list_):
return np.cos(x)*apply(x,list_)
base = [f, g]
list_ = list()
for i in range(n):
testvar=np.random.randint(1, 100, 1)
if testvar> 50 and i!=0:
func_idx = 0 # choose a random operation: 0=ten, 1=inv
else:
func_idx= 1
list_.append(func_idx)
# now you have a list of indexes referencing your base functions so you can apply them:
def apply(x,list_):
y = 1
for i in range(len(list_)):
y *= base[list_[i]](x)
return y
print(list_)
#testint=integrate.quad(apply(x,list_),-d,d)[0]
#print(testint)
print(apply(list_, x))
I am now getting the error:
TypeError: list indices must be integers or slices, not numpy.float64
I am also attempting to get this to integrate the new function after each iteration but it seems that the form of this function is not callable by scipys quad integrator, any suggestions on how to integrate the evolving function on each iteration would also be appreciated.
Original:
I am creating a simulation in python where I consider a function that evolves over a loop. This function starts off defined as:
def f(x):
return 1.0
So simply a flat distribution. After each iteration of the loop, I want the function to be redefined depending on certain (random) conditions. It could be multiplied by cos(b*x) or it could be multiplied by some function A(x), the evolution will not be the same each time due to the randomness, so I cannot simply multiply by the same value each time.
The progression in one instance could be:
f(x)----> f(x)*A(x)----> f(x)*A(x)*A(x)...
but in another instance it could be:
f(x)----> f(x)*A(x)----> f(x)*A(x)*cos(x)...
or
f(x)----> f(x)*cos(x)----> f(x)*cos(x)*cos(x)...
etc.
after each, of n iterations of this evolution, I have to compute an integral that is related to the function, so I need to essentially update the function after each iteration to be called by scipys quad integrator.
I have tried to use arrays to manipulate the distribution instead and it works as far as the function evolution goes, but upon integration, it gives the incorrect result with numpy.trapz and I cannot work out why. Sci-pys quad integrator is more accurate anyway and I had managed to get this to work previously for the first iteration only, but it requires a function based input, so without this function evolution I cannot use it.
If someone could show me if/how this function evolution is possible that'd be great. If it is not possible, perhaps someone could try to help me understand what numpy.trapz actually does so I can workout how to fix it?
How about this:
class MyFunction:
def __init__(self):
def f1(x):
return 1.0
self.functions = [f1]
def append_function(self, fn):
self.functions.append(fn)
def __call__(self, x):
product = 1.0
for f in self.functions:
product *= f(x)
return product
This object starts off as simply returning 1.0. Later you add more functions and it returns the product of all of them.
Your description suggests your iterated values are combined through a product and are not in fact a composition of functions. A simple way of recording these is to have a set of base functions:
import numpy as np
import scipy.integrate as int
def two(x):
return x*2
def inv(x):
return 1/x
base = [two, inv]
funcs = np.random.choice(base, size=10)
def apply(x, funcs):
y = 1
for func in funcs:
y *= func(x)
return y
print('function value at 1.5 ', apply(1.5, funcs))
answer = int.quad(apply, 1, 2, args=(funcs,))
print('integration over [1,2]: ', answer)
I am using the numpy.random.choice module to generate an 'array' of choices based on an array of functions:
def f(x):
return np.sin(x)
def g(x):
return np.cos(x)
base=[f, g]
funcs=np.random.choice(base,size=2)
This code will produce an 'array' of 2 items referencing a function from the base array.
The reason for this post is, I have printed the outcome of funcs and recieved:
[<function f at 0x00000225AC94F0D0> <function f at 0x00000225AC94F0D0>]
Clearly this returns a reference to the functions in some form, not that I understand what that form is or how to manipulate it, this is where the problem comes in. I want to change the choice of function, so that it is no longer random and instead depends on some conditions, so it might be:
for i in range(2):
if testvar=='true':
choice[i] = 0
if testvar== 'false':
choice[i] = 1
This would return an array of indicies to be put in later function
The problem is, the further operations of the code (I think) require this previous form of function reference: [ ] as an input, instead of a simple array of 0,1 Indicies and I don't know how I can get an array of form [ ] by using if statements.
I could be completely wrong about the rest of the code requiring this input, but I don't know how I can amend it, so am hence posting it here. The full code is as follows: (it is a slight variation of code provided by #Attack68 on Evolving functions in python) It aims to store a function that is multiplied by a random function on each iteration and integrates accordingly. (I have put a comment on the code above the function that is causing the problem)
import numpy as np
import scipy.integrate as int
def f(x):
return np.sin(x)
def g(x):
return np.cos(x)
base = [f, g]
funcs = np.random.choice(base, size=2)
print(funcs)
#The below function is where I believe the [<function...>] input to be required
def apply(x, funcs):
y = 1
for func in funcs:
y *= func(x)
return y
print('function value at 1.5 ', apply(1.5, funcs))
answer = int.quad(apply, 1, 2, args=(funcs,))
print('integration over [1,2]: ', answer)
Here is my attempt of implementing a non-random event:
import numpy as np
import scipy.integrate as int
import random
def f(x):
return np.sin(x)
def g(x):
return np.cos(x)
base = [f, g]
funcs = list()
for i in range(2):
testvar=random.randint(0,100) #In my actual code, this would not be random but dependent on some other situation I have not accounted for here
if testvar>50:
func_idx = 0 # choose a np.random operation: 0=f, 1=g
else:
func_idx= 1
funcs.append(func_idx)
#funcs = np.random.choice(base, size=10)
print(funcs)
def apply(x, funcs):
y = 1
for func in funcs:
y *= func(x)
return y
print('function value at 1.5 ', apply(1.5, funcs))
answer = int.quad(apply, 1, 2, args=(funcs,))
print('integration over [1,2]: ', answer)
This returns the following error:
TypeError: 'int' object is not callable
If: You are trying to refactor your original code that operates on a list of randomly chosen functions to a version that operates with random indices which correspond to items in a list of functions. Refactor apply.
def apply(x,indices,base=base):
y = 1
for i in indices:
f = base[i]
y *= f(x)
return y
...this returns a reference to the functions in some form, not that I understand what that form is or how to manipulate it...
Functions are objects, the list contains a reference to the objects themselves. They can be used by either assigning them to a name then calling them or indexing the list and calling the object:
>>> def f():
... return 'f'
>>> def g():
... return 'g'
>>> a = [f,g]
>>> q = a[0]
>>> q()
'f'
>>> a[1]()
'g'
>>> for thing in a:
print(thing())
f
g
Or you can pass them around:
>>> def h(thing):
... return thing()
>>> h(a[1])
'g'
>>>
If you still want to use your function apply as-is, you need to keep your input a list of functions. Instead of providing a list of indices, you can use those indices to create your list of functions.
Instead of apply(1.5, funcs), try:
apply(1.5, [base(n) for n in funcs])
In Python, I would like to have a function working on different input types. Something like this:
def my_square(x):
return x ** 2
my_square(2) #return 4
my_square(range(10)) #should return a list [0 ... 81]
npa = numpy.zeros(10)
my_square(npa) # should return a numpy array with the squares of zeros
Basically, what is good practice to write functions for both scalars and iterables? Can this be done with *args or *kwargs perhaps?
A typical way to do this is to use numpy.asarray to convert the input of your function to an ndarray. If the input is a scalar, then the result is an array with 0 dimensions, which acts essentially like a Python number type. For instance:
def square(x):
x = np.asarray(x)
return x**2
So that:
>>> square(4)
16
>>> square([1, 2, 3, 4])
array([ 1, 4, 9, 16])
Note that I gave a list as input and received an ndarray as output. If you absolutely must receive the same type of output as you provided as input, you can convert the result before returning it:
def square(x):
in_type = type(x)
x = np.asarray(x)
return in_type(x**2)
But this incurs an additional cost for little benefit.
Since Python is dynamically typed, and the design philosophy of Python is against the idea of wanting a difference to be obvious when the function is called, this isn't considered Pythonic. You can however use the isInstance() method to accomplish what you want, if you must that is. For example:
def mySquare(x):
if isinstance(x, int):
return x**2
elif isinstance(x, range):
return [i ** 2 for i in x]
print(mySquare(2)); //4
print(mySquare(range(10))); //[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
IDEOne link here.
However, just because you can do something, doesn't mean you should.
Refer to this question for further information on isinstance, and I suggest you take a look at Duck Typing as well.
Additionally, a single dispatch function might also provide what you need, but I am not experienced enough to provide an explanation for this, however, it might be something you want to look into.
As suggested in the comments, just write a scalar version of your function and use map for lists etc and imap for iterables (map will not work on those):
map(myFunc,myList)
and
import itertools
itertools.imap(myFunc,myIterable)
It would be much better practice and much more maintainable to just do:
def foo(n):
return n ** 2
And then build lists, dicts, etc when needed (I'm not super familiar with numpy but I imagine there is something similar you could do):
foo_list = [foo(n) for n in range(10)]
foo_dict = {n: foo(n) for n in range(10)}
And it seems that there is for numpy using numpy.fromiter(). From the docs:
iterable = (foo(n) for n in range(5))
foo_arr = np.fromiter(iterable, np.whatever_numpy_datatype)
you could even make those into functions if you really needed to:
def foo_list(start=0, stop=0):
return [foo(n) for n in range(start, stop)]
def foo_dict(start=0, stop=0):
return {n: foo(n) for n in range(10)}
Another option as it's easier to ask for forgiveness than permission:
def foo(scalar_or_iter):
try:
return [n ** 2 for n in scalar_or_iter]
except TypeError:
return scalar_or_iter ** 2
How can I write a function that works like array.append() for numpy.array?
I have tried this
import numpy as np
def append_np(ar, el):
ar = np.append(ar, el)
z = np.array([5], dtype='int32')
z = np.append(z, 6)
append_np(z, 7)
print z
but this code appends only '6':
[5 6]
"that works like array.append()"
First of all, the data structure in Python you most likely are referring to here as "array" is called "list".
Then, the append() methods for Python lists and Numpy arrays behave fundamentally different. Say that l is a Python list. l.append() modifies the list in-place and returns None. In contrast, Numpy's append() method for arrays does not change the array it operates on. It returns a new array object.
See: http://docs.scipy.org/doc/numpy/reference/generated/numpy.append.html
A copy of arr with values appended to axis. Note that append does not
occur in-place: a new array is allocated and filled.
This explains why you need to return the result of your append_np() function and assign the return value, as in new_z = append_np(z, 7).
You have probably used this function for a Python list:
def append(ar, el):
ar = ar.append(el)
and called it like this:
z = [1, 2]
append(z, 7)
print z
And you have seen that it has modified your z, indeed. But why, what has happened in this function? The object that was passed as first argument (bound to the name ar) got modified in-place. That is why z "on the outside" changed. You made use of this side effect of the function without knowing, and this is dangerous. Within the function the name ar got re-assigned to the None singleton object (which is the return value of the list append method). You did not return this object or use it, so this assignment served no purpose in your program. You discovered yourself that this approach is problematic, because when you re-structured your function to append_np() you suddenly realized that it did not have a "side effect" on z.
That is, for Python lists you would not outsource the append operation into another function. You would just, from the very beginning, state:
z = [1, 2]
z.append(7)
print z
I've recently run into issues when creating Numpy object arrays using e.g.
a = np.array([c], dtype=np.object)
where c is an instance of some complicated class, and in some cases Numpy tries to access some methods of that class. However, doing:
a = np.empty((1,), dtype=np.object)
a[0] = c
solves the issue. I'm curious as to what the difference is between these two internally. Why in the first case might Numpy try and access some attributes or methods of c?
EDIT: For the record, here is example code that demonstrates the issue:
import numpy as np
class Thing(object):
def __getitem__(self, item):
print "in getitem"
def __len__(self):
return 1
a = np.array([Thing()], dtype='object')
This prints out getitem twice. Basically if __len__ is present in the class, then this is when one can run into unexpected behavior.
In the first case a = np.array([c], dtype=np.object), numpy knows nothing about the shape of the intended array.
For example, when you define
d = range(10)
a = np.array([d])
Then you expect numpy to determine the shape based on the length of d.
So similarly in your case, numpy will attempt to see if len(c) is defined, and if it is, to access the elements of c via c[i].
You can see the effect by defining a class such as
class X(object):
def __len__(self): return 10
def __getitem__(self, i): return "x" * i
Then
print numpy.array([X()], dtype=object)
produces
[[ x xx xxx xxxx xxxxx xxxxxx xxxxxxx xxxxxxxx xxxxxxxxx]]
In contrast, in your second case
a = np.empty((1,), dtype=np.object)
a[0] = c
Then the shape of a has already been determined. Thus numpy can just directly assign the object.
However to an extent this is true only since a is a vector. If it had been defined with a different shape then method accesses will still occur. The following for example will still call ___getitem__ on a class
a = numpy.empty((1, 10), dtype=object)
a[0] = X()
print a
returns
[[ x xx xxx xxxx xxxxx xxxxxx xxxxxxx xxxxxxxx xxxxxxxxx]]