Develop Reference

matlab string array to python numpy

I have an array of strings in Matlab and I want to use it with python. I tried to do the same process as for numeric arrays i.e.
np_array = scipy.io.loadmat(mat_array)
which works with numeric data.
But when I try to use a string array like
mat_str_array = [“This”, “is”, “a”, “string”]
I get an error/warning stating
cannot be transformed using the same method
Any suggestions?
Thanks in advance.

Method expects argument of type 'double *'?

I'm new to python and I'm trying to write some code using python wrappers for epanet which I believe are SWIG-wrapped. There is a method I need to use, as outlined below from the source. However I can't seem to get the method to accept any type of value for the 3rd argument "values:".. Anything I try I get the error: TypeError: in method 'ptrn_set', argument 3 of type 'double *'
How do I convert my value/values to 'double *' within Python? what is the solution?
Thankful for any help...
"""
ptrn_set(ph, index, values, len) -> int
Parameters
----------
ph: Handle
index: int
values: double *
len: int
"""
return _toolkit.ptrn_set(ph, index, values, len)

I found this here:
https://docs.python.org/2.4/lib/typesnumeric.html:
Floating point numbers are implemented using double in C
Did you try float?

Python itself does not support values of type double *. In C, that is a pointer-to-a-double-precision-floating-point-number. In Python, effectively every name is a pointer, but you can't have a (native) value that is a pointer.
From context, it looks like they actually want an array of doubles. They really ought to write that as double [], but double * is compatible with it. You could try
values = [1.0, 2.0]
_toolkit.ptrn_set(ph, index, values, len(values))
It's not very Pythonic to do that, but if it works, it works. If that doesn't work, see if your expanet python wrappers provide a way of constructing such values.
If your expanet Python wrappers don't provide a way to construct such values, you can try the cffi library, but it's quite tricky. I think x = ffi.new("double[10]") will give you an array of 10 doubles.

I know some time passed an you probably already solved it, but it might be useful for someone else.
you can use the following function to create an array that will be accepted by epanet:
def make_array(values):
dbl_arr = en.doubleArray(len(values))
for i in range(len(values)):
dbl_arr[i] = values[i]
return dbl_arr
the input is an array of float, like [1.2 , 0.8].
This assuming you imported the epanet python library as
from epanet import toolkit as en,
source: https://github.com/OpenWaterAnalytics/epanet-python/issues/57

Python String representation of numpy array

Iam trying to get a FULL string representation of an 2D float32 512x512 array. I can either use numpys string2array(array) or repr(array). But the problem is that I always get a shortened output like that:
'...[2.0886018e-04 1.7029114e-04 2.8904244e-05 ... 4.1985390e-06
1.4982919e-06 4.7537060e-06]]'
Is there a possibility of full representation?

The answer is that numpy array2string has a threshold option which can be set to np.inf. Then a full representation is possible! :)

Does numpy methods work correctly on numbers too big to fit numpy dtypes?

I would like to know if numbers bigger than what int64 or float128 can be correctly processed by numpy functions
EDIT: numpy functions applied to numbers/python objects outside of any numpy array. Like using a np function in a list comprehension that applies to the content of a list of int128?
I can't find anything about that in their docs, but I really don't know what to think and expect. From tests, it should work but I want to be sure, and a few trivial test won't help for that. So I come here for knowledge:
If np framework is not handling such big numbers, are its functions able to deal with these anyway?
EDIT: sorry, I wasn't clear. Please see the edit above
Thanks by advance.

See the Extended Precision heading in the Numpy documentation here. For very large numbers, you can also create an array with dtype set to 'object', which will allow you essentially to use the Numpy framework on the large numbers but with lower performance than using native types. As has been pointed out, though, this will break when you try to call a function not supported by the particular object saved in the array.
import numpy as np
arr = np.array([10**105, 10**106], dtype='object')
But the short answer is that you you can and will get unexpected behavior when using these large numbers unless you take special care to account for them.

When storing a number into a numpy array with a dtype not sufficient to store it, you will get truncation or an error
arr = np.empty(1, dtype=np.int64)
arr[0] = 2**65
arr
Gives OverflowError: Python int too large to convert to C long.
arr = np.empty(1, dtype=float16)
arr[0] = 2**64
arr
Gives inf (and no error)
arr[0] = 2**15 + 2
arr
Gives [ 32768.] (i.e., 2**15), so truncation occurred. It would be harder for this to happen with float128...

You can have numpy arrays of python objects, which could be a python integer which is too big to fit in np.int64. Some of numpy's functionality will work, but many functions call underlying c code which will not work. Here is an example:
import numpy as np
a = np.array([123456789012345678901234567890]) # a has dtype object now
print((a*2)[0]) # Works and gives the right result
print(np.exp(a)) # Does not work, because "'int' object has no attribute 'exp'"
Generally, most functionality will probably be lost for your extremely large numbers. Also, as it has been pointed out, when you have an array with a dtype of np.int64 or similar, you will have overflow problems, when you increase the size of your array elements over that types limit. With numpy, you have to be careful about what your array's dtype is!

Return type of argmin method

As usual, I become quite confused about numpy data types.
Suppose I define a numpy array:
>>> a = np.array([1.,2.,3.])
Somewhere in the code I want to know if a is a numpy array versus some other type of data type. The following provides the answer:
>>> print type(a)
<type 'numpy.ndarray'>
I did this from within iPython. It appears an object of type "type" is returned, but I don't really know how to process "type" objects in a program. I can also use isinstance to verify that a is a numpy array:
>>> isinstance(a, np.ndarray)
True
(Side question: how would I return an easily interpretable object to figure out what a is? The documentation says type returns the type of an object, but what kind of thing is a type of an object? It's not a string, or a number. It's some sort of object. I seem to be able to convert the type object to a string and then parse it with split() for example, but this seems dangerous. Is there sufficient documentation on the objects of type "type" that I could guarantee that when I pass it to str I get something I can parse in a standard way?)
Back to the main question.
I now use the argmin method on a:
>>> f = a.argmin()
I expected that the type of f is an integer, but its not. For example:
>>> type(3)
int
gives "int". But type f gives:
>>> type(f)
numpy.int32
I don't know what the answer numpy.int32 is telling me. It's not a numpy array:
>>> isinstance(f, np.ndarray)
False
But f is also not an integer. What is it? This is relevant, because I passed this numpy.int32 object to a function, and it created a mess. When I first convert to int, and then pass it to the function, everything worked fine. Clearly, the function wanted an int, but I passed it a numpy.int32 instead and got bad results.
The doc string for the argmin() method says it returns "indices". I am not sure what is. I assumed it would be an integer, or a numpy array of integers. But "indices" is neither. It's this "numpy.int32" thing. What is that?
Thanks for your help.
(Side discussion continued: I still seek a numpy function or method that returns the type of an object in some easily-interpretable form. type returns an object of type "type", but I don't know how to write a program that queries type objects. I don't really know what a type object is or how to interact with it. It would be great to have a numpy function that returns something simple, and tells me "this is a numpy array of floats" or "this is a scalar float" or "this is a numpy.int32" (whatever that is). I have yet to find a way to do this).

You want to check for a numpy ndarray's .dtype attribute. If you want a human to interpret the results, then things like np.int32 mean "numpy integer type of 32 bits." If you want to know this programatically, then probably the best is to use np.typecodes:
>>> np.typecodes
{'All': '?bhilqpBHILQPefdgFDGSUVOMm', 'Complex': 'FDG', 'AllFloat': 'efdgFDG',
'Integer': 'bhilqp', 'UnsignedInteger': 'BHILQP', 'Float': 'efdg',
'Character': 'c', 'Datetime': 'Mm', 'AllInteger': 'bBhHiIlLqQpP'}
Each of those codes represents a different type:
>>> for t in np.typecodes['UnsignedInteger']:
... print(np.dtype(t).name)
...
uint8
uint16
uint32
uint32
uint64
uint64
Note that the return of e.g np.dtype(...) is the same that would be returned by a.dtype if a were an ndarray of that type.
Lastly, if you simply want to check whether it is an integer or a float, you can check the dtype's .kind attribute, which also return a single character:
>>> np.dtype(np.float32).kind
'f'
>>> np.dtype(np.int32).kind
'i'
>>> np.dtype(np.uint32).kind
'u'
>>> np.dtype(np.bool).kind
'b'
It is a little more complicated, so it may get confussing, but reading the relevant part of the docs is probably a good idea.