Can someone please explain?
import numpy
a = ([1,2,3,45])
b = ([6,7,8,9,10])
numpy.savetxt('test.txt',(a,b))
This script can save well the data. But when I am running it through a loop it can print all but cannot not save all. why?
import numpy
a = ([1,2,3,4,5])
b = ([6,7,8,9,10])
for i,j in zip(a,b):
print i,j
numpy.savetxt('test.txt',(i,j))
You overwrite the previous data each time you call numpy.savetext().
A solution, using a temporary buffer array :
import numpy
a = ([1,2,3,4,5])
b = ([6,7,8,9,10])
out = []
for i,j in zip(a,b):
print i,j
out.append( (i,j) )
numpy.savetxt('test.txt',out)
numpy.savetxt will overwrite the previously written file, so you only get the result of the last iteration.
The faster way will be to use open with
import numpy
a = ([1,2,3,4,5])
b = ([6,7,8,9,10])
with open('test.txt','wb') as ftext: #Wb if you want to create a new one,
for i,j in zip(a,b): #ab if you want to appen it. Her it's wb
print i,j
numpy.savetxt(ftext,(i,j))
It will be really faster with a large array
You should append (i,j) rather than overwriting previous ones
import numpy as np
a = np.array([1,2,3,4,5])
b = np.array([6,7,8,9,10])
np.savetxt('test.txt', np.column_stack((a, b)))
Related
There are 2 arrays A and B:
import numpy as np
A = np.array([1,2,3,4])
B = np.array([2,5,2,1,1,6])
If the element in A exists in B, output their index in B. The ideal output C is：
C = np.array([3,4,0,2])
While a bit ugly, this should work. You want to use np.where and np.concatenate. I'm using a placeholder list to store values and recombine, there may be a smoother method, but this should do the trick until the further reading of the docs may provide a better solution.
import numpy as np
A = np.array([1,2,3,4])
B = np.array([2,5,2,1,1,6])
preC= []
for i in A:
if len(np.where(B == i)[0]) > 0:
preC.append(np.where(B == i)[0])
C = np.concatenate(preC)
print(C)
I have a strong background in Matlab, and I am trying to switch to python. I am trying to write a nested for loop with numpy array and storing output values.
My code reads like:
import numpy as np
import math
# T parameter
kk = np.arange(0, 20, 0.1)
print(len(kk))
# V parameter
pp = np.arange(1, 5, 1)
print(len(pp))
a = len(kk)
b = len(pp)
P = np.zeros((a,b))
for T in kk:
print(T)
for V in pp:
print(V)
P = math.exp(-T*V/10)
print(P)
Explanation/Question
kk, pp are the vectors. In for loop(s) correct values of T and V parameters are being called. However, values of P are not being stored.
I tried the following change P[T][V] = math.exp(-T*V/10), I get the following error: IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices
Any help will be appreciated. Thank you in advance.
In this code you define P as a 2d array. But the loop you assign the scalar result of the math.exp expression to that variable. That replaces the original P value, and also replaces the value calculated in the previous loop. This kind of loop doesn't work in MATLAB does it? Don't you have to assign the scalar value to some 'slot' in P?
P = np.zeros((a,b))
for T in kk:
print(T)
for V in pp:
print(V)
P = math.exp(-T*V/10)
A better way:
In [301]: kk = np.arange(0,20,0.1)
In [302]: kk.shape
Out[302]: (200,)
In [303]: pp = np.arange(1, 5,1)
In [304]: pp.shape
Out[304]: (4,)
In numpy we prefer to use fast whole-array methods. Here I use broadcasting to perform an outer like calculation of kk with pp.
In [305]: P = np.exp(-kk[:,None]*pp/10)
In [306]: P.shape
Out[306]: (200, 4)
(I believe MATLAB added broadcasting in recent years; numpy has had it from the beginning.)
Comparing this with the iterative version:
In [309]: P1 = np.zeros((200,4))
...: for i in range(0,len(kk)):
...: for j in range(0,len(pp)):
...: T = kk[i]
...: V = pp[j]
...: P1[i,j] = math.exp(-T*V/10)
...:
In [310]: P1.shape
Out[310]: (200, 4)
In [311]: np.allclose(P,P1)
Out[311]: True
A cleaner way of writing indexed iteration in Python is with enumerate:
In [312]: P1 = np.zeros((200,4))
...: for i,T in enumerate(kk):
...: for j,V in enumerate(pp):
...: P1[i,j] = math.exp(-T*V/10)
Based on the line where you mentioned trying P[T][V] = math.exp(-T*V/10), you might also be interested in this option:
import numpy as np
import math
# T parameter
kk = np.arange(0, 20, 0.1)
print(len(kk))
# V parameter
pp = np.arange(1, 5, 1)
print(len(pp))
a = len(kk)
b = len(pp)
P = np.zeros((a,b))
for i in range(0,len(kk)):
for j in range(0,len(pp)):
T = kk[i]
V = pp[j]
P[i][j] = math.exp(-T*V/10)
# you can also simply do this:
#P[i][j] = math.exp(-kk[i]*pp[j]/10)
Although it's straightforward, it's not particularly clean. Since you mentioned that you're switching to python, I'd take a look at hpaulj's answer for a more thorough explanation and as well as a nice alternative to iterating through arrays.
You can make a dictionary if you want to see the keys and values per your comment. This might make more sense actually. I would recommend against a plethora of dynamically created variables, as with a dictionary, you can call the entire dictionary OR specific values, which you could store as variables later anyway. Obviously, it depends on the scope of your project and what solution makes sense, but you could also turn the dictionary into a pandas dataframe with pd.DataFrame() for analysis, so it gives you flexibility. You said you are new to python, so you might want to check out pandas if you haven't heard of it, but you probably have as it is one of or the most popular library.
import numpy as np
import math
P_dict = {}
# T parameter
kk = np.arange(0, 20, 0.1)
# print(len(kk))
# V parameter
pp = np.arange(1, 5, 1)
# print(len(pp))
a = len(kk)
b = len(pp)
P = np.zeros((a,b))
for T in kk:
# print(T)
for V in pp:
# print(V)
P = math.exp(-T*V/10)
key = f'{T},{V}'
value = P
P_dict[key] = value
print(P_dict)
This is how you would call a value in the dict based on the key.
P_dict['19.900000000000002,3']
You can also edit this line of code to whatever format you want: key = f'{T},{V}' and call the key acording to the format as I have done in my example.
Output:
0.002554241418992996
Either way, a list or a dict prints some interesting python abstract art!
I would like to apply a function to each of the 3x3 matrices in my (6890,6890,3,3) numpy array. Until now, I have tried using vectorization on a smaller example and with a simpler function which didn't work out.
def myfunc(x):
return np.linalg.norm(x)
m = np.arange(45).reshape(5,3,3)
t = m.shape[0]
r = np.zeros((t, t))
q = m[:,None,...] # m.swapaxes(1,2) # m[i] # m[j].T
f = np.vectorize(q, otypes=[np.float])
res = myfunc(f)
Is vectorization even the right approach to solve this problem efficiently or should I try something else? I've also looked into numpy.apply_along_axis but this only applies to 1D-subarrays.
You need loop over each element and apply function:
import numpy as np
# setup function
def myfunc(x):
return np.linalg.norm(x*2)
# setup data array
data = np.arange(45).reshape(5, 3, 3)
# loop over elements and update
for item in np.nditer(data, op_flags = ['readwrite']):
item[...] = myfunc(item)
If you need apply function for entire 3x3 array then use:
out_data = []
for item in data:
out_data.append(myfunc(item))
Output:
[14.2828568570857, 39.761790704142086, 66.4529909033446, 93.32202312423365, 120.24974012445931]
I have two numpy arrays.I want to shuffle only their row index simultaneously. Although I equalize their row index then when i shuffle a why it doesn't automatically shuffled b
a = np.arange(100).reshape(10,10)
b = np.arange(10).reshape(10,1)
a.shape[0]==b.shape[0]
np.random.shuffle(a)
print a
print b
It makes no sense, what you want to do can be down like below:
a = np.arange(100).reshape(10,10)
b = np.arange(10).reshape(10,1)
p = np.random.permutation(a.shape[0])
a = a[p]
b = b[p]
print a
print b
You can't marry two numpy arrays that way, to force them to stay in the same order. What you can do is save the order in a separate array and then sort both of them by it.
a = np.arange(100).reshape(10,10)
b = np.arange(10).reshape(10,1)
i = np.random.shuffle(np.arange(a.size[0]))
print a[i]
print b[i]
Use numpy.random.permutation:
import numpy as np
a = np.arange(100).reshape(10,10)
b = np.arange(10).reshape(10,1)
perm = np.random.permutation(a.shape[0])
print(a[perm, :])
print(b[perm, :])
While numpy.random.shuffle sorts the array in-place, in the last two lines of my code there is only a so called view of the arrays a and b created. If you check a and b afterwards, they are still the same. So if you want to use the shuffled version you should use something like a = a[perm, :] or c = (a[perm, :]).copy().
I have 3000000 ints' long array which I want to output to a file. How can I do that?
Also, is this
for i in range(1000):
for k in range(1000):
(r, g, b) = rgb_im.getpixel((i, k))
rr.append(r)
gg.append(g)
bb.append(b)
d.extend(rr)
d.extend(gg)
d.extend(bb)
a good practice to join array together?
All of the arrays are declared like this d = array('B')
EDIT:
Managed to output all int`s delimited by ' ' with this
from PIL import Image
import array
side = 500
for j in range(1000):
im = Image.open(r'C:\Users\Ivars\Desktop\RS\Shape\%02d.jpg' % (j))
rgb_im = im.convert('RGB')
d = array.array('B')
rr = array.array('B')
gg = array.array('B')
bb = array.array('B')
f = open(r'C:\Users\Ivars\Desktop\RS\ShapeData\%02d.txt' % (j), 'w')
for i in range(side):
for k in range(side):
(r, g, b) = rgb_im.getpixel((i, k))
rr.append(r)
gg.append(g)
bb.append(b)
d.extend(rr)
d.extend(gg)
d.extend(bb)
o = ' '.join(str(t) for t in d)
print('#', j, ' - ', len(o))
f.write(o)
f.close()
if you're using python >= 2.6 then you can use print functions from the future!
from __future__ import print_function
#your code
# This will print out a string representation of list to the file.
# If you need it formatted differently, then you'll have to construct the string yourself
print(d, file=open('/path/to/file.txt','w')
#you can join the list items with an empty string to get only the numbers
print("".join(d),file=('/path/to/file.txt','w'))
This has the side effect of turning print from a statement into a function, so you'll have to wrap whatever you want printed in ()
You want tofile(), which requires you to open a file object. See https://docs.python.org/2/library/array.html and https://docs.python.org/2/library/stdtypes.html#bltin-file-objects. Also, have you considered using NumPy?
import array
a = array.array('B')
b = array.array('B')
a.append(3)
a.append(4)
print a
print b
with open('c:/test.dat', 'w') as f:
a.tofile(f)
with open('c:/test.dat', 'r') as f:
b.fromfile(f, 2)
print b
EDIT: Based on your edit, you can use numpy with PIL and generate the array in a line or two, without looping. See, e.g., Conversion between Pillow Image object and numpy array changes dimension for example code.