I have a nested dictionary that looks like this:
dictionary = {time: {pixels: {intensity}}}
len(time) = 65
len(pixels) = 6/time
len(intensity) = 6/pixel
Just to be clear, for 1 time value --> [1,2,3,4,5,6] pixel values --> sub values for each of the pixel values are 6 intensity values.
Example:
dictionary = {time1 : {1: array([i11,i12,i13,i14,i15,i16]), 2: array([i21,i22,i23,i24,i25,i26]), 3: array([i31,i32,i33,i34,i35,i36]), 4: array([i41,i42,i43,i44,i45]), 5: array([i51,i52,i53,i54,i55,i56]), 6: array([i61,i62,i63,i64,i65,i66])}}
My question is,
how do I plot these values (3D plot) with time in the z axis and intensity values and pixel values (since both are length 6) on y and x values respectively?
The following is what I have tried so far and have been unsuccessful:
x = []
y = []
z = []
for i in dictionary:
z1 = i
z.append(z1)
x1 = dictionary[i].keys()
x.append(x1)
y1 = dictionary[i].values()
y.append(y1)
from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure()
ax = Axes3D(fig)
ax.plot(x, y, zs = 0, zdir='z', label='zs=0,zdir=z')
Your y is a list of lists. It is easy to see the mistake, if you use listed for loops.
Corrected code:
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
x, y, z = [], [], []
for tim, pixels in dictionary.items():
for pixel, intensities in pixels.items():
for intensity in intensities:
x.append(intensity)
y.append(pixel)
z.append(tim)
fig = plt.figure()
ax = Axes3D(fig)
ax.plot(x, y, z, zdir='z')
ax.show()
Example use:
Using the simple dataset:
{1: {1: array([11, 12, 13, 14, 15, 16]), 2: array([21, 22, 23, 24, 25, 26]),
3: array([31, 32, 33, 34, 35, 36]), 4: array([41, 42, 43, 44, 45]),
5: array([51, 52, 53, 54, 55, 56]), 6: array([61, 62, 63, 64, 65, 66])},
2: {1: array([71, 72, 73, 74, 75, 76]), 2: array([21, 22, 23, 24, 25, 26]),
3: array([31, 32, 33, 34, 35, 36]), 4: array([41, 42, 43, 44, 45]),
5: array([51, 52, 53, 54, 55, 56]), 6: array([61, 62, 63, 64, 65, 66])}}
That would result in:
Related
I am triying to solve the following problem in a more numpy-friendly way (without loops):
G is NxM matrix fill with 0, 1 or 2
D is a 3xNxM matrix
We want the a NxM matrix (R) with R[i,j] = D[k,i,j] being k=g[i,j]
A loop base solution is:
def getVals(g, d):
arr=np.zeros(g.shape)
for row in range(g.shape[0]):
for column in range(g.shape[1]):
arr[row,column]=d[g[row,column],row,column]
return arr
Try with ogrid and advanced indexing:
x,y = np.ogrid[:N,:M]
out = D[G, x[None], y[None]]
Test:
N,M=4,5
G = np.random.randint(0,3, (N,M))
D = np.random.rand(3,N,M)
np.allclose(getVals(G,D), D[G, x[None], y[None]])
# True
You could also use np.take_along_axis
Then you can simply extract your values along one specific axis:
# Example input data:
G = np.random.randint(0,3,(4,5)) # 4x5 array
D = np.random.randint(0,9,(3,4,5)) # 3x4x5 array
# Get the results:
R = np.take_along_axis(D,G[None,:],axis=0)
Since G should have the same number of dimension as D, we simply add a new dimension to G with G[None,:].
Here's my try (I assume g and d are Numpy Ndarrays):
def getVals(g, d):
m,n = g.shape
indexes = g.flatten()*m*n + np.arange(m*n)
arr = d.flatten()[indexes].reshape(m,n)
return arr
So if
d = [[[96, 89, 51, 40, 51],
[31, 72, 39, 77, 33]],
[[34, 11, 54, 86, 73],
[12, 21, 74, 39, 14]],
[[14, 91, 38, 77, 97],
[44, 55, 93, 88, 55]]]
and
g = [[2, 1, 2, 1, 1],
[0, 2, 0, 0, 2]]
then you are going to get
arr = [[14, 11, 38, 86, 73],
[31, 55, 39, 77, 55]]
I have two sets of data here:
data_feb = ['1st February', 45, 68, 70, 61, 54, 80, 72, 69, 73, 72, 58, 72, 64, 45, 42]
data_aug = ['1st August', 19, 27, 41, 42, 9, 14, 29, 34, 25, 29, 44, 43, 6, 17]
I loop over it to create another list here:
for i in data_feb:
#
if type(i) == int:
feb_numbers.append(i)
for i in data_aug:
if type(i) == int:
aug_numbers.append(i)
But here i have an algorithm to sort them:
feb_zero_to_ten = []
feb_ten_to_twenty = []
feb_twenty_to_thirty = []
feb_thirty_to_forty = []
feb_forty_to_fifty = []
feb_fifty_to_sixty = []
feb_sixty_to_seventy = []
feb_seventy_to_eighty = []
feb_eighty_to_ninety = []
feb_ninety_to_hundred = []
aug_zero_to_ten = []
aug_ten_to_twenty = []
aug_twenty_to_thirty = []
aug_thirty_to_forty = []
aug_forty_to_fifty = []
aug_fifty_to_sixty = []
aug_sixty_to_seventy = []
aug_seventy_to_eighty = []
aug_eighty_to_ninety = []
aug_ninety_to_hundred = []
# for loop to iterate over months numbers, sorting them into their correct columns by the 'tens' digit
for i, j in zip(feb_numbers, aug_numbers):
if 0 <= i < 10 and 0 <= j < 10:
feb_zero_to_ten.append(i)
aug_zero_to_ten.append(j)
elif 10 <= i < 20 and 10 <= j < 20:
feb_ten_to_twenty.append(i)
aug_ten_to_twenty.append(j)
elif 20 <= i < 30 and 20 <= j < 30:
feb_twenty_to_thirty.append(i)
aug_twenty_to_thirty.append(j)
elif 30 <= i < 40 and 30 <= j < 40:
feb_thirty_to_forty.append(i)
aug_thirty_to_forty.append(j)
elif 40 <= i < 50 and 40 <= j < 50:
feb_forty_to_fifty.append(i)
aug_forty_to_fifty.append(j)
elif 50 <= i < 60 and 50 <= j < 60:
feb_fifty_to_sixty.append(i)
aug_fifty_to_sixty.append(j)
elif 60 <= i < 70 and 60 <= j < 70:
feb_sixty_to_seventy.append(i)
aug_sixty_to_seventy.append(j)
elif 70 <= i < 80 and 70 <= j < 80:
feb_seventy_to_eighty.append(i)
aug_seventy_to_eighty.append(j)
elif 80 <= i < 90 and 80 <= j < 90:
feb_eighty_to_ninety.append(i)
aug_eighty_to_ninety.append(j)
elif 90 <= i < 100 and 90 <= j < 100:
feb_ninety_to_hundred.append(i)
aug_ninety_to_hundred.append(j)
This approach using zip() is not working. I am wondering if using this approach is not worth it, also I am trying to make this code as efficient as possible so any pointers would be very helpful. Thank you.
data_feb = ['1st February', 45, 68, 70, 61, 54, 80, 72, 69, 73, 72, 58, 72, 64, 45, 42]
data_aug = ['1st August', 19, 27, 41, 42, 9, 14, 29, 34, 25, 29, 44, 43, 6, 17]
feb_numbers=[i for i in data_feb if isinstance(i,int) ]
aug_numbers=[i for i in data_aug if isinstance(i,int) ]
from itertools import groupby
[list(g) for k,g in groupby(sorted(aug_numbers),key=lambda x :x//10)]
Output:
[[6, 9], [14, 17, 19], [25, 27, 29, 29], [34], [41, 42, 43, 44]]
You can use itertools groupby to group those numbers
Your approach is flawed here. you test that both i and j are BOTH in a range but if you look at your numbers you will see that might not happen, thing like (45, 19) do not fit in any of the ifs. If you look at the logic you are trying to achieve then you will notice that you actually want to separate you number by their leading digit (the tens), An easy approach is to make buckets and fill them like this:
feb_buckets = [[] for item in range(10)] # this makes a list of 10 buckets (lists)
aug_buckets = [[] for item in range(10)]
for feb, aug in zip(feb_numbers,aug_numbers):
feb_bucket[feb//10].append(feb) # // is integer division (which rounds down)
aug_bucket[aug//10].append(aug)
once you understand the logic you can then simplify the code even further by taking #ajay approach and using itertools.groupby
Don't create all list by hand, just create list of lists and then access them by index.
For numbers in range 0 to 10 use feb[0] to 10 to 20 feb[1] etc.
If you don't know if the lists will have the same length, use function find_in_range for each list.
You can use this code below for this:
data_feb = ['1st February', 45, 68, 70, 61, 54, 80, 72, 69, 73, 72, 58, 72, 64, 45, 42]
data_aug = ['1st August', 19, 27, 41, 42, 9, 14, 29, 34, 25, 29, 44, 43, 6, 17]
#Dont create all list by hand, just create list of lists and then acces them by index
feb = [[] for i in range(10)]
aug = [[] for i in range(10)]
def find_in_range(in_list, out_list):
for x in sorted(in_list[1:]): #exclude the first index, because it is a string
for i in range(10):
if i*10 < x < (i+1)*10:
out_list[i].append(x)
find_in_range(data_feb, feb)
find_in_range(data_aug, aug)
print("Feb: ", feb)
print("Aug", aug)
This is the output:
Feb: [[], [], [], [], [42, 45, 45], [54, 58], [61, 64, 68, 69], [72, 72, 72, 73], [], []]
Aug [[6, 9], [14, 17, 19], [25, 27, 29, 29], [34], [41, 42, 43, 44], [], [], [], [], []]
As you can see the first four arrays of feb list are empty because in the list data_feb were not any numbers between 0-40.
One good way to accomplish the sort is by having a sorted list of lists. The final output would be fed_sorted and aug_sorted. The i-th list would be in range of [i*10, (i+1)*10).
data_feb = ['1st February', 45, 68, 70, 61, 54, 80, 72, 69, 73, 72, 58, 72, 64, 45, 42]
data_aug = ['1st August', 19, 27, 41, 42, 9, 14, 29, 34, 25, 29, 44, 43, 6, 17]
feb_numbers = [x for x in data_feb if type(x) == int]
aug_numbers = [x for x in data_aug if type(x) == int]
GROUP_SIZE = 10 # 0-9, 10-19, 20-29....
feb_sorted = [[x for x in feb_numbers if x in range(i * 10, (i + 1) * 10)] for i in range(GROUP_SIZE)]
aug_sorted = [[x for x in aug_numbers if x in range(i * 10, (i + 1) * 10)] for i in range(GROUP_SIZE)]
print(feb_sorted)
print(aug_sorted)
The first part of your code can be made more effecient like so:
data_feb = ['1st February', 45, 68, 70, 61, 54, 80, 72, 69, 73, 72, 58, 72, 64, 45, 42]
data_aug = ['1st August', 19, 27, 41, 42, 9, 14, 29, 34, 25, 29, 44, 43, 6, 17]
data_feb = [x for x in data_feb if x == int]
data_aug = [x for x in data_aug if x == int]
For the second part it is unclear what you are trying to achieve. Can you try to give some more background? Why do you need this many lists? What do you mean by sorting them and why do you need them sorted in this way?
I have a numpy array of size 8x8.
Here is the numpy array:
QuantTable = np.array([[16, 11 ,10, 16, 24, 40, 51, 61],
[12, 12, 14, 19, 26, 58, 60, 55],
[14, 13, 16, 24, 40, 57, 69, 56],
[14, 17, 22, 29, 51, 87, 80, 62],
[18, 22, 37, 29, 51, 87, 80, 62],
[24, 35, 55, 64, 81, 109, 103, 77],
[49, 64, 78, 87, 103, 121, 120, 101],
[72, 92, 95, 98, 112, 100, 103, 99]])
I would like to perform the operations on the elements in the array.
I have created a function that accepts a scaling factor value and a Numpy Array.
Here it is:
def quantizationTable(Qval, QuantTable):
if Qval < 50:
scalingFactor = 5000/Qval
for x in range(QuantTable):
for y in range(QuantTable):
QuantTable[x][y] = ((scalingFactor * QuantTable[x][y] + 50/100)
if QuantTable[x][y] == 0:
QuantTable[x][y] = 1
return QuantTable
else:
scalingFactor = 200 - 2(Qval)
for x in range(QuantTable):
for y in range(QuantTable):
QuantTable[x][y] = ((scalingFactor * QuantTable[x][y] + 50/100)
if QuantTable[x][y] == 0:
QuantTable[x][y] = 1
return QuantTable
I am having trouble iterating over the numpy array and performing my operations. I am trying to apply the formula
((Scaling factor value * element of numpy array + 50)/100) to every element of the numpy array and return the modified array.
Can someone please help?
Just remove the loops, and the indexing. Numpy automatically broadcasts those operations. Also, a lot of your code can be taken out of the if...else statements.
def quantizationTable(Qval, QuantTable):
QuantTable = np.asarray(QuantTable, dtype=np.float32)
if int(Qval) < 50:
scalingFactor = 5000 / Qval
else:
scalingFactor = 200 - 2 * Qval # confirm that this is what you want?
QuantTable *= scalingFactor + 0.5
QuantTable[QuantTable == 0] = 1
return QuantTable
I have three arrays of equal length x, y, and z. The x and y arrays are the x-axis and y-axis for the grid. The z array will determine the color of the the grid block. For example,
x = [10, 10, 10, 20, 20, 20, 30, 30, 30]
y = [10, 20, 30, 10, 20, 30, 10, 20, 30]
z = [99, 54, 32, 67, 71, 88, 100, 15, 29]
It is easy to make 3D plots out of this like
ax.plot_trisurf(x, y, z, cmap=cm.RdYlGn)
or
ax.bar3d(x, y, [0] * len(x), 100, 100, z, cmap=cm.RdYlGn)
But I am looking for something like this grid
Another problem is that the way my z array is generated, it isn't in order by index. So my x, y, and z arrays could look like this.
x = [30, 10, 20, 20, 30, 10, 10, 30, 20]
y = [10, 20, 30, 10, 30, 30, 10, 20, 20]
z = [100, 54, 88, 67, 29, 32, 99, 15, 71]
Here is a small example for your specific problem. I'm converting your x and y indices to positions in an array looking at your data -- you might need to change this yourself.
import numpy as np
import matplotlib.pyplot as plt
x = [10, 10, 10, 20, 20, 20, 30, 30, 30]
y = [10, 20, 30, 10, 20, 30, 10, 20, 30]
z = [99, 54, 32, 67, 71, 88, 100, 15, 29]
# Convert x/y to indices. This only works if you have a rigid grid (which seems to be the case, but you might have to change the transform for your case)
x = (np.array(x)/10 - 1).astype(int)
y = (np.array(y)/10 - 1).astype(int)
# Create the image. Default color is black
z_im = np.zeros((x.max() + 1, y.max() + 1, 3))
# Go through z and color acoordingly -- only gray right now
for i, v in enumerate(z):
z_im[x[i], y[i]] = (v, v, v)
fig, ax = plt.subplots()
ax.imshow(z_im)
plt.show()
I have three arrays of equal length x, y, and z. The x and y arrays are the x-axis and y-axis for the grid. The z array will determine the color of the the grid block. For example,
x = [10, 10, 10, 20, 20, 20, 30, 30, 30]
y = [10, 20, 30, 10, 20, 30, 10, 20, 30]
z = [100, 54, 32, 67, 71, 88, 100, 15, 29]
It is easy to make 3D plots out of this like
ax.plot_trisurf(x, y, z, cmap=cm.RdYlGn)
or
ax.bar3d(x, y, [0] * len(x), 100, 100, z, cmap=cm.RdYlGn)
But I am looking for something like
this
np.meshgrid returns a tuple of two 2D arrays, which you can unpack directly
X,Y = np.meshgrid(x,y)
However, you don't need to those for an imshow plot. What you need and what you lack in your code is the 2D array of z values. This would be the array to provide to imshow.
img = plt.imshow(Z)
If you want to use meshgrid instead, you can use your X and Y values,
plt.pcolormesh(X,Y,Z)
Seeing the example data, you can use imshow:
x = [10, 10, 10, 20, 20, 20, 30, 30, 30]
y = [10, 20, 30, 10, 20, 30, 10, 20, 30]
z = [100, 54, 32, 67, 71, 88, 100, 15, 29]
import matplotlib.pyplot as plt
import numpy as np
z = np.array(z).reshape(3,3)
plt.imshow(z,extent=[5,35,5,35])
plt.show()