Edit: I've found what the problem boils down to:
If you run this code:
A = ones((10,4))
view = A[:,1]
view.fill(7)
A
or
A = ones((10,4))
view = A[:,1:3]
view.fill(7)
A
You'll see that the columns of A change
If you run this:
A = ones((10,4))
view = A[:,(1,2)]
view.fill(7)
A
There's no side effects on A. Is this behavior on purpose or a bug?
I have a function that calculates the amount I have to rotate certain columns of x,y points in a matrix. The function only takes one input - a matrix mat:
def rotate(mat):
In the function, I create views to make working with each section easier:
rot_mat = mat[:,(col,col+1)]
Then, I calculate a rotation angle and apply it back on the view that I had created before:
rot_mat[row,0] = cos(rot)*x - sin(rot)*y
rot_mat[row,1] = sin(rot)*x + cos(rot)*y
If I perform this in the main body of my program, the changes to my rot_mat view would propagate to the original matrix mat. When I turned it into a function, the views stopped having side effects on the original matrix. What's the reasoning for this and is there any way to get around it? I should also note that it isn't changing mat within the function itself. At the end, I just try to return mat but no changes have been made.
Full code for function:
def rotate(mat):
# Get a reference shape
ref_sh = 2*random.choice(range(len(filelist)))
print 'Reference shape is '
print (ref_sh/2)
# Create a copy of the reference point matrix
ref_mat = mat.take([ref_sh,ref_sh+1],axis=1)
# Calculate rotation for each set of points
for col in range(len(filelist)):
col = col * 2 # Account for the two point columns
rot_mat = mat[:,(col,col+1)]
# Numerator = sum of wi*yi - zi*xi
numer = inner(ref_mat[:,0],rot_mat[:,1]) - inner(ref_mat[:,1],rot_mat[:,0])
# Denominator = sum of wi*xi + zi*yi
denom = inner(ref_mat[:,0],rot_mat[:,0]) + inner(ref_mat[:,1],rot_mat[:,1])
rot = arctan(numer/denom)
# Rotate the points in rot_mat. As it's a view of mat, the effects are
# propagated.
for row in range(num_points):
x = rot_mat[row,0]
y = rot_mat[row,1]
rot_mat[row,0] = cos(rot)*x - sin(rot)*y
rot_mat[row,1] = sin(rot)*x + cos(rot)*y
return mat
When you do view = A[:,(1,2)] you are using advanced indexing (Numpy manual: Advanced Indexing), which means that the array returns a copy, not a view. It's advanced because your indexing object is a tuple "containing at least one sequence" (the sequence being the tuple (1,2)). The total explicit selection object obj in your case would equal (slice(None), (1,2)), i.e. A[(slice(None), (1,2))] returns the same thing as A[:,(1,2)].
As larsmans suggests above, it seems that __getitem__ and __setitem__ behave differently for advanced indexing, which makes sense, because assigning values to a copy would have no use (the copy would not be stored).
Related
I want to change a parameters in a list of elements QF and QD as following:
Lattice is a list includes these elements for example :
lattice = [QF, QD, QF, QD]
And the indexes of these elements in the list are given in quad_indexes.
QF and QD has two parameters (K and FamName):
QF.K = 1.7 and QD.K = -2.1
QF.FamName = 'QF'
QD.FamName = 'QD'
I want to gave the random value for the parameter K for each element individually
I tried:
i =0
Quad_strength_err =[]
while(i < len(quad_indexes)):
if(lattice[quad_indexes[i]].FamName == 'QF'):
lattice[quad_indexes[i]].K *= (1 + errorQF*random())
i += 1
elif(lattice[quad_indexes[i]].FamName == 'QD'):
lattice[quad_indexes[i]].K *= (1 + errorQD * random())
i += 1
for j in quad_indexes:
quad_strength_err = lattice[j].K
Quad_strength_err.append(quad_strength_err)
The problem is when i print (Quad_strength_err) i got fixed value for each of QF and QD, for example:
[1.8729820159805597, -2.27910323371567, 1.8729820159805597, -2.27910323371567]
I am looking foe a result for example:
[1.7729820159805597, -2.17910323371567, 1.8729820159805597, -2.27910323371567]
TL;DR
You need to make copies of QF and QD - you're making aliases.
The problem
The problem is almost certainly due to aliasing.
When you initialize lattice with this line:
lattice = [QF, QD, QF, QD]
what you are doing is creating a structure with two pointers to QF and two pointers to QD.
In your loop, you then modify QF twice, once via lattice[0] and again via lattice[2], and ditto for QD.
The solution
What you need to do is create copies, maybe shallow, maybe deep.
Try this option:
from copy import copy
lattice = [copy(QF), copy(QD), copy(QF), copy(QD)]
and if that still doesn't work, you might need a deepcopy:
from copy import deepcopy
lattice = [deepcopy(QF), deepcopy(QD), deepcopy(QF), deepcopy(QD)]
Or, a more compact version of the same code, just cause I like comprehensions:
from copy import deepcopy
lattice = [deepcopy(x) for x in (QF, QD, QF, QD)]
If your code is correct, you can't expect a different result for your first and third element of lattice (resp second and last), since it's (to simplify) the same element.
Using id, you can easily show that, lattice[0] and lattice[2] share the same id, therefore modifying lattice[0] will modify lattice[2].
You can duplicate each object QF and QD to git rid of this behavior
I tryed to build a working sample of your problem starting by building simple QF and QD classes :
from random import random
class QF():
def __init__(self):
self.K = 1.7
self.FamName = "QF"
class QD():
def __init__(self):
self.K = -2.1
self.FamName = "QD"
then I create the lattice with different instance of the classis by calling them with ()
lattice = [QF(), QD(), QF(), QD()]
I think your mistake comes from this step as QF refers to the class it self and QF creats a brand new instance that you can modify separatly from the other one. For example if you do QF.K = 3 and then a = QF(), a.K should return you 3.
finally I apply some randomness using the random imported previously :
for i in lattice:
if i.FamName == "QF":
i.K = (1 + errorQF*random())
elif i.FamName == "QD":
i.K = (1 + errorQD*random())
form which I get :
print(*[i.K for i in lattice])
>>>> 1.148989048860669 0.9324164812782919 1.0652187255939742 0.6860911849022292
I'm trying to translate this line of code from Python to MATLAB:
new_img[M[0, :] - corners[0][0], M[1, :] - corners[1][0], :] = img[T[0, :], T[1, :], :]
So, naturally, I wrote something like this:
new_img(M(1,:)-corners(2,1),M(2,:)-corners(2,2),:) = img(T(1,:),T(2,:),:);
But it gives me the following error when it reaches that line:
Requested 106275x106275x3 (252.4GB) array exceeds maximum array size
preference. Creation of arrays greater than this limit may take a long
time and cause MATLAB to become unresponsive. See array size limit or
preference panel for more information.
This has made me believe that it is not assigning things correctly. Img is at most a 1000 × 1500 RGB image. The same code works in less than 5 seconds in Python. How can I do vector assignment like the code in the first line in MATLAB?
By the way, I didn't paste all lines of my code for this post not to get too long. If I need to add anything else, please let me know.
Edit:
Here's an explanation of what I want my code to do (basically, this is what the Python code does):
Consider this line of code. It's not a real MATLAB code, I'm just trying to explain what I want to do:
A([2 3 5], [1 3 5]) = B([1 2 3], [2 4 6])
It is interpreted like this:
A(2,1) = B(1,2)
A(3,1) = B(2,2)
A(5,1) = B(3,2)
A(2,3) = B(1,4)
A(3,3) = B(2,4)
A(5,3) = B(3,4)
...
...
...
Instead, I want it to be interpreted like this:
A(2,1) = B(1,2)
A(3,3) = B(2,4)
A(5,5) = B(3,6)
When you do A[vector1, vector2] in Python, you index the set:
A[vector1[0], vector2[0]]
A[vector1[1], vector2[1]]
A[vector1[2], vector2[2]]
A[vector1[3], vector2[3]]
...
In MATLAB, the similar-looking A(vector1, vector2) instead indexes the set:
A(vector1(1), vector2(1))
A(vector1(1), vector2(2))
A(vector1(1), vector2(3))
A(vector1(1), vector2(4))
...
A(vector1(2), vector2(1))
A(vector1(2), vector2(2))
A(vector1(2), vector2(3))
A(vector1(2), vector2(4))
...
That is, you get each combination of indices. You should think of it as a sub-array composed of the rows and columns specified in the two vectors.
To accomplish the same as the Python code, you need to use linear indexing:
index = sub2ind(size(A), vector1, vector2);
A(index)
Thus, your MATLAB code should do:
index1 = sub2ind(size(new_img), M(1,:)-corners(2,1), M(2,:)-corners(2,2));
index2 = sub2ind(size(img), T(1,:), T(2,:));
% these indices are for first 2 dims only, need to index in 3rd dim also:
offset1 = size(new_img,1) * size(new_img,2);
offset2 = size(img,1) * size(img,2);
index1 = index1.' + offset1 * (0:size(new_img,3)-1);
index2 = index2.' + offset2 * (0:size(new_img,3)-1);
new_img(index1) = img(index2);
What the middle block does here is add linear indexes for the same elements along the 3rd dimension. If ii is the linear index to an element in the first channel, then ii + offset1 is an index to the same element in the second channel, and ii + 2*offset1 is an index to the same element in the third channel, etc. So here we're generating indices to all those matrix elements. The + operation is doing implicit singleton expansion (what they call "broadcasting" in Python). If you have an older version of MATLAB this will fail, you need to replace that A+B with bsxfun(#plus,A,B).
Given an initial 2-D array:
initial = [
[0.6711999773979187, 0.1949000060558319],
[-0.09300000220537186, 0.310699999332428],
[-0.03889999911189079, 0.2736999988555908],
[-0.6984000205993652, 0.6407999992370605],
[-0.43619999289512634, 0.5810999870300293],
[0.2825999855995178, 0.21310000121593475],
[0.5551999807357788, -0.18289999663829803],
[0.3447999954223633, 0.2071000039577484],
[-0.1995999962091446, -0.5139999985694885],
[-0.24400000274181366, 0.3154999911785126]]
The goal is to multiply some random values inside the array by a random percentage. Lets say only 3 random numbers get replaced by a random multipler, we should get something like this:
output = [
[0.6711999773979187, 0.52],
[-0.09300000220537186, 0.310699999332428],
[-0.03889999911189079, 0.2736999988555908],
[-0.6984000205993652, 0.6407999992370605],
[-0.43619999289512634, 0.5810999870300293],
[0.84, 0.21310000121593475],
[0.5551999807357788, -0.18289999663829803],
[0.3447999954223633, 0.2071000039577484],
[-0.1995999962091446, 0.21],
[-0.24400000274181366, 0.3154999911785126]]
I've tried doing this:
def mutate(array2d, num_changes):
for _ in range(num_changes):
row, col = initial.shape
rand_row = np.random.randint(row)
rand_col = np.random.randint(col)
cell_value = array2d[rand_row][rand_col]
array2d[rand_row][rand_col] = random.uniform(0, 1) * cell_value
return array2d
And that works for 2D arrays but there's chance that the same value is mutated more than once =(
And I don't think that's efficient and it only works on 2D array.
Is there a way to do such "mutation" for array of any shape and more efficiently?
There's no restriction of which value the "mutation" can choose from but the number of "mutation" should be kept strict to the user specified number.
One fairly simple way would be to work with a raveled view of the array. You can generate all your numbers at once that way, and make it easier to guarantee that you won't process the same index twice in one call:
def mutate(array_anyd, num_changes):
raveled = array_anyd.reshape(-1)
indices = np.random.choice(raveled.size, size=num_changes, replace=False)
values = np.random.uniform(0, 1, size=num_changes)
raveled[indices] *= values
I use array_anyd.reshape(-1) in favor of array_anyd.ravel() because according to the docs, the former is less likely to make an inadvertent copy.
The is of course still such a possibility. You can add an extra check to write back if you need to. A more efficient way would be to use np.unravel_index to avoid creating a view to begin with:
def mutate(array_anyd, num_changes):
indices = np.random.choice(array_anyd.size, size=num_changes, replace=False)
indices = np.unravel_indices(indices, array_anyd.shape)
values = np.random.uniform(0, 1, size=num_changes)
raveled[indices] *= values
There is no need to return anything because the modification is done in-place. Conventionally, such functions do not return anything. See for example list.sort vs sorted.
Using shuffle instead of random_choice, this would be a different solution. It works on an array of any shape.
def mutate(arrayIn, num_changes):
mult = np.zeros(arrayIn.ravel().shape[0])
mult[:num_changes] = np.random.uniform(0,1,num_changes)
np.random.shuffle(mult)
mult = mult.reshape(arrayIn.shape)
arrayIn = arrayIn + mult*arrayIn
return arrayIn
This question may be a little specialist, but hopefully someone might be able to help. I normally use IDL, but for developing a pipeline I'm looking to use python to improve running times.
My fits file handling setup is as follows:
import numpy as numpy
from astropy.io import fits
#Directory: /Users/UCL_Astronomy/Documents/UCL/PHASG199/M33_UVOT_sum/UVOTIMSUM/M33_sum_epoch1_um2_norm.img
with fits.open('...') as ima_norm_um2:
#Open UVOTIMSUM file once and close it after extracting the relevant values:
ima_norm_um2_hdr = ima_norm_um2[0].header
ima_norm_um2_data = ima_norm_um2[0].data
#Individual dimensions for number of x pixels and number of y pixels:
nxpix_um2_ext1 = ima_norm_um2_hdr['NAXIS1']
nypix_um2_ext1 = ima_norm_um2_hdr['NAXIS2']
#Compute the size of the images (you can also do this manually rather than calling these keywords from the header):
#Call the header and data from the UVOTIMSUM file with the relevant keyword extensions:
corrfact_um2_ext1 = numpy.zeros((ima_norm_um2_hdr['NAXIS2'], ima_norm_um2_hdr['NAXIS1']))
coincorr_um2_ext1 = numpy.zeros((ima_norm_um2_hdr['NAXIS2'], ima_norm_um2_hdr['NAXIS1']))
#Check that the dimensions are all the same:
print(corrfact_um2_ext1.shape)
print(coincorr_um2_ext1.shape)
print(ima_norm_um2_data.shape)
# Make a new image file to save the correction factors:
hdu_corrfact = fits.PrimaryHDU(corrfact_um2_ext1, header=ima_norm_um2_hdr)
fits.HDUList([hdu_corrfact]).writeto('.../M33_sum_epoch1_um2_corrfact.img')
# Make a new image file to save the corrected image to:
hdu_coincorr = fits.PrimaryHDU(coincorr_um2_ext1, header=ima_norm_um2_hdr)
fits.HDUList([hdu_coincorr]).writeto('.../M33_sum_epoch1_um2_coincorr.img')
I'm looking to then apply the following corrections:
# Define the variables from Poole et al. (2008) "Photometric calibration of the Swift ultraviolet/optical telescope":
alpha = 0.9842000
ft = 0.0110329
a1 = 0.0658568
a2 = -0.0907142
a3 = 0.0285951
a4 = 0.0308063
for i in range(nxpix_um2_ext1 - 1): #do begin
for j in range(nypix_um2_ext1 - 1): #do begin
if (numpy.less_equal(i, 4) | numpy.greater_equal(i, nxpix_um2_ext1-4) | numpy.less_equal(j, 4) | numpy.greater_equal(j, nxpix_um2_ext1-4)): #then begin
#UVM2
corrfact_um2_ext1[i,j] == 0
coincorr_um2_ext1[i,j] == 0
else:
xpixmin = i-4
xpixmax = i+4
ypixmin = j-4
ypixmax = j+4
#UVM2
ima_UVM2sum = total(ima_norm_um2[xpixmin:xpixmax,ypixmin:ypixmax])
xvec_UVM2 = ft*ima_UVM2sum
fxvec_UVM2 = 1 + (a1*xvec_UVM2) + (a2*xvec_UVM2*xvec_UVM2) + (a3*xvec_UVM2*xvec_UVM2*xvec_UVM2) + (a4*xvec_UVM2*xvec_UVM2*xvec_UVM2*xvec_UVM2)
Ctheory_UVM2 = - alog(1-(alpha*ima_UVM2sum*ft))/(alpha*ft)
corrfact_um2_ext1[i,j] = Ctheory_UVM2*(fxvec_UVM2/ima_UVM2sum)
coincorr_um2_ext1[i,j] = corrfact_um2_ext1[i,j]*ima_sk_um2[i,j]
The above snippet is where it is messing up, as I have a mixture of IDL syntax and python syntax. I'm just not sure how to convert certain aspects of IDL to python. For example, the ima_UVM2sum = total(ima_norm_um2[xpixmin:xpixmax,ypixmin:ypixmax]) I'm not quite sure how to handle.
I'm also missing the part where it will update the correction factor and coincidence correction image files, I would say. If anyone could have the patience to go over it with a fine tooth comb and suggest the neccessary changes I need that would be excellent.
The original normalised image can be downloaded here: Replace ... in above code with this file
One very important thing about numpy is that it does every mathematical or comparison function on an element-basis. So you probably don't need to loop through the arrays.
So maybe start where you convolve your image with a sum-filter. This can be done for 2D images by astropy.convolution.convolve or scipy.ndimage.filters.uniform_filter
I'm not sure what you want but I think you want a 9x9 sum-filter that would be realized by
from scipy.ndimage.filters import uniform_filter
ima_UVM2sum = uniform_filter(ima_norm_um2_data, size=9)
since you want to discard any pixel that are at the borders (4 pixel) you can simply slice them away:
ima_UVM2sum_valid = ima_UVM2sum[4:-4,4:-4]
This ignores the first and last 4 rows and the first and last 4 columns (last is realized by making the stop value negative)
now you want to calculate the corrections:
xvec_UVM2 = ft*ima_UVM2sum_valid
fxvec_UVM2 = 1 + (a1*xvec_UVM2) + (a2*xvec_UVM2**2) + (a3*xvec_UVM2**3) + (a4*xvec_UVM2**4)
Ctheory_UVM2 = - np.alog(1-(alpha*ima_UVM2sum_valid*ft))/(alpha*ft)
these are all arrays so you still do not need to loop.
But then you want to fill your two images. Be careful because the correction is smaller (we inored the first and last rows/columns) so you have to take the same region in the correction images:
corrfact_um2_ext1[4:-4,4:-4] = Ctheory_UVM2*(fxvec_UVM2/ima_UVM2sum_valid)
coincorr_um2_ext1[4:-4,4:-4] = corrfact_um2_ext1[4:-4,4:-4] *ima_sk_um2
still no loop just using numpys mathematical functions. This means it is much faster (MUCH FASTER!) and does the same.
Maybe I have forgotten some slicing and that would yield a Not broadcastable error if so please report back.
Just a note about your loop: Python's first axis is the second axis in FITS and the second axis is the first FITS axis. So if you need to loop over the axis bear that in mind so you don't end up with IndexErrors or unexpected results.
I'm trying to find out if there is a way to change the viewport angle in blender using Python.
I would like a result like you would get from pressing 1, 3, or 7 on the num. pad.
Thank you for any help
First of all, note that you can have multiple 3D views open at once, and each can have its own viewport angle, perspective/ortho settings etc. So your script will have to look for all the 3D views that might be present (which might be none) and decide which one(s) it’s going to affect.
Start with the bpy.data object, which has a window_managers attribute. This collection always seems to have just one element. However, there might be one or more open windows. Each window has a screen, which is divided into one or more areas. So you need to search through all the areas for one with a space type of "VIEW_3D". And then hunt through the spaces of this area for the one(s) with type "VIEW_3D". Such a space will be of subclass SpaceView3D. This will have a region_3d attribute of type RegionView3D. And finally, this object in turn has an attribute called view_matrix, which takes a value of type Matrix that you can get or set.
Got all that? :)
Once you've located the right 'view', you can modify:
view.spaces[0].region_3d.view_matrix
view.spaces[0].region_3d.view_rotation
Note that the region_3d.view_location is the 'look_at' target, not the location of the camera; you have to modify the view_matrix directly if you want to move the position of the camera (as far as I know), but you can subtly adjust the rotation using view_rotation quite easily. You'll probably need to read this to generate a valid quaternion though: http://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation
Perhaps something like this may be useful:
class Utils(object):
def __init__(self, context):
self.context = context
#property
def views(self):
""" Returns the set of 3D views.
"""
rtn = []
for a in self.context.window.screen.areas:
if a.type == 'VIEW_3D':
rtn.append(a)
return rtn
def camera(self, view):
""" Return position, rotation data about a given view for the first space attached to it """
look_at = view.spaces[0].region_3d.view_location
matrix = view.spaces[0].region_3d.view_matrix
camera_pos = self.camera_position(matrix)
rotation = view.spaces[0].region_3d.view_rotation
return look_at, camera_pos, rotation
def camera_position(self, matrix):
""" From 4x4 matrix, calculate camera location """
t = (matrix[0][3], matrix[1][3], matrix[2][3])
r = (
(matrix[0][0], matrix[0][1], matrix[0][2]),
(matrix[1][0], matrix[1][1], matrix[1][2]),
(matrix[2][0], matrix[2][1], matrix[2][2])
)
rp = (
(-r[0][0], -r[1][0], -r[2][0]),
(-r[0][1], -r[1][1], -r[2][1]),
(-r[0][2], -r[1][2], -r[2][2])
)
output = (
rp[0][0] * t[0] + rp[0][1] * t[1] + rp[0][2] * t[2],
rp[1][0] * t[0] + rp[1][1] * t[1] + rp[1][2] * t[2],
rp[2][0] * t[0] + rp[2][1] * t[1] + rp[2][2] * t[2],
)
return output