Develop Reference

Multiple lists in a for loop, numpy shape command requiring list entry?

I created the following for loop
for ItemTemplate in ItemTemplates:
x = 0
self.needle_images.append(cv.imread(ItemTemplate, cv.IMREAD_UNCHANGED))
self.needle_widths.append(self.needle_images[x].shape[1])
self.needle_heights.append(self.needle_images[x].shape[0])
x = x+1
I originally tried to write the for loop like this:
for ItemTemplate in ItemTemplates:
self.needle_images.append(cv.imread(ItemTemplate, cv.IMREAD_UNCHANGED))
self.needle_widths.append(self.needle_images.shape[1])
self.needle_heights.append(self.needle_images.shape[0])
I was assuming I didn't need to add the list entries to this code and perhaps there is a better way to do this but my Python skills are very young. The top example is fine and my code runs ok with it, but I am looking to see if there was a better way to accomplish this task.

I don't know why you need to store the image, width and height in three lists since image already has the width and height as its shape member, which you are already using. I also wonder why your top example works, since x will always stay 0 and this way all the upcoming Images will have the size of the first Image. The second example clearly doesn't work since you try to take the shape member of an array.
Changes I would make
if you need the index (x in your case) I would use the enumerate() function to get it:
for x, ItemTemplate in enumerate(ItemTemplates):
# x : index starting from 0
...
you could also first get the image into a local variable and then add it:
for ItemTemplate in ItemTemplates:
img = cv.imread(ItemTemplate, cv.IMREAD_UNCHANGED)
self.needle_images.append(img)
self.needle_widths.append(img.shape[0]) # if needing to store widths in own array
...
if you don't want to get the width and height from the shape member you could store the Image in an own class or dict, in two separate members width and height though this wouldn't make a big difference from just using the shape.
So if the for loop doesn't get more functionality, I would write:
for ItemTemplate in ItemTemplates:
self.needle_images.append(cv.imread(ItemTemplate, cv.IMREAD_UNCHANGED))
and use the shape member for the size since you can always store the shape value in a local variable when using it later:
for img in self.needle_images:
width, height, channels = img.shape
display.blit(img, (width, height)) # pseudo operation
...

The last two lines of the loop are always using the last image appended to needle_images. The index of the last item in a list is -1.
for ItemTemplate in ItemTemplates:
self.needle_images.append(cv.imread(ItemTemplate, cv.IMREAD_UNCHANGED))
self.needle_widths.append(self.needle_images[-1].shape[1])
self.needle_heights.append(self.needle_images[-1].shape[0])
Or just bite the bullet and assign the image to a name at the top of the loop.
for ItemTemplate in ItemTemplates:
temp = cv.imread(ItemTemplate, cv.IMREAD_UNCHANGED)
self.needle_images.append(temp)
self.needle_widths.append(temp.shape[1])
self.needle_heights.append(temp.shape[0])
Or even...
for ItemTemplate in ItemTemplates:
temp = cv.imread(ItemTemplate, cv.IMREAD_UNCHANGED)
w,h = temp.shape
self.needle_images.append(temp)
self.needle_widths.append(w)
self.needle_heights.append(h)

Can morphology.remove_small_objects remove big objects?

I was wondering if morphology.remove_small_objects could be used to remove big objects. I am using this tool to detect the objects as seen in the figure.
,
However,there are big objects as seen in the left. Is there any way I could use morphology.remove_small_objects as a threshold, for example:
mask=morphology.remove_small_objects(maske, 30)
Could I use like a range? between 30 and 200 so I can ignore the red detection in the image.
Otherwise, I will just count the white pixels in the image and remove the ones that have the highest.

This might be a good contribution to the scikit-image library itself, but for now, you need to roll your own. As suggested by Christoph, you can subtract the result of remove_small_objects from the original image to remove large objects. So, something like:
def filter_objects_by_size(label_image, min_size=0, max_size=None):
small_removed = remove_small_objects(label_image, min_size)
if max_size is not None:
mid_removed = remove_small_objects(small_removed, max_size)
large_removed = small_removed - mid_removed
return large_removed
else:
return small_removed

Converting pixels into wavelength using 2 FITS files

I am new to python and FITS image files, as such I am running into issues. I have two FITS files; the first FITS file is pixels/counts and the second FITS file (calibration file) is pixels/wavelength. I need to convert pixels/counts into wavelength/counts. Once this is done, I need to output wavelength/counts as a new FITS file for further analysis. So far I have managed to array the required data as shown in the code below.
import numpy as np
from astropy.io import fits
# read the images
image_file = ("run_1.fits")
image_calibration = ("cali_1.fits")
hdr = fits.getheader(image_file)
hdr_c = fits.getheader(image_calibration)
# print headers
sp = fits.open(image_file)
print('\n\nHeader of the spectrum :\n\n', sp[0].header, '\n\n')
sp_c = fits.open(image_calibration)
print('\n\nHeader of the spectrum :\n\n', sp_c[0].header, '\n\n')
# generation of arrays with the wavelengths and counts
count = np.array(sp[0].data)
wave = np.array(sp_c[0].data)
I do not understand how to save two separate arrays into one FITS file. I tried an alternative approach by creating list as shown in this code
file_list = fits.open(image_file)
calibration_list = fits.open(image_calibration)
image_data = file_list[0].data
calibration_data = calibration_list[0].data
# make a list to hold images
img_list = []
img_list.append(image_data)
img_list.append(calibration_data)
# list to numpy array
img_array = np.array(img_list)
# save the array as fits - image cube
fits.writeto('mycube.fits', img_array)
However I could only save as a cube, which is not correct because I just need wavelength and counts data. Also, I lost all the headers in the newly created FITS file. To say I am lost is an understatement! Could someone point me in the right direction please? Thank you.
I am still working on this problem. I have now managed (I think) to produce a FITS file containing the wavelength and counts using this website:
https://www.mubdirahman.com/assets/lecture-3---numerical-manipulation-ii.pdf
This is my code:
# Making a Primary HDU (required):
primaryhdu = fits.PrimaryHDU(flux) # Makes a header # or if you have a header that you’ve created: primaryhdu = fits.PrimaryHDU(arr1, header=head1)
# If you have additional extensions:
secondhdu = fits.ImageHDU(wave)
# Making a new HDU List:
hdulist1 = fits.HDUList([primaryhdu, secondhdu])
# Writing the file:
hdulist1.writeto("filename.fits", overwrite=True)
image = ("filename.fits")
hdr = fits.open(image)
image_data = hdr[0].data
wave_data = hdr[1].data
I am sure this is not the correct format for wavelength/counts. I need both wavelength and counts to be contained in hdr[0].data

If you are working with spectral data, it might be useful to look into specutils which is designed for common tasks associated with reading/writing/manipulating spectra.
It's common to store spectral data in FITS files using tables, rather than images. For example you can create a table containing wavelength, flux, and counts columns, and include the associated units in the column metadata.
The docs include an example on how to create a generic "FITS table" writer with wavelength and flux columns. You could start from this example and modify it to suit your exact needs (which can vary quite a bit from case to case, which is probably why a "generic" FITS writer is not built-in).
You might also be able to use the fits-wcs1d format.
If you prefer not to use specutils, that example still might be useful as it demonstrates how to create an Astropy Table from your data and output it to a well-formatted FITS file.

Slice image with Wand

I want to extract multiple parts of an image with Wand.
I've just found a function for cropping (in-place) the image img.crop(left, top, right, bottom) but note the slicing one as they say in the doc.
Note
If you want to crop the image but not in-place, use slicing
operator.

Take a look at the test_slice_crop method in the test directory for examples.
with Image(filename='source.jpg') as img:
with img[100:200, 100:200] as cropped:
# The `cropped' is an instance if wand.image.Image,
# and can be manipulated independently of `img' instance.
pass
Edit
For completion, slice is a built-in function in python to represent a set of iterations (i.e. a[start:stop:step]). In wand, this is used to allow short-hand matrix iterations
wand_instance[x:width, y:height]
Here's an example of generating 10px columns...
from wand.image import Image
with Image(filename="rose:") as rose:
x = 0
chunk_size = 10
while True:
try:
with rose[x:x+chunk_size, 0:rose.height] as chunk:
chunk.save(filename='rose_{0}.png'.format(x))
x += chunk_size
except IndexError:
break

Using astropy.fits and numpy to apply coincidence corrections to SWIFT fits image

This question may be a little specialist, but hopefully someone might be able to help. I normally use IDL, but for developing a pipeline I'm looking to use python to improve running times.
My fits file handling setup is as follows:
import numpy as numpy
from astropy.io import fits
#Directory: /Users/UCL_Astronomy/Documents/UCL/PHASG199/M33_UVOT_sum/UVOTIMSUM/M33_sum_epoch1_um2_norm.img
with fits.open('...') as ima_norm_um2:
#Open UVOTIMSUM file once and close it after extracting the relevant values:
ima_norm_um2_hdr = ima_norm_um2[0].header
ima_norm_um2_data = ima_norm_um2[0].data
#Individual dimensions for number of x pixels and number of y pixels:
nxpix_um2_ext1 = ima_norm_um2_hdr['NAXIS1']
nypix_um2_ext1 = ima_norm_um2_hdr['NAXIS2']
#Compute the size of the images (you can also do this manually rather than calling these keywords from the header):
#Call the header and data from the UVOTIMSUM file with the relevant keyword extensions:
corrfact_um2_ext1 = numpy.zeros((ima_norm_um2_hdr['NAXIS2'], ima_norm_um2_hdr['NAXIS1']))
coincorr_um2_ext1 = numpy.zeros((ima_norm_um2_hdr['NAXIS2'], ima_norm_um2_hdr['NAXIS1']))
#Check that the dimensions are all the same:
print(corrfact_um2_ext1.shape)
print(coincorr_um2_ext1.shape)
print(ima_norm_um2_data.shape)
# Make a new image file to save the correction factors:
hdu_corrfact = fits.PrimaryHDU(corrfact_um2_ext1, header=ima_norm_um2_hdr)
fits.HDUList([hdu_corrfact]).writeto('.../M33_sum_epoch1_um2_corrfact.img')
# Make a new image file to save the corrected image to:
hdu_coincorr = fits.PrimaryHDU(coincorr_um2_ext1, header=ima_norm_um2_hdr)
fits.HDUList([hdu_coincorr]).writeto('.../M33_sum_epoch1_um2_coincorr.img')
I'm looking to then apply the following corrections:
# Define the variables from Poole et al. (2008) "Photometric calibration of the Swift ultraviolet/optical telescope":
alpha = 0.9842000
ft = 0.0110329
a1 = 0.0658568
a2 = -0.0907142
a3 = 0.0285951
a4 = 0.0308063
for i in range(nxpix_um2_ext1 - 1): #do begin
for j in range(nypix_um2_ext1 - 1): #do begin
if (numpy.less_equal(i, 4) | numpy.greater_equal(i, nxpix_um2_ext1-4) | numpy.less_equal(j, 4) | numpy.greater_equal(j, nxpix_um2_ext1-4)): #then begin
#UVM2
corrfact_um2_ext1[i,j] == 0
coincorr_um2_ext1[i,j] == 0
else:
xpixmin = i-4
xpixmax = i+4
ypixmin = j-4
ypixmax = j+4
#UVM2
ima_UVM2sum = total(ima_norm_um2[xpixmin:xpixmax,ypixmin:ypixmax])
xvec_UVM2 = ft*ima_UVM2sum
fxvec_UVM2 = 1 + (a1*xvec_UVM2) + (a2*xvec_UVM2*xvec_UVM2) + (a3*xvec_UVM2*xvec_UVM2*xvec_UVM2) + (a4*xvec_UVM2*xvec_UVM2*xvec_UVM2*xvec_UVM2)
Ctheory_UVM2 = - alog(1-(alpha*ima_UVM2sum*ft))/(alpha*ft)
corrfact_um2_ext1[i,j] = Ctheory_UVM2*(fxvec_UVM2/ima_UVM2sum)
coincorr_um2_ext1[i,j] = corrfact_um2_ext1[i,j]*ima_sk_um2[i,j]
The above snippet is where it is messing up, as I have a mixture of IDL syntax and python syntax. I'm just not sure how to convert certain aspects of IDL to python. For example, the ima_UVM2sum = total(ima_norm_um2[xpixmin:xpixmax,ypixmin:ypixmax]) I'm not quite sure how to handle.
I'm also missing the part where it will update the correction factor and coincidence correction image files, I would say. If anyone could have the patience to go over it with a fine tooth comb and suggest the neccessary changes I need that would be excellent.
The original normalised image can be downloaded here: Replace ... in above code with this file

One very important thing about numpy is that it does every mathematical or comparison function on an element-basis. So you probably don't need to loop through the arrays.
So maybe start where you convolve your image with a sum-filter. This can be done for 2D images by astropy.convolution.convolve or scipy.ndimage.filters.uniform_filter
I'm not sure what you want but I think you want a 9x9 sum-filter that would be realized by
from scipy.ndimage.filters import uniform_filter
ima_UVM2sum = uniform_filter(ima_norm_um2_data, size=9)
since you want to discard any pixel that are at the borders (4 pixel) you can simply slice them away:
ima_UVM2sum_valid = ima_UVM2sum[4:-4,4:-4]
This ignores the first and last 4 rows and the first and last 4 columns (last is realized by making the stop value negative)
now you want to calculate the corrections:
xvec_UVM2 = ft*ima_UVM2sum_valid
fxvec_UVM2 = 1 + (a1*xvec_UVM2) + (a2*xvec_UVM2**2) + (a3*xvec_UVM2**3) + (a4*xvec_UVM2**4)
Ctheory_UVM2 = - np.alog(1-(alpha*ima_UVM2sum_valid*ft))/(alpha*ft)
these are all arrays so you still do not need to loop.
But then you want to fill your two images. Be careful because the correction is smaller (we inored the first and last rows/columns) so you have to take the same region in the correction images:
corrfact_um2_ext1[4:-4,4:-4] = Ctheory_UVM2*(fxvec_UVM2/ima_UVM2sum_valid)
coincorr_um2_ext1[4:-4,4:-4] = corrfact_um2_ext1[4:-4,4:-4] *ima_sk_um2
still no loop just using numpys mathematical functions. This means it is much faster (MUCH FASTER!) and does the same.
Maybe I have forgotten some slicing and that would yield a Not broadcastable error if so please report back.
Just a note about your loop: Python's first axis is the second axis in FITS and the second axis is the first FITS axis. So if you need to loop over the axis bear that in mind so you don't end up with IndexErrors or unexpected results.