Develop Reference

explain uplaoding .RAW data images into the jupyter notebook?

Greeting I know how to import .raw data images into the jupyter notebook but I don.t seem to understand the logic explanation.. Below is the code give, please explain these line to me in as simple way as possible
# Read input RAW file
raw_file = np.fromfile('Sandstones/'+name+'_2d25um_binary.raw', dtype=np.uint8)
im = (raw_file.reshape(1000,1000,1000))
im = im==0;
I tried alternate ways but this was the best as well as the preferred. I have run it but I don't seem to understand the last line

name is some variable. Let's say it is name of your file. np.fromfile takes in a path, and the datatype it wants to read, and convert it into an array.
so
'Sandstones/'+name+'_2d25um_binary.raw'
is the path, if the name is file1 then it will be
'Sandstones/file1_2d25um_binary.raw'
You read this file into a numpy array which is a special data structure as uint8.
In the next line of code, you simply reshape the values you read from the file to a 3D array with the shape
1000, 1000, 1000
The last line changes the im array values. If you notice, im is your numpy array, All the values, which are equal to 0 will be changed to 1 as im==0 will be true, rest will be changed to false. You can think of these as a Boolean array of 0s and 1s where 1 are those number which were 0 in the original array. You can learn more about it here

Is there any way to use arithmetic ops on FITS files in Python?

I'm fairly new to Python, and I have been trying to recreate a working IDL program to Python, but I'm stuck and keep getting errors. I haven't been able to find a solution yet.
The program requires 4 FITS files in total (img and correctional images dark, flat1, flat2). The operations are as follows:
flat12 = (flat1 + flat2)/2
img1 = (img - dark)/flat12
The said files have dimensions (1024,1024,1). I have resized them to (1024,1024) to be able to even use im_show() function.
I have also tried using cv2.add(), but I get this:
TypeError: Expected Ptr for argument 'src1'
Is there any workaround for this? Thanks in advance.

To read your FITS files use astropy.io.fits: http://docs.astropy.org/en/latest/io/fits/index.html
This will give you Numpy arrays (and FITS headers if needed, there are different ways to do this, as explained in the documentation), so you could do something like:
>>> from astropy.io import fits
>>> img = fits.getdata('image.fits', ext=0) # extension number depends on your FITS files
>>> dark = fits.getdata('dark.fits') # by default it reads the first "data" extension
>>> darksub = img - dark
>>> fits.writeto('out.fits', darksub) # save output
If your data has an extra dimension, as shown with the (1024,1024,1) shape, and if you want to remove that axis, you can use the normal Numpy array slicing syntax: darksub = img[0] - dark[0].
Otherwise in the example above it will produce and save a (1024,1024,1) image.

Numpy fromfile function does not work properly

So i'm working on a code where i need to write and read files. I'm using python and numpy, but the numpy fromfile function does not seem to work propperly. First i create an array with 500 elements, and save it with the savetxt function. I check the file and it is all right, just how i wanted.
import numpy as np
w = np.zeros(500, float)
np.savetxt("weights.txt", weight, '%.100f', )
print(weight[2])
But after i change the line where i create the array with the one where i read it from a file a problem accours. The zeros turn into really small numbers. I can't guess why. Here is the line where i read from file:
weight = np.fromfile("weights.txt", float, -1)
Should i write a custom function that turns files into arrays or is there a way to make it work?

Creating an image from single bits in Python 3

I have created an array of bits by using this:
Data = []
Bytes = numpy.fromfile(filename, dtype = "uint8")
Bits = numpy.unpackbits(Bytes)
for b in Bits:
Data.append(b)
"filename" ends with ".png".
Later on, I do some stuff with these bits. I want to save an image with another(or the same) set of bits. How do I do it? The best option would be using: saveToPNG(Data)

You can save those bits as a PNG file by simply reversing the steps you've used.
BTW, there's no need to create the Data list: you can access the bits in the Bits array with normal Python functions & operators as well as with Numpy. But if you really do want those bits in a plain Python list then there's no need for that slow for ... append loop: just pass the array to the list constructor.
I've changed your variable names to make them conform to the PEP-8 style guide.
import numpy as np
# File names
in_name = 'square.png'
out_name = 'square_out.png'
# Read data and convert to a list of bits
in_bytes = np.fromfile(in_name, dtype = "uint8")
in_bits = np.unpackbits(in_bytes)
data = list(in_bits)
# Convert the list of bits back to bytes and save
out_bits = np.array(data)
print(np.all(out_bits == in_bits))
out_bytes = np.packbits(out_bits)
print(np.all(out_bytes == in_bytes))
out_bytes.tofile(out_name)
However, I don't know why you want to do this. If you want access to the image data in the PNG file then you need to decode it properly. A simple way to do that is to use PIL (Pillow) to load the image file into a PIL Image object; Numpy can make an array from a PIL Image. You can then use standard Numpy tools to analyze or manipulate the raw image data, and then pass it back to PIL to save it as a PNG (or various other image file formats). See the final code block in this answer for an example.

Creating a new Image file in astropy with specified dimensions of another image file

I'm having a problem wit fits file manipulation in the astropy package, and I'm in need of some help.
I essentially want to take an image I have in fits file format, and create a new file I need to start inputing correction factors to and a new image which can then be used with the correction factors and the original image to produce a correction image. Each of these will have the same dimensions.
Starting with this:
from astropy.io import fits
# Compute the size of the images (you can also do this manually rather than calling these keywords from the header):
#URL: /Users/UCL_Astronomy/Documents/UCL/PHASG199/M33_UVOT_sum/UVOTIMSUM/M33_sum_epoch1_um2_norm.img
nxpix_um2_ext1 = fits.open('...')[1]['NAXIS1']
nypix_um2_ext1 = fits.open('...')[1]['NAXIS2']
#nxpix_um2_ext1 = 4071 #hima_sk_um2[1].header['NAXIS1'] # IDL: nxpix_uw1_ext1 = sxpar(hima_sk_uw1_ext1,'NAXIS1')
#nypix_um2_ext1 = 4321 #hima_sk_um2[1].header['NAXIS2'] # IDL: nypix_uw1_ext1 = sxpar(hima_sk_uw1_ext1,'NAXIS2')
# Make a new image file with the same dimensions (and headers, etc) to save the correction factors:
coicorr_um2_ext1 = ??[nxpix_um2_ext1,nypix_um2_ext1]
# Make a new image file with the same dimensions (and headers, etc) to save the corrected image:
ima_sk_coicorr_um2_ext1 = ??[nxpix_um2_ext1,nypix_um2_ext1]
Can anyone give me the obvious knowledge I am missing to do this...the last two lines are just there to outline what is missing. I have included ?? to perhaps signal I need something else there perhaps fits.writeto() or something similar...

The astropy documentation takes you though this task step by step: create an array with size (NAXIS1,NAXIS2), put the data in the primary HDU, make an HDUlist and write it to disk:
import numpy as np
from astropy.io import fits
data = np.zeros((NAXIS2,NAXIS1))
hdu = fits.PrimaryHDU(data)
hdulist = fits.HDUList([hdu])
hdulist.writeto('new.fits')

I think #VincePs answer is correct but I'll add some more information because I think you are not using the capabilities of astropy well here.
First of all Python is zero-based so the primary extension has the number 0. Maybe you got that wrong, maybe you don't - but it's uncommon to access the second HDU so I thought I better mention it.
hdu_num = 0 # Or use = 1 if you really want the second hdu.
First you do not need to open the same file twice, you can open it once and close it after extracting the relevant values:
with fits.open('...') as hdus:
nxpix_um2_ext1 = hdus[hdu_num]['NAXIS1']
nxpix_um2_ext1 = hdus[hdu_num]['NAXIS2']
# Continue without indentation and the file will be closed again.
or if you want to keep the whole header (for saving it later) and the data you can use:
with fits.open('...') as hdus:
hdr = hdus[hdu_num].header
data = hdus[hdu_num].data # I'll also take the data for comparison.
I'll continue with the second approach because I think it's a lot cleaner and you'll have all the data and header values ready.
new_data = np.zeros((hdr['NAXIS2'], hdr['NAXIS1']))
Please note that Python interprets the axis different than IRAF (and I think IDL, but I'm not sure) so you need axis2 as first and axis1 as second element.
So do a quick check that the shapes are the same:
print(new_data.shape)
print(data.shape)
If they are not equal I got confused about the axis in Python (again) but I don't think so. But instead of creating a new array based on the header values you can also create a new array by just using the old shape:
new_data_2 = np.zeros(data.shape)
That will ensure the dimensions and shape is identical. Now you have an empty image. If you rather like a copy then you can, but do not need to, explicitly copy the data (except if you opened the file explicitly in write/append/update mode then you should always copy it but that's not the default.)
new_data = data # or = data.copy() for explicitly copying.
Do your operations on it and if you want to save it again you can use what #VinceP suggested:
hdu = fits.PrimaryHDU(new_data, header=hdr) # This ensures the same header is written to the new file
hdulist = fits.HDUList([hdu])
hdulist.writeto('new.fits')
Please note that you don't have to alter the shape-related header keywords even if you changed the data's shape because during writeto astropy will update these (by default)