I am trying to change the value of a keyword in the header of a FITS file.
Quite simple, this is the code:
import pyfits
hdulist = pyfits.open('test.fits') # open a FITS file
prihdr = hdulist[1].header
print prihdr['AREASCAL']
effarea = prihdr['AREASCAL']/5.
print effarea
prihdr['AREASCAL'] = effarea
print prihdr['AREASCAL']
I print the steps many times to check the values are correct. And they are.
The problem is that, when I check the FITS file afterwards, the keyword value in the header is not changed. Why does this happen?
You are opening the file in read-only mode. This won't prevent you from modifying any of the in-memory objects, but closing or flushing to the file (as suggested in other answers to this question) won't make any changes to the file. You need to open the file in update mode:
hdul = pyfits.open(filename, mode='update')
Or better yet use the with statement:
with pyfits.open(filename, mode='update') as hdul:
# Make changes to the file...
# The changes will be saved and the underlying file object closed when exiting
# the 'with' block
You need to close the file, or explicitly flush it, in order to write the changes back:
hdulist.close()
or
hdulist.flush()
Interestingly, there's a tutorial for that in the astropy tutorials github. Here is the ipython notebook viewer version of that tutorial that explains it all.
Basically, you are noticing that the python instance does not interact with disk instance. You have to save a new file or overwrite the old one explicitly.
Related
Sorry if the question is not well formulated, will reformulated if necessary.
I have a file with an array that I filled with data from an online json db, I imported this array to another file to use its data.
#file1
response = urlopen(url1)
a=[]
data = json.loads(response.read())
for i in range(len(data)):
a.append(data[i]['name'])
i+=1
#file2
from file1 import a
'''do something with "a"'''
Does importing the array means I'm filling the array each time I call it in file2?
If that is the case, what can I do to just keep the data from the array without "building" it each time I call it?
If you saved a to a file, then read a -- you will not need to rebuild a -- you can just open it. For example, here's one way to open a text file and get the text from the file:
# set a variable to be the open file
OpenFile = open(file_path, "r")
# set a variable to be everything read from the file, then you can act on that variable
file_guts = OpenFile.read()
From the Python docs on the Modules section - link - you can read:
When you run a Python module with
python fibo.py <arguments>
the code in the module will be executed, just as if you imported it
This means that importing a module has the same behavior as running it as a regular Python script, unless you use the __name__ as mentioned right after this quotation.
Also, if you think about it, you are opening something, reading from it, and then doing some operations. How can you be sure that the content you are now reading from is the same as the one you had read the first time?
i have a fits file and i want to add a new header to the fits file.
I've actually added a new fits header but it didn't save it. How to save and add new fits header?
Codes here:
from astropy.io import fits
hdul = fits.open('example.fits.gz')[0]
hdul.header.append('GAIN')
hdul.header['GAIN'] = 0.12
hdul.header.comments['GAIN']="e-/ADU"
print(hdul.header)
Thank in advance
open() opens the FITS file in read-only mode by default. If you want to modify the file in place you need to open it with mode='update'. Also, appending the new header can be done in a single line (as documented in Header.append like:
with open('example.fits', mode='update') as hdul:
hdul[0].header.append(('GAIN', 0.12, 'e-/ADU'))
Or, if you already have a FITS file open in read-only mode, you can write the modified file out to a new file using the writeto method as mentioned here.
One caveat I noticed in your original example is you were opening a gzipped FITS file. I'm not actually sure off the top of my head if that can be modified in 'update' mode, in which case you'll definitely need to write to a new file. I believe it does work, so try it, but I forget how well tested that is.
I don't have the 50 reputation points to comment on #Iguananaut's answer, so I'll leave my comment here: Make sure it is fits.open(). Otherwise, it will give you the following error ValueError: invalid mode: 'update'.
Using #Iguananaut's example, it should be:
with fits.open('example.fits', mode='update') as hdul:
hdul[0].header.append(('GAIN', 0.12, 'e-/ADU'))
Also, using append() will append the same 'new' card for every time you run the code. To prevent this, I suggest a minor adjustment. It will not just add the new card you want, but it will also update that same card if you run the code multiple times, avoiding card multiples.
with fits.open('example.fits', mode='update') as hdul:
hdr = hdul[0].header
hdr['GAIN'] = (0.12, 'e-/ADU')
I already have a fits file written (someone sent it to me) and I want to add a checksum and datasum to the header. The only examples I found with adding a checksum using astropy.io.fits involve building a new fits HDU and verifying it on adding each section to the HDU. Which I could do, but that seems a like it would have a lot more overhead then is needed.
Is there a way to add a checksum and datasum to an existing HDU?
ImageHDU objects have a method called add_checksum(). That should do exactly what you want.
So you could open a FITS file in update mode and then call this and close the file again.
from astropy.io import fits
with fits.open(filename, mode='update') as hdus:
hdus[0].add_checksum() # Fill in the arguments like you need them
The with is preferred because it automatically closes the file when the with context is exited (even if an exception happens) but you could also open and close it without the with:
from astropy.io import fits
hdul = fits.open(filename, mode='update')
hdul[0].add_checksum()
hdul.close()
In our lab we store our data in hdf5 files trough the python package h5py.
At the beginning of an experiment we create an hdf5 file and store array after array of array of data in the file (among other things). When an experiment fails or is interrupted the file is not correctly closed.
Because our experiments run from iPython the reference to the data object remains (somewhere) in memory.
Is there a way to scan for all open h5py data objects and close them?
This is how it could be done (I could not figure out how to check for closed-ness of the file without exceptions, maybe you will find):
import gc
for obj in gc.get_objects(): # Browse through ALL objects
if isinstance(obj, h5py.File): # Just HDF5 files
try:
obj.close()
except:
pass # Was already closed
Another idea:
Dpending how you use the files, what about using the context manager and the with keyword like this?
with h5py.File("some_path.h5") as f:
f["data1"] = some_data
When the program flow exits the with-block, the file is closed regardless of what happens, including exceptions etc.
pytables (which h5py uses) keeps track of all open files and provides an easy method to force-close all open hdf5 files.
import tables
tables.file._open_files.close_all()
That attribute _open_files also has helpful methods to give you information and handlers for the open files.
I've found that hFile.bool() returns True if open, and False otherwise. This might be the simplest way to check.
In other words, do this:
hFile = h5py.File(path_to_file)
if hFile.__bool__():
hFile.close()
I'm using xlwt in python to create a Excel spreadsheet. You could interchange this for almost anything else that generates a file; it's what I want to do with the file that's important.
from xlwt import *
w = Workbook()
#... do something
w.save('filename.xls')
I want to I have two use cases for the file: I stream it out to the user's browser or I attach it to an email. In both cases the file only needs to exist the duration of the web request that generates it.
What I'm getting at, the reason for starting this thread is saving to a real file on the filesystem has its own hurdles (stopping overwriting, cleaning up the file once done). Is there somewhere I could "save" it where it lives only in memory and only for the duration of the request?
cStringIO
(or mmap if it should be mutable)
Generalising the answer, as you suggested: If the "anything else that generates a file" won't accept a file-like object as well as a filepath, then you can reduce the hassle by using tempfile.NamedTemporaryFile