I'm using gdal.RasterizeLayer() to convert a shapefile to a GeoTiff using a GeoTiff template while burning the output values by ATTRIBUTE. What I want to output is a .tif where the burn value corresponds to the value of a given attribute. What I find is that gdal.RasterizeLayer() is burning to strange values that do not correspond to the values in my attribute field. Here's what I have currently:
gdalformat = 'GTiff'
datatype = gdal.GDT_Byte
# Open Shapefile
shapefile = ogr.Open(self.filename)
shapefile_layer = shapefile.GetLayer()
# Get projection info from reference image
image = gdal.Open(ref_image, gdal.GA_ReadOnly)
output = gdal.GetDriverByName(gdalformat).Create(output_tif, image.RasterXSize, image.RasterYSize, 1, datatype,
options=['COMPRESS=DEFLATE'])
output.SetProjection(image.GetProjectionRef())
output.SetGeoTransform(image.GetGeoTransform())
# Write data to band 1
band = output.GetRasterBand(1)
band.SetNoDataValue(0)
gdal.RasterizeLayer(output, [1], shapefile_layer, options=['ATTRIBUTE=FCode'])
# Close datasets
band = None
output = None
image = None
shapefile = None
# Build image overviews
subprocess.call("gdaladdo --config COMPRESS_OVERVIEW DEFLATE " + output_tif + " 2 4 8 16 32 64", shell=True)
What occurs is that the output .tif correctly assigns different burn values for each attribute, but the value does not correspond to the attribute value. For example, the input attribute value FCode=46006 turns into a burn value of 182 (and it's not clear why!). I tried adding and removing the 'COMPRESS=DEFLATE' option, and adding and removing the '3D' option for gdal.RasterizeLayer(). None affect the output burn values.
You can see the input shapefile and attribute values here: input .shp
And the output, with the incorrect values, here: output raster
I fixed this myself by changing the type to gdal.GDT_Int32.
Related
I am new to python and FITS image files, as such I am running into issues. I have two FITS files; the first FITS file is pixels/counts and the second FITS file (calibration file) is pixels/wavelength. I need to convert pixels/counts into wavelength/counts. Once this is done, I need to output wavelength/counts as a new FITS file for further analysis. So far I have managed to array the required data as shown in the code below.
import numpy as np
from astropy.io import fits
# read the images
image_file = ("run_1.fits")
image_calibration = ("cali_1.fits")
hdr = fits.getheader(image_file)
hdr_c = fits.getheader(image_calibration)
# print headers
sp = fits.open(image_file)
print('\n\nHeader of the spectrum :\n\n', sp[0].header, '\n\n')
sp_c = fits.open(image_calibration)
print('\n\nHeader of the spectrum :\n\n', sp_c[0].header, '\n\n')
# generation of arrays with the wavelengths and counts
count = np.array(sp[0].data)
wave = np.array(sp_c[0].data)
I do not understand how to save two separate arrays into one FITS file. I tried an alternative approach by creating list as shown in this code
file_list = fits.open(image_file)
calibration_list = fits.open(image_calibration)
image_data = file_list[0].data
calibration_data = calibration_list[0].data
# make a list to hold images
img_list = []
img_list.append(image_data)
img_list.append(calibration_data)
# list to numpy array
img_array = np.array(img_list)
# save the array as fits - image cube
fits.writeto('mycube.fits', img_array)
However I could only save as a cube, which is not correct because I just need wavelength and counts data. Also, I lost all the headers in the newly created FITS file. To say I am lost is an understatement! Could someone point me in the right direction please? Thank you.
I am still working on this problem. I have now managed (I think) to produce a FITS file containing the wavelength and counts using this website:
https://www.mubdirahman.com/assets/lecture-3---numerical-manipulation-ii.pdf
This is my code:
# Making a Primary HDU (required):
primaryhdu = fits.PrimaryHDU(flux) # Makes a header # or if you have a header that you’ve created: primaryhdu = fits.PrimaryHDU(arr1, header=head1)
# If you have additional extensions:
secondhdu = fits.ImageHDU(wave)
# Making a new HDU List:
hdulist1 = fits.HDUList([primaryhdu, secondhdu])
# Writing the file:
hdulist1.writeto("filename.fits", overwrite=True)
image = ("filename.fits")
hdr = fits.open(image)
image_data = hdr[0].data
wave_data = hdr[1].data
I am sure this is not the correct format for wavelength/counts. I need both wavelength and counts to be contained in hdr[0].data
If you are working with spectral data, it might be useful to look into specutils which is designed for common tasks associated with reading/writing/manipulating spectra.
It's common to store spectral data in FITS files using tables, rather than images. For example you can create a table containing wavelength, flux, and counts columns, and include the associated units in the column metadata.
The docs include an example on how to create a generic "FITS table" writer with wavelength and flux columns. You could start from this example and modify it to suit your exact needs (which can vary quite a bit from case to case, which is probably why a "generic" FITS writer is not built-in).
You might also be able to use the fits-wcs1d format.
If you prefer not to use specutils, that example still might be useful as it demonstrates how to create an Astropy Table from your data and output it to a well-formatted FITS file.
I have an original DICOM file that has DICOM tags BitsAllocated (0028|0100)=16 and BitsStored (0028|0101)=12. I use SimpleITK to read this series, modify it and then I would like to save it again as a DICOM series using the same values for the two tags specified above.
After modifying the dataset the data format is uint16.
This is the code I use:
writer = sitk.ImageFileWriter()
writer.KeepOriginalImageUIDOn()
# Copy relevant tags from the original meta-data dictionary (private tags are also accessible).
tags_to_copy = ["0010|0010", # Patient Name
"0010|0020", # Patient ID
"0010|0030", # Patient Birth Date
"0020|000D", # Study Instance UID, for machine consumption
"0020|0010", # Study ID, for human consumption
"0008|0020", # Study Date
"0008|0030", # Study Time
"0008|0050", # Accession Number
"0008|0060" # Modality
]
modification_time = time.strftime("%H%M%S")
modification_date = time.strftime("%Y%m%d")
# Copy some of the tags and add the relevant tags indicating the change.
# For the series instance UID (0020|000e), each of the components is a number, cannot start
# with zero, and separated by a '.' We create a unique series ID using the date and time.
# tags of interest:
direction = sitk_stack.GetDirection()
series_tag_values = [(k, rs.GetMetaData(0, k)) for k in tags_to_copy if rs.HasMetaDataKey(0, k)] + \
[("0008|0031", modification_time), # Series Time
("0008|0021", modification_date), # Series Date
("0008|0008", "DERIVED\\SECONDARY"), # Image Type
("0020|000e", "1.2.826.0.1.3680043.2.1125." + modification_date + ".1" + modification_time), # Series Instance UID
("0020|0037", '\\'.join(map(str, (direction[0], direction[3], direction[6], # Image Orientation (Patient)
direction[1], direction[4], direction[7])))),
("0008|103e", rs.GetMetaData(0, "0008|103e") + " Processed-SimpleITK"),
("0028|0101", '12'), # gray values window
("0028|0102", '11'), # gray values window
("0028|0100", '16'),
('0028|0103', '0'), ]
for i in range(sitk_stack.GetDepth()):
image_slice = sitk_stack[:, :, i]
# Tags shared by the series.
for tag, value in series_tag_values:
image_slice.SetMetaData(tag, value)
# Slice specific tags.
image_slice.SetMetaData("0008|0012", time.strftime("%Y%m%d")) # Instance Creation Date
image_slice.SetMetaData("0008|0013", time.strftime("%H%M%S")) # Instance Creation Time
image_slice.SetMetaData("0020|0032", '\\'.join(map(str, sitk_stack.TransformIndexToPhysicalPoint((0, 0, i))))) # Image Position (Patient)
image_slice.SetMetaData("0020,0013", str(i)) # Instance Number
# Write to the output directory and add the extension dcm, to force writing in DICOM format.
writer.SetFileName(os.path.join(output_dir, str(i) + '.dcm'))
writer.Execute(image_slice)
'''
When I look at the images afterwards using MeVisLab, I notice that the BitsAlocated and BitsStored are both 16 rather than 16 and 12. What am I doing wrong? Is it possible to store the images using only 12 bits?
SimpleITK doesn't support 12-bit pixels, so it cannot write them. Because of that, when a 12-bit pixel image is read, it automatically gets converted to 16-bit.
I don't know of python DICOM packages that support writing of 12-bit pixel images. SimpleITK uses GDCM or DCMTK for DICOM I/O. If you use those libraries directly, you might be able to do it, although I don't know. But via SimpleITK you can't.
I have a script that's supposed to open a png image and then resize it and then save it as an jpg in numerical sequence. But the code for the number sequencing I copied from the internet isn't working with PIL. It gives me the exception "KeyError: 'W'"
import os
from PIL import Image
os.chdir('C:\\Users\\paul\\Downloads')
# open canvas.png
original = Image.open('canvas.png')
# resize image height to 2160
size = (3000, 2160)
original.thumbnail(size)
# convert to RGB
RGB = original.convert('RGB')
# save image as sequence
i = 0
while os.path.exists("image%s.jpg" % i):
i += 1
RGB.save("image%s.jpg" % i, "w")
Is there another way to do this?
Edit based on Haken Lid's comment
The PIL documentation says that the function save accepts these argument:
Image.save(fp, format=None, **params)
The parameter w you passed is not within the set of accepted file format.
Here you can see which formats are accepted. To make it works, just drop the w argument and substitute the %s with %d (i is an integer, not a string):
RGB.save("image%d.jpg" % i)
Note: from your tags it is not clear if you're using python2 or python3. If you are using python 3, I suggest to use the new method to format string:
RGB.save("image{}.jpg".format(i))
You can even specify a padding so that you can sort your file by name later on:
RGB.save("image{:04d}.jpg".format(i))
where 4 means that your number will be padded with zeros as to have length of at least 4.
I have two files, one an esri shapefile (.shp), the other a point cloud (.las).
Using laspy and shapefile modules I've managed to find which points of the .las file fall within specific polygons of the shapefile. What I now wish to do is to add an index number that enables identification between the two datasets. So e.g. all points that fall within polygon 231 should get number 231.
The problem is that as of yet I'm unable to append anything to the list of points when writing the .las file. The piece of code that I'm trying to do it in is here:
outFile1 = laspy.file.File("laswrite2.las", mode = "w",header = inFile.header)
outFile1.points = truepoints
outFile1.points.append(indexfromshp)
outFile1.close()
The error I'm getting now is: AttributeError: 'numpy.ndarray' object has no attribute 'append'. I've tried multiple things already including np.append but I'm really at a loss here as to how to add anything to the las file.
Any help is much appreciated!
There are several ways to do this.
Las files have classification field, you could store the indexes in this field
las_file = laspy.file.File("las.las", mode="rw")
las_file.classification = indexfromshp
However if the Las file has version <= 1.2 the classification field can only store values in the range [0, 35], but you can use the 'user_data' field which can hold values in the range [0, 255].
Or if you need to store values higher than 255 / you need a separate field you can define a new dimension (see laspy's doc on how to add extra dimensions).
Your code should be close to something like this
outFile1 = laspy.file.File("laswrite2.las", mode = "w",header = inFile.header)
# copy fields
for dimension in inFile.point_format:
dat = inFile.reader.get_dimension(dimension.name)
outFile1.writer.set_dimension(dimension.name, dat)
outFile1.define_new_dimension(
name="index_from_shape",
data_type=7, # uint64_t
description = "Index of corresponding polygon from shape file"
)
outFile1.index_from_shape = indexfromshp
outFile1.close()
Folks,
I am using this link as starting point to convert my CompressedDepth (image of type: "32FC1; compressedDepth," in meters) image to OpenCV frames:
Python CompressedImage Subscriber Publisher
I get an empty data when I try to print, or I get a NonType when I see the result of my array, etc.
What is the right way to convert a compressedDepth image?
Republishing is not gonna work do to wifi/router bandwidth and speed constraints.
The right way to decode compressedDepth is to first remove the header from the raw data and then convert the remaining data.
This is documented in image_transport_plugins/compressed_depth_image_transport/src/codec.cpp.
On my machine the header size is 12 bytes. This might however be different on other architectures since the size of an enum is not defined.
The following python code snippet exports compressed 16UC1 and 32FC1 depth images as png file:
# 'msg' as type CompressedImage
depth_fmt, compr_type = msg.format.split(';')
# remove white space
depth_fmt = depth_fmt.strip()
compr_type = compr_type.strip()
if compr_type != "compressedDepth":
raise Exception("Compression type is not 'compressedDepth'."
"You probably subscribed to the wrong topic.")
# remove header from raw data
depth_header_size = 12
raw_data = msg.data[depth_header_size:]
depth_img_raw = cv2.imdecode(np.fromstring(raw_data, np.uint8), cv2.CV_LOAD_IMAGE_UNCHANGED)
if depth_img_raw is None:
# probably wrong header size
raise Exception("Could not decode compressed depth image."
"You may need to change 'depth_header_size'!")
if depth_fmt == "16UC1":
# write raw image data
cv2.imwrite(os.path.join(path_depth, "depth_" + str(msg.header.stamp) + ".png"), depth_img_raw)
elif depth_fmt == "32FC1":
raw_header = msg.data[:depth_header_size]
# header: int, float, float
[compfmt, depthQuantA, depthQuantB] = struct.unpack('iff', raw_header)
depth_img_scaled = depthQuantA / (depth_img_raw.astype(np.float32)-depthQuantB)
# filter max values
depth_img_scaled[depth_img_raw==0] = 0
# depth_img_scaled provides distance in meters as f32
# for storing it as png, we need to convert it to 16UC1 again (depth in mm)
depth_img_mm = (depth_img_scaled*1000).astype(np.uint16)
cv2.imwrite(os.path.join(path_depth, "depth_" + str(msg.header.stamp) + ".png"), depth_img_mm)
else:
raise Exception("Decoding of '" + depth_fmt + "' is not implemented!")