I am working on bayer raw(.raw format) image domain where I need to edit the pixels according to my needs(applying affine matrix) and save them back .raw format.so There are two sub-problems.
I am able to edit pixels but can save them back as .raw
I am using a robust library called rawpy that allows me to read pixel values as numpy array, while I try to save them back I am unable to persist the value
rawImage = rawpy.imread('Filename.raw') // this gives a rawpy object
rawData = rawImage.raw_image //this gives pixels as numpy array
.
.//some manipulations performed on rawData, still a numpy array
.
imageio.imsave('newRaw.raw', rawData)
This doesn't work, throws error unknown file type. Is there a way to save such files in .raw format.
Note: I have tried this as well:-
rawImageManipulated = rawImage
rawImageManipulated.raw_image[:] = rawData[:] //this copies the new
data onto the rawpy object but does not save or persists the values
assigned.
Rotating a bayer image - I know rawpy does not handle this, nor does any other API or Library acc to my knowledge. The existing image rotation Apis of opencv and pillow alter the sub-pixels while rotating. How do I come to know? After a series of small rotations(say,30 degrees of rotation 12 times) when I get back to a 360 degree of rotation the sub-pixels are not the same when compared using a hex editor.
Are there any solutions to these issues? Am I going in the wrong direction? Could you please guide me on this. I am currently using python i am open to solutions in any language or stack. Thanks
As far as I know, no library is able to rotate an image directly in the Bayer pattern format (if that's what you mean), for good reasons. Instead you need to convert to RGB, and back later. (If you try to process the Bayer pattern image as if it was just a grayscale bitmap, the result of rotation will be a disaster.)
Due to numerical issues, accumulating rotations spoils the image and you will never get the original after a full turn. To minimize the loss, perform all rotations from the original, with increasing angles.
Related
I am working on a dataset that has two features, real and imaginary impedances. I applied data-to-image conversion using MTF in order to represent each one as an image (50x50). I was thinking of creating a 3-D image (50x50x2). I tried doing
Image = np.array([tag_gadf_re[0],tag_gadf_im[0]])
where tag_gadf_re[0] and tag_gadf_im[0] are the real and imaginary impedance image arrays. However, I tried saving the image using:
plt.imsave("Directory", Image)
However, I am getting the following error:
ValueError: Third dimension must be 3 or 4
Also note that the shape of Image is (2x50x50), when it should be (50x50x2). The solution seems simple, but I am a bit lost in the process. How can I combine both arrays appropriately and save the image, or do I need a 3rd layer in order to appropriately represent it as an RGB image?
If you want to store data as an image you need to be aware of its type and range so that you can choose an appropriate format. You also need to be aware of whether you can tolerate a "lossy" format which, when read, will not return identical values to those you stored.
If your data is integer and 16-bit or less, you can store it in a PNG. If it's multi-channel and 16-bit, you'll come unstuck with PIL. You can use tifffile though to store a 2-channel TIFF - maybe that can be greyscale + transparency or maybe 2 IFDs.
If your data is floating point, you pretty much have to use TIFF, PFM or EXR format. Again, tifffile can do this for you.
tifffile is here.
wand can also do whatever tifffile can do.
Of course, you might choose to represent your two arrays/images as one above the other in a double-height image. It's your data.
I have a dataset of meshes, which I want to use to generate partial view data as point clouds. In other words, simulating the way an RGB-D sensor would work.
My solution is very naive, so feel free to disregard it and suggest another method.
It consists of taking a rendered RGB and a rendered D image from an o3d visualization, as such:
vis.add_geometry(tr_mesh)
... # set some params to have a certain angle
vis.capture_screen_image('somepath.png')
vis.capture_depth_image('someotherpath.png')
These are saved as PNG files. Then I combine them into an o3d RGBDImage:
# Load the RGB image
rgb = o3d.io.read_image(rgb_path)
# Load the depth image
depth = o3d.io.read_image(depth_path)
# Convert the RGB and depth images into pointcloud
rgbd = o3d.geometry.RGBDImage.create_from_color_and_depth(color=o3d.geometry.Image(rgb),
depth=o3d.geometry.Image(depth),
convert_rgb_to_intensity=False)
And convert this to a PointCloud
pcd = o3d.geometry.PointCloud.create_from_rgbd_image(image=rgbd,
intrinsic=o3d.camera.PinholeCameraIntrinsic(o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
This has many limitations:
the depth is quantized to 256 values. These are literally in 0...255 so they don't match the scale of the image height and width. Not to mention, it completely loses the original scale of the mesh object itself.
the camera params (focal length etc) are not recreated identically so the point cloud is deformed.
Again, this is a very naive solution, so completely different approaches are very welcome.
This is not a duplicate of Can I generate Point Cloud from mesh? as I want partial views. So think that question, but with back-face culling.
Getting back with the solution.
No image capture is needed. There is a function called vis.capture_depth_point_cloud()
So the partial view can be generated by simply running
vis.add_geometry(tr_mesh)
... # set some params to have a certain angle
vis.capture_depth_point_cloud("somefilename.pcd")
This also has a parameter called convert_to_world_coordinate which is very useful.
There doesn't seem to be a way to change the resolution of the sensor. Though up-(or down-)scaling the object, capturing the point cloud, then down-(or up-)scaling the point cloud should obtain the same effect.
I am a bit confused about when an image is gamma encoded/decoded and when I need to raise it to a gamma function.
Given an image 'boat.jpg' where the colour representation is labeled 'sRGB'. My assumption is that the pixel values are encoded in the file by raising the arrays to ^(1/2.2) during the save process.
When I import the image into numpy using scikit-image or opencv I end up with a 3-dim array of uint8 values. Do these values need to be raised to ^2.2 in order to generate a histogram of the values, or when I apply the imread function, does that map the image into linear space in the array?
from skimage import data,io
boat = io.imread('boat.jpg')
if you get your image anywhere on the internet, it has gamma 2.2.
unless the image has an image profile encoded, then you get the gamma from that profile.
imread() reads the pixel values 'as-is', no conversion.
there's no point converting image to gamma 1.0 for any kind of the processing, unless you specifically know that you have to. basically, nobody does that.
As you probably know, skimage uses a handful of different plugins when reading in images (seen here). The values you get should not have to be adjusted...that happens under the hood. I would also recommend you don't use the jpeg file format because you lose data with the compression.
OpenCV (as of v 4) usually does the gamma conversion for you, depending on the image format. It appears to do it automatically with PNG, but it's pretty easy to test. Just generate a 256x256 8-bit color image with a linear color ramps along x and y, then check to see what the pixel values at given image coords. If the sRGB mapping/unmapping is done correctly at every point, x=i should have pixel value i and so on. If you imwrite to PNG in OpenCV, it will convert to sRGB, tag that in the image format, and GIMP or whatever will happily decode it back back to linear.
Most image files are stored as sRGB, and there's a tendency for most image manipulation APIs to handle it correctly, since well, if they didn't, they'd work wrong most of the time. In the odd instance where you read an sRGB file as linear or vice versa, it will make a significant difference though, especially if you're doing any kind of image processing. Mixing up sRGB and linear causes very significant problems, and you will absolutely notice it if it gets messed up; fortunately, the software world usually handles it automagically in the file read/write stage so casual app developers don't usually have to worry about it.
I need to convert raw NEF images into numpy arrays for quantitative analysis. I'm currently using rawpy to do this, but I've failed to find a combination of postprocess parameters that leave the pixel data untouched. (I'll be the first to admit I don't entirely understand how raw files work also...)
Here is what I have right now:
rawArray = raw.postprocess(demosaic_algorithm=rawpy.DemosaicAlgorithm.AHD,
half_size=False,
four_color_rgb=False,use_camera_wb=False,
use_auto_wb=False,user_wb=(1,1,1,1),
output_color=rawpy.ColorSpace.raw,
output_bps=16,user_flip=None,
user_black=None,user_sat=None,
no_auto_bright=False,auto_bright_thr=0.01,
adjust_maximum_thr=0,bright=100.0,
highlight_mode=rawpy.HighlightMode.Ignore,
exp_shift=None,exp_preserve_highlights=0.0,
no_auto_scale=True,gamma=(2.222, 4.5),
chromatic_aberration=None,
bad_pixels_path=None)
Postprocessing, which means demosaicing here, will always change the original pixel values. Typically what you want is to get a linearly postprocessed image so that roughly the number of photons are in linear relation to the pixel values. You can do that with:
rgb = raw.postprocess(gamma=(1,1), no_auto_bright=True, output_bps=16)
In most cases you will not be able to get a calibrated image out where you can directly infer the number of photons that hit the sensor at each pixel. See also https://www.dpreview.com/forums/post/56232710.
Note that you can also access the raw image data via raw.raw_image (see also Bayer matrix) which is as close to the sensor data as you can get -- no interpolation or camera curves etc are applied here, but I would say scientifically you don't get much more than you would get with a linearly postprocessed image as described above.
I have a large 2D array (4000x3000) saved as a numpy array which I would like to display and save while keeping the ability to look at each individual pixels.
For the display part, I currently use matplotlib imshow() function which works very well.
For the saving part, it is not clear to me how I can save this figure and preserve the information contained in all 12M pixels. I tried adjusting the figure size and the resolution (dpi) of the saved image but it is not obvious which figsize/dpi settings should be used to match the resolution of the large 2D matrix displayed. Here is an example code of what I'm doing (arr is a numpy array of shape (3000,4000)):
fig = pylab.figure(figsize=(16,12))
pylab.imshow(arr,interpolation='nearest')
fig.savefig("image.png",dpi=500)
One option would be to increase the resolution of the saved image substantially to be sure all pixels will be properly recorded but this has the significant drawback of creating an image of extremely large size (at least much larger than the 4000x3000 pixels image which is all that I would really need). It also has the disadvantage that not all pixels will be of exactly the same size.
I also had a look at the Python Image Library but it is not clear to me how it could be used for this purpose, if at all.
Any help on the subject would be much appreciated!
I think I found a solution which works fairly well. I use figimage to plot the numpy array without resampling. If you're careful in the size of the figure you create, you can keep full resolution of your matrix whatever size it has.
I figured out that figimage plots a single pixel with size 0.01 inch (this number might be system dependent) so the following code will for example save the matrix with full resolution (arr is a numpy array of shape (3000,4000)):
rows = 3000
columns = 4000
fig = pylab.figure(figsize=(columns*0.01,rows*0.01))
pylab.figimage(arr,cmap=cm.jet,origin='lower')
fig.savefig("image.png")
Two issues I still have with this options:
there is no markers indicating column/row numbers making it hard to know which pixel is which besides the ones on the edges
if you decide to interactively look at the image, it is not possible to zoom in/out
A solution that also solves the above 2 issues would be terrific, if it exists.
The OpenCV library was designed for scientific analysis of images. Consequently, it doesn't "resample" images without your explicitly asking for it. To save an image:
import cv2
cv2.imwrite('image.png', arr)
where arr is your numpy array. The saved image will be the same size as your array arr.
You didn't mention the color-model that you are using. Pngs, like jpegs, are usually 8-bit per color channel. OpenCV will support up to 16-bits per channel if you request it.
Documentation on OpenCV's imwrite is here.