I have a 64 bit unsigned binary file that contains multiple images, this file is an output of a numeric analysis software that is designed to store graphic information in binary form. The software itself has its builtin function to export images but it's an old software, doing that is an pain in the ass.
So I am trying to convert this file to multiple images using python. I found an potential solution here.
The following is the code I copied from aforementioned post with minimal change for my specific file:
import numpy as np
import matplotlib.pyplot as plt
def main():
data = read_data('test21.SGR', 8192, 8192)
visualize(data)
def read_data(filename, width, height):
with open(filename, 'r') as infile:
# Skip the header
infile.seek(8192)
data = np.fromfile(infile, dtype=np.uint64)
# Reshape the data into a 3D array. (-1 is a placeholder for however many
# images are in the file... E.g. 2000)
return data.reshape((width, height, -1))
def visualize(data):
# There are better ways to do this, but let's keep it simple
plt.ion()
fig, ax = plt.subplots()
im = ax.imshow(data[:,:,0], cmap=plt.cm.gray)
for i in xrange(data.shape[-1]):
image = data[:,:,i]
im.set(data=image, clim=[image.min(), image.max()])
fig.canvas.draw()
main()
But when I use this code the error says:
ValueError: total size of new array must be unchanged
If this will work out I will be basically saving at least 2 hours at work from image extraction. I am a newbie in Python so I don't quite understand how to resolve this issue, any help will be appreciated here.
Related
I am trying to make a palettized version of my height image data (using Python/Matplotlib) and for some reason...it is giving me quite weird horizontal lines which I know are not actually present in the dataset.
Both images (mine and the "better" one).
Is this something weird with how Matplotlib normalizes the data? I just don't quite understand how this could happen, so I am at a loss for where to start. I have provided my code below (sorry if there is a typo, I slightly changed it to make sense outside of the code).
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
# file location of the raw data
fileloc = r'C:\Users\...\raw_height_profile.csv'
# generate height profile map
palettized_image = getheightprofile(fileloc)
def getheightprofile(fileloc, color_palette='jet'):
# read data from file
data = pd.read_csv(fileloc, skiprows=0)
# generate colormap (I'm using the jet colormap rn)
colormap = plt.get_cmap(color_palette)
# normalize the height data to the range [0, 1]
norm = (data - np.min(data)) / (np.max(data) - np.min(data))
# convert the height data to RGB values using the palette
palettized_data = (colormap(norm)*255).astype(np.uint8)
# save the file as a png (to check quality)
saveloc = r'C:\Users\...\palletized_height_profile.png'
plt.imsave(saveloc, palettized_data)
# return the nice numbers for later analysis
return palettized_data
But instead of returning the nice image that I think I should get, it returns a super weird image with lines across it. note: I know these images aren't quite the same palettization, but I think you can understand the issue.
Does anyone understand how, why, etc.? I have also attached a link to the dataset, because maybe that is helpful...but I am quite sure there is nothing wrong with the data.
I'm fairly new to Python, and I have been trying to recreate a working IDL program to Python, but I'm stuck and keep getting errors. I haven't been able to find a solution yet.
The program requires 4 FITS files in total (img and correctional images dark, flat1, flat2). The operations are as follows:
flat12 = (flat1 + flat2)/2
img1 = (img - dark)/flat12
The said files have dimensions (1024,1024,1). I have resized them to (1024,1024) to be able to even use im_show() function.
I have also tried using cv2.add(), but I get this:
TypeError: Expected Ptr for argument 'src1'
Is there any workaround for this? Thanks in advance.
To read your FITS files use astropy.io.fits: http://docs.astropy.org/en/latest/io/fits/index.html
This will give you Numpy arrays (and FITS headers if needed, there are different ways to do this, as explained in the documentation), so you could do something like:
>>> from astropy.io import fits
>>> img = fits.getdata('image.fits', ext=0) # extension number depends on your FITS files
>>> dark = fits.getdata('dark.fits') # by default it reads the first "data" extension
>>> darksub = img - dark
>>> fits.writeto('out.fits', darksub) # save output
If your data has an extra dimension, as shown with the (1024,1024,1) shape, and if you want to remove that axis, you can use the normal Numpy array slicing syntax: darksub = img[0] - dark[0].
Otherwise in the example above it will produce and save a (1024,1024,1) image.
I'm working on a project that involves Python. I've NEVER used it along with OpenCV. The objective is to take a 16x16 section of an video, I'm practicing with a single Image, and get it's RBG value. I'm suppose to run this for thousands of frame on a video, which i dont know how to loop. Once I have it ([ 71 155 90]) for example, I want to save it to a notepad, excel sheet, or some simple way or referring to my results.
I've tried looking up tutorials on how to export values, but they've used so many different terms that I don't know where to start.
import numpy as np
import cv2
img = cv2.imread('dog.jpg', cv2.IMREAD_COLOR)
px = img[16,16]
print(px)
The only thing I get is the RBG output [ 71 155 90] in the terminal. I don't know where to do from there. I don't know how to export the value.
you can use openpyxl or
import numpy as np import cv2
img = cv2.imread('dog.jpg', cv2.IMREAD_COLOR)
px = img[16,16]
import pandas as px
df = pd.DataFrame(px)
df.to_excel('filename.xlsx')
You'll need to open a file and then write the results to that file, here is one possible way to do this (although perhaps not the most optimal):
fp = open('output.csv', 'w')
fp.write('{},{},{}'.format(px[0],px[1],px[2])
# write more values here
fp.close() # do this at the end of your writes
I am currently working on something similar, instead of videos I am working with images so I went around searching for tutorials on how to do bulk export of images/ frames from a folder and saving the data into a numpy array.
This is a sample of my code *not sure how much errors are inside but it is able to load and save image frames into an array. I use tqdm to show a simple progress bar so I know what is the status of the image loading when I call this function.
def img_readph(path):
readph =[i for i in listdir(path) if isfile(join(path,i))]
img = np.empty(len(readph), dtype=object)
for j in tqdm(range(0, len(readph))):
img[j] = cv2.imread(join(path,readph[j]))
return img
In order to load and work on the images that are currently saved in a numpy array stack, I use this set of code to do the extraction, perform a basic psnr calculation and save the data to a .txt (learning how to convert the result to a .csv that I can load/save/append in python for future edits as well).
for index in tqdm(range(len(img))):
(psnr, meanerror) = calculate_psnr(imgGT[index], imgSR[index])
print('Image No.{} has average mean square error of {} and the average PSNR is {}'.format(index,meanerror,psnr))
Doing it this way lets me loop every video frame I have in the previous numpy array to perform my operation to calculate psnr.
What you could do is to try write your code to get the RGB values into a function using,
txtfilename = input("enter filename: ")
with open(str(txtfilename)+".txt","w") as results:
for index in tqdm(range(0, len(img))) #the array from imread
img = cv2.imread('img', cv2.IMREAD_COLOR)
px = img[width, height]
print("The RBG values are {}".format(px), file=results)
Something along the lines of this I guess, hope it helps.
I have created an array of bits by using this:
Data = []
Bytes = numpy.fromfile(filename, dtype = "uint8")
Bits = numpy.unpackbits(Bytes)
for b in Bits:
Data.append(b)
"filename" ends with ".png".
Later on, I do some stuff with these bits. I want to save an image with another(or the same) set of bits. How do I do it? The best option would be using: saveToPNG(Data)
You can save those bits as a PNG file by simply reversing the steps you've used.
BTW, there's no need to create the Data list: you can access the bits in the Bits array with normal Python functions & operators as well as with Numpy. But if you really do want those bits in a plain Python list then there's no need for that slow for ... append loop: just pass the array to the list constructor.
I've changed your variable names to make them conform to the PEP-8 style guide.
import numpy as np
# File names
in_name = 'square.png'
out_name = 'square_out.png'
# Read data and convert to a list of bits
in_bytes = np.fromfile(in_name, dtype = "uint8")
in_bits = np.unpackbits(in_bytes)
data = list(in_bits)
# Convert the list of bits back to bytes and save
out_bits = np.array(data)
print(np.all(out_bits == in_bits))
out_bytes = np.packbits(out_bits)
print(np.all(out_bytes == in_bytes))
out_bytes.tofile(out_name)
However, I don't know why you want to do this. If you want access to the image data in the PNG file then you need to decode it properly. A simple way to do that is to use PIL (Pillow) to load the image file into a PIL Image object; Numpy can make an array from a PIL Image. You can then use standard Numpy tools to analyze or manipulate the raw image data, and then pass it back to PIL to save it as a PNG (or various other image file formats). See the final code block in this answer for an example.
I'm trying to get data from "data.txt" into a numpy array and plot it with matplotlib. This is what each line of the data looks like:
"1" 11.658870417634 4.8159509459201
with about ten million lines.
I'm trying to get it into a memory map, but keep getting this error:
ValueError: Size of available data is not a multiple of data-type size.
Here is the code I am using:
import numpy
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
datatype=[('index',numpy.int), ('floati',numpy.float32), ('floatq',numpy.float32)]
filename='data.txt'
def main():
data = numpy.memmap(filename, datatype, 'r')
plt.plot(data['floati'],data['floatq'],'r,')
plt.grid(True)
plt.title("Signal-Diagram")
plt.xlabel("Sample")
plt.ylabel("In-Phase")
plt.savefig('foo2.png')
if __name__ == "__main__":
main()
If you could please help me figure out where I'm going wrong, I would greatly appreciate it.
That's a text file, not a binary file. memmap only works if the data is stored as binary (i.e. stored as the array would be stored in memory).
You can convert the file to binary by reading it in, line-by-line, and storing each converted line in a writable memmap.
You're better off using numpy.genfromtxt or numpy.loadtxt. For example:
datatype=[('index',numpy.int), ('floati',numpy.float32), ('floatq',numpy.float32)]
filename='data.txt'
def main():
data = numpy.genfromtxt(filename, dtype=datatype)
...