There are 192 x 144 pixel images. They should be imported to a Python list so that the items in the list are NDArray instances. New dataframe should be created from the list and that dataframe should be given to Isomap. iso.fit(df) fails with the errors
array = array.astype(np.float64)
ValueError: setting an array element with a sequence.
I have spent more than one day trying to figure out how the NDArrays should be processed and the dataframe loaded with them. No luck. Any help would be appreciated.
import pandas as pd
from scipy import misc
import glob
from sklearn import manifold
samples = []
for filename in glob.glob('Datasets/ALOI/32/*.png'):
img = misc.imread(filename, mode='I')
samples.append(img)
df = pd.DataFrame.from_records(samples, coerce_float=True)
iso = manifold.Isomap(n_neighbors=6, n_components=3)
iso.fit(df)
If those are gray scale images from the ALOI, you probably want to treat each pixel's brightness as a feature. Therefore, you should flatten the img array with img.reshape(-1). The revised code follows:
import pandas as pd
from scipy import misc
import glob
from sklearn import manifold
samples = []
for filename in glob.glob('Datasets/ALOI/32/*.png'):
img = misc.imread(filename, mode='I')
# the following line changed
samples.append(img.reshape(-1))
df = pd.DataFrame.from_records(samples, coerce_float=True)
iso = manifold.Isomap(n_neighbors=6, n_components=3)
iso.fit(df)
Related
I am trying to align 4 fits (V2.fits,V3.fits,V4.fits and, V5.fits) with respect to a reference fits file (V1.fits). I tried the following by employing the python fits_align package :
from fits_align.ident import make_transforms
from fits_align.align import affineremap
from glob import glob
from astropy.io import fits
from numpy import shape
import os
img_list = sorted(glob(os.path.join("*.fits")))
ref_image = img_list[0]
images_to_align = img_list[1:]
print(img_list) # List of all images
print(ref_image) # Reference image
print(images_to_align) # Images to align
identifications = make_transforms(ref_image, images_to_align)
aligned_images = [ref_image]
for id in identifications:
if id.ok:
alignedimg = affineremap(id.ukn.filepath, id.trans, outdir=tmpdir)
aligned_images.append(alignedimg)
I get the following index error:
What could be the reasons for this error? please suggest possible solutions
I know this topic has been asked before, but as i'm new to python I couldn't fully understand how to do that and I would like to get explanations about.
I have ndarray cube (stack of images from the same location with the same size and shape which differs in the wavelength they were taken).
I want to convert this image into pandas dataframe in order to be able to iterate through specific rows.
i'm really confused because of the big number of columns I have: I ahve 1024 columns in each image and that confuse me when I need to index those images.
My end goal is to get in the end the images in structure of df, so maybe it means to have kind of imagecollection that I can iterate rows in each one of them.
this is the code I have written until now:
import spectral.io.envi as envi
import matplotlib.pyplot as plt
import os
from spectral import *
import numpy as np
#Create the image path
#the path
img_path = r'N:\this\is\a\path\capture'
cali_path=r'N:\location\Image_Python'
#the specific file
img_file = 'emptyname_2019-08-13_11-05-46.hdr'
img_dark= 'DARKREF_emptyname_2019-08-13_11-05-46.hdr'
cali_hdr= 'Radiometric_1x1.hdr'
cali_img = 'Radiometric_1x1.cal'
img= envi.open(os.path.join(img_path,img_file)).load()
img_dark= envi.open(os.path.join(img_path,img_dark)).load()
img_cali= envi.open(os.path.join(cali_path,cali_hdr), image = os.path.join(cali_path,cali_img)).load()
cali_shape=img_cali.shape
dark_shape=img_dark.shape
img_shape=img.shape
print('shape image:',img_shape,'shape dark:',dark_shape,'calibration shape:',cali_shape)
wavelength=[float(i) for i in img.metadata['wavelength']]
#get the exposure time
tint=float(img.metadata['tint'])
print(tint)
#goak: need to reduce the dark reference from DN image.
#step 1: for each column in the dark reference, calculate mean. then reduce this mean line from the DN image.
#we have created average according to the horizontal axix- axis=0, it calculates the mean for the whole column and we get one row.
dark_1024=img_dark.mean(axis=0)
from numpy import asarray
import pandas as pd
img_np=asarray(img)
dark_np=asarray(img_dark)
cali_np=asarray(img_cali)
I am trying to read a bunch of png images in a directory into a python list. After getting the images in the list I would like to resize and do some reshaping. But it always throws an error. Below is the sample of my code.
Well it doesn't really throw an error but it always prints out an empty dataset.
import pandas as pd
from sklearn.manifold import Isomap
from scipy import misc
from os import listdir
import glob
dset = []
for image_path in glob.glob("/Documents/Python/DAT210x-master/Module4 /Datasets/ALOI/32/*.png"):
img = misc.imread(image_path)
img = img[::2, ::2]
X = (img / 255.0).reshape(-1)
dset.append(X)
dset = pd.DataFrame(dset)
print(dset)
If I'm not mistaken, DataFrame objects are typically initialized from dict objects. What happens if you add:
dset = {'Pictures': dset}
dset = pd.DataFrame(dset)
I load images with scipy's misc.imread, which returns in my case 2304x3 ndarray. Later, I append this array to the list and convert it to a DataFrame. The purpose of doing so is to later apply Isomap transform on the DataFrame. My data frame is 84 rows/samples (images in the folder) and 2304 features each feature is array/list of 3 elements. When I try using Isomap transform I get error:
ValueError: setting an array element with a sequence.
I think error is there because elements of my data frame are of the object type. First I tried using a conversion to_numeric on each column, but got an error, then I wrote a loop to convert each element to numeric. The results I get are still of the object type. Here is my code:
import pandas as pd
from scipy import misc
from mpl_toolkits.mplot3d import Axes3D
import matplotlib
import matplotlib.pyplot as plt
import glob
from sklearn import manifold
samples = []
path = 'Datasets/ALOI/32/*.png'
files = glob.glob(path)
for name in files:
img = misc.imread(name)
img = img[::2, ::2]
x = (img/255.0).reshape(-1,3)
samples.append(x)
df = pd.DataFrame.from_records(samples, coerce_float = True)
for i in range(0,2304):
for j in range(0,84):
df[i][j] = pd.to_numeric(df[i][j], errors = 'coerce')
df[i] = pd.to_numeric(df[i], errors = 'coerce')
print df[2303][83]
print df[2303].dtype
print df[2303][83].dtype
#iso = manifold.Isomap(n_neighbors=6, n_components=3)
#iso.fit(df)
#manifold = iso.transform(df)
#print manifold.shape
Last four lines commented out because they give an error. The output I get is:
[ 0.05098039 0.05098039 0.05098039]
object
float64
As you can see each element of DataFrame is of the type float64 but whole column is an object.
Does anyone know how to convert whole data frame to numeric?
Is there another way of applying Isomap?
Do you want to reshape your image to a new shape instead of the original one?
If that is not the case then you should change the following line in your code
x = (img/255.0).reshape(-1,3)
with
x = (img/255.0).reshape(-1)
Hope this will resolve your issue
I have a script that reads in image data, and then iterates over the images with the median filter in scipy.ndimage. From the iteration i create new arrays.
However when i attempt to run the script with
run filtering.py
The filtering does not seem to work. The new arrays (month_f) are the same as the old ones.
import matplotlib.pyplot as plt
import numpy as numpy
from scipy import ndimage
import Image as Image
# Get images
#Load images
jan1999 = Image.open('jan1999.tif')
mar1999 = Image.open('mar1999.tif')
may1999 = Image.open('may1999.tif')
sep1999 = Image.open('sep1999.tif')
dec1999 = Image.open('dec1999.tif')
jan2000 = Image.open('jan2000.tif')
feb2000 = Image.open('feb2000.tif')
#Compute numpy arrays
jan1999 = numpy.array(jan1999)
mar1999 = numpy.array(mar1999)
may1999 = numpy.array(may1999)
sep1999 = numpy.array(sep1999)
dec1999 = numpy.array(dec1999)
jan2000 = numpy.array(jan2000)
feb2000 = numpy.array(feb2000)
########### Put arrays into a list
months = [jan1999, mar1999, may1999, sep1999, dec1999, jan2000, feb2000]
############ Filtering = 3,3
months_f = []
for image in months:
image = scipy.ndimage.median_filter(image, size=(5,5))
months_f.append(image)
Any help would be much appreciated :)
This is rather a comment but due to reputation limits I'm not able to write one.
The way you import your modules is a bit strange. Especially "import .. as" with the idential name. I think a more pythonian way would be
import matplotlib.pyplot as plt
import numpy as np
from scipy import ndimage
from PIL import Image
and then call
image = ndimage.median_filter(image, size=(...))
When I run your steps with a RGB test image it seems to work.
What does jan1999.shape return?