I try get mean from csv line. I get data from csv in string list, further i convert it to array with numpy. Its work perfect when i try plot some graphics.
But when i calculate mean i get some errors with my data.
If i use NumPy i get:
TypeError: cannot perform reduce with flexible type
If i use statistics library i get:
TypeError: can't convert type 'string' to numerator/denominator
If i check my array with comand 'type' on iPython i see that it numpy.ndarray type.
Whats wrong with my array? Can you explain, why convert numpy.asarray for matplotlib work perfect, but get wrong type for different operation.
import csv
import numpy as np
import statistics as stat
life_exp=[]
with open('country.csv') as csvfile:
datareader = csv.reader(csvfile)
for row in datareader:
if datareader.line_num!=1:
life_exp.append(row[1])
array_life_exp = np.asarray(life_exp)
print(stat.mean(array_life_exp))
print(np.mean(array_life_exp))
Try this:
from pandas import read_csv
data = read_csv('country.csv')
print(data.iloc[:,1].mean())
This code will convert your csv to pandas dataframe with automatic type conversion and print mean of the second column.
Related
So I've been tasked with creating a suitable 2D array to contain all of the data from a csv with data on rainfall from the whole year. In the csv file, the rows represent the weeks of the year and the columns represent the day of the week.
I'm able to display the date I want using the following code.
import csv
data = list(csv.reader(open("rainfall.csv")))
print(data[1][2])
My issue is I'm not sure how to store this data in a 2D array.
I'm not sure how to go about doing this. Help would be appreciated, thanks!
You could use numpy for that. It seems to me, that you have created a list of lists in data. With that you can directly create a 2D numpy-array:
import numpy as np
2d_data = np.array(data)
Or you could even try to directly read the file with numpy:
import numpy as np
# Use the appropriate delimiter here
2d_data = np.genfromtxt("rainfall.csv", delimiter=",")
With pandas:
import pandas as pd
# Use the appropriate delimiter here
2d_data = pd.read_csv("rainfall.csv")
I am trying to import a csv file as an array in python using the ""numpy.loadtxt"" method. It keeps returning ""ValueError: could not convert string to float: ''"" despite there not being any blank cells in the csv file. Here is my code
import csv
import torch
import numpy as np
import pandas as pd
array = np.loadtxt("HIP Only 2.csv", dtype=np.float32, delimiter=",", skiprows=1)
It seems that there are some Non-numerics in the cells, which may be related to specified strings or errors created due to unsuccessful formula in some cells e.g. #DIV/0! in excel files which is appeared when corresponding cells have not filled or numbers divided by zero. numpy.loadtxt is for using when no data is missed. If getting array is the main goal, not using numpy.loadtxt, numpy.genfromtxt is more flexible and could be used instead e.g.:
array = np.genfromtxt("HIP Only 2.csv", dtype=np.float32, delimiter=",", skip_header=1)
Hope it e helpful.
I wanna find the median of a dataset using np.median . But for unexpected reasons, the numpy results differ from each other. If I'm converting the dataframe into a list and than use np.median(li) I've got 1.0791015625 as a result. However if I'm using np.median(df['diesel'])I've got 1.079 as a result. Interestingly using statistics.median() works for both versions (using a list or a dataframe). Does anyone know what I did wrong or what could caused this problem?
import pandas as pd
import numpy as np
import statistics
import math
df = pd.read_csv("2020-08-09-prices.csv",sep=',', usecols=['diesel'], dtype={'diesel': np.float16})
df.info()
li=df['diesel'].tolist()
print(df.describe())
print(np.median(li))
print(statistics.median(df['diesel']))
print(np.median(df['diesel']))
This is where I got the csv file from: https://dev.azure.com/tankerkoenig/_git/tankerkoenig-data?path=%2Fprices%2F2020%2F08
Is it possible to retrieve index point from PCL pointcloud file?
I have pointcloud data in txt file with XYZ and some other colum information. I use the following code to convert the txt file into pcl cloud file:
import pandas as pd
import numpy as np
import pcl
data = pd.read_csv('data.txt', usecols=[0,1,2], delimiter=' ')
pcl_cloud = pcl.PointCLoud()
cloud = pcl_cloud.from_array(np.array(data, dtype = np.float32))
As I know, the module from_array only need the XYZ column. After some processing (eg. filtering), the number of raw and result most probably different. Is it possible to know which point number from the result file, so I can mix it with another information from the raw data?
I tried to filter by comparing the coordinates, but it doesn't work because the coordinate slightly changes during the converting from double to float.
Any idea? Thank you very much
I just got the answer, by using extract indices.
eg:
filter = pcl.RadiusOutlierRemoval(data)
indeces = filter.Extract()
Thanks
I'm using this code:
import arcpy
import numpy as np
f = open("F:\INTRO_PY\LAB_7\lab_7.csv","w")
array = np.random.rand(1000,1000)
f.write(array)
f.close
in order to create a 1000x1000 random array in arcpy.
This is what I get when I open the csv:
CSV
I have absolutely no idea why it's doing this, and I'm at my wit's end. Any advice would be really, really appreciated!
In order to save it to CSV, you need to can use numpy's numpy.savetxt [numpy-doc]:
np.savetxt(
r"F:\INTRO_PY\LAB_7\lab_7.csv",
np.random.rand(1000,1000),
delimiter=','
)
The `delimeter thus specifies what one uses to split the different values.
Note that you can only save 1D arrays or 2D arrays to a text file.
I think you are trying to store a numpy in a file, you should convert it to a string first.
Something like the following:
f = open("test.csv","w")
array = np.random.rand(1000,1000)
f.write(str(array))
f.close