I want to do some math on my data and stuck with it.
I am reading string lists from csv files and can plot them so far.
Now I am trying to convert the string lists to float lists as i want to do some math on the list items and create new lists and plot them.
import numpy as np
import math
import matplotlib.pyplot as plt
with open("testlog.csv") as f:
data = f.read()
data = data.split('\n')
time = [row.split(',')[0] for row in data]
gyro_x_lsb = [row.split(',')[1] for row in data]
gyro_y = [row.split(',')[2] for row in data]
gyro_z = [row.split(',')[3] for row in data]
accel_x = [row.split(',')[4] for row in data]
accel_y = [row.split(',')[5] for row in data]
accel_z = [row.split(',')[6] for row in data]
comp_x = [row.split(',')[7] for row in data]
comp_y = [row.split(',')[8] for row in data]
comp_z = [row.split(',')[9] for row in data]
temp = [row.split(',')[10] for row in data]
gyro_x_lsb = float(gyro_x_lsb)# make floats in a new list
gyro_x_dps = [gyro_x_lsb / (32768*2000) for gyro_x_dps_f in gyro_x_lsb]
fig = plt.figure()
ax1 = fig.add_subplot(211)
ax1.set_title("Gyro X AR [LSB]")
ax1.set_xlabel('Time[ms]')
ax1.set_ylabel('AR [LSB]')
ax1.plot(time,gyro_x_lsb, c='r', label='X')
leg = ax1.legend()
ax2 = fig.add_subplot(212)
ax2.set_title("MPU9250_test_Accel")
ax2.set_xlabel('Time[ms]')
ax2.set_ylabel('Acceleration')
ax2.plot(time,accel_x, c='r', label='Accel_X')
leg = ax2.legend()
plt.show()
I am trying to calculate every item in "gyro_x_lsb"/32768*2000. But it will not work.
I have tried map also already.
Thanks in advance...
TMP36
BTW: I use Anaconda 3(2.5.0) with Python 3.5
I am new to Python and to this forum.
List comprehensions will do a lot for you.
Like:
gyro_x_lsb_sum = sum(float(item) for item in gyro_x_lsb)
Here I am using a generator expression which is the same syntax as a list comprehension, but better performance in this case.
Hope it helps a bit.
PS. Consider using the Python csv module to read CSV files.
Thanks for the hints with the CSV module. This read my data now as tuple. This was also new to me, and I found a possibility to make some math on my data. Here is the code now that works for me, maybe not perfect, but i am happy for now.
import csv
import matplotlib.pyplot as plt
with open("testlog.csv") as data:
#reader = csv.reader(data) #read columns as strings
reader = csv.reader(data, quoting=csv.QUOTE_NONNUMERIC) #read columns as numbers
time, gyro_x, gyro_y, gyro_z, accel_x, accel_y, accel_z, comp_x, comp_y, comp_z, temp = zip(*reader)
#gyro_x_dps = gyro_x
def gyro_x_dps(gyro_x):
return tuple( [e / 65536000 for e in gyro_x])
fig = plt.figure()
ax1 = fig.add_subplot(211)
ax1.set_title("Gyro X AR [LSB]")
ax1.set_xlabel('Time[ms]')
ax1.set_ylabel('AR [LSB]')
ax1.plot(time,gyro_x, c='r', label='X')
leg = ax1.legend()
ax2 = fig.add_subplot(212)
ax2.set_title("Gyro X AR [dps]")
ax2.set_xlabel('Time[ms]')
ax2.set_ylabel('AR [dps]')
ax2.plot(time,gyro_x_dps(gyro_x), c='r', label='X')
leg = ax2.legend()
plt.show()
Related
I need to plot the data from csv file. yvalues are not in order thats why i need to sort them. I rotate the x axis because the strings are so long. Ho can I do?
####My data:####
####Code:#####
x = []
y = []
with open('output.csv','r') as csvfile:
plots = csv.reader(csvfile, delimiter=',')
for row in plots:
x.append(row[1])
y.append(row[2])
plt.plot(x[1:],y[1:], label='Loaded from file!')
plt.xlabel('x')
plt.xticks(rotation=90)
plt.ylabel('y')
plt.title('Interesting Graph\nCheck it out')
plt.legend()
plt.gcf().autofmt_xdate()
plt.show()
EDIT:
It seems problem is simpler then I expected and solution is
y[1:] = sorted(y[1:])
Old answer
You should keep x and y as pairs on one list - like this
data = [ [x1, y1], [x2, y2], ... ]
so you could read it as
data = list(plots)
and then you can sort it
data.sort(key=lambda i:i[2])
or
data = sorted(data, key=lambda i:i[2])
And after that you should split data
x = [i[1] for i in data]
y = [i[2] for i in data]
OR you should read it to pandas.DataFrame
import pandas as pd
data = pd.read_csv('output.csv')
and then you can sort
data = data.sort_values(by='TotalTime')
and plot it without spliting
plt.plot(data['Name'], data['TotalTime']
I have data that this script outputs as a scatterplot of ID_all[:,0],ID_all[:,1], but I want to also save the data used for the scatter plot as two columns (as a csv file). Any help really appreciated! Thanks in advance.
More Info:
Here is a link to the initial csv data so that you can see my problem (only first two columns used).
https://www.dropbox.com/s/966iviv01tccv1k/contacts.csv?dl=0
If you plot the input csv file (see figure 1), you will see that it only has half of the values that I need. So when I plot this with the below script, I transpose the 'matrix', making a mirror image of this vrt the main diagonal. Now with this script I plot both the original and the transpose (figure 2), but I somehow cannot figure out how to just output columns of data in addition to the figure.
import numpy as np
import matplotlib.pyplot as plt
def parsefile(filename):
i = 0
print('Opening File')
try:
for line in open(filename):
line = line.split()
ID = np.hstack((int(line[0]), int(line[1])))
if i==0:
ID_list = ID
i+=1
if i>0:
ID_list = np.vstack((ID_list, ID))
print('File Opened')
return ID_list
except:
return ID_list
def get_scatter_data(ID_list):
ID_swapped1 = ID_list[:, 1]
ID_swapped2 = ID_list[:, 0]
ID_swapped = np.transpose([ID_swapped1.T, ID_swapped2.T])
N = np.max(ID_list)
diagonal = np.linspace(1,N,N)
ID_diagonal = np.transpose([diagonal.T,diagonal.T])
ID_all = np.vstack((ID_list,ID_swapped,ID_diagonal))
return ID_all
def plot_contactmap(ID_all, savefilename):
f = plt.figure()
plt.scatter(ID_all[:,0],ID_all[:,1], marker='o', s=0.01)
plt.axis('square')
plt.xlabel('Residue Number', fontsize = 14)
plt.ylabel('Residue Number', fontsize = 14)
plt.tick_params(axis='both', which='major', labelsize=14)
plt.show()
f.savefig(savefilename+'.png', bbox_inches='tight')
f.savefig(savefilename+'.pdf', bbox_inches='tight')
def main(filename, savefilename):
ID_list = parsefile(filename+'.csv')
ID_all = get_scatter_data(ID_list)
plot_contactmap(ID_all, savefilename)
main('contacts', 'test')
You can save using numpy.savetxt() method
import numpy as np
ID_all = np.array([[1,2],[3,4],[5,6]])
np.savetxt('output.csv', ID_all, delimiter=',', fmt="%d")
If you want to give column names you can use header argument np.savetxt() method.
np.savetxt('output.csv', ID_all, delimiter=',', header='first, second', fmt="%d")
I'm having issues labeling all the points plotted with the code below,
each row in the CSV file is name, lat,lon.
Points are plotting just fine.
Everything I've tried to have names be plotted along x,y has borked everything in one way or another.
Anyone have any tips?
with open('/.../geo/ALL.csv') as csvfile:
reader = csv.reader(csvfile,delimiter=',')
for data in reader:
names.append(str(data[0]))
lats.append(float(data[1]))
lons.append(float(data[2]))
x,y = map(lons,lats)
map.plot(x,y,'r*',markersize=0.02,color='yellow',marker='D')
You goal is to plot (lons,lats) points in the xy-plane with a label on each. This is one way to do it:
import csv, matplotlib.pyplot as plt
names, lats, lons = [], [], []
with open('file.csv') as csvfile:
reader = csv.reader(csvfile,delimiter=',')
for data in reader:
names.append(str(data[0]))
lats.append(float(data[1]))
lons.append(float(data[2]))
fig, ax = plt.subplots()
ax.scatter(lats, lons, marker='*', s=400) # play with 's' to control the dot size
pad = " " # play with this to pad the label from the dot
for i, n in enumerate(names):
ax.annotate(pad+str(n), (lats[i], lons[i]))
plt.savefig("scatter.png") # you will find scatter.png file in the same directory
This is the scatter.png I get:
with this file.csv CSV file:
n1,1,8
n4,7,2
n2,3,14
n7,13,4
n5,9,6
n6,11,12
n3,5,10
n8,15,18
n9,17,16
I've written the code below to import a number of files from a folder, read and convert them before plotting them in a 3D plot. The number of files is usually larger than 30 and lower than 200 but exceptions might occur. Each file has arround 5000 lines with 3 plottable values. It works and produces a nice 3D plot, but it is very slow. I suspect I have made an array or list grow inside itself. I am particularly suspecting the third for loop. I've tried to run it using 121 files and takes about half an hour to plot.
Each data file is a diffractogram and what I want to do is essentially something like this:
http://www.carbonhagen.com/_/rsrc/1404718703333/abstracts/insitux-raydiffractionsynthesisofgrapheneoxideandreducedgrapheneoxide/M%C3%B8ller%20Storm%20res%20pic.png?height=371&width=522
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib.collections import PolyCollection
import glob
import os
file_list = glob.glob(os.path.join(os.getcwd(),'C:\Users\mkch\Python_Scripts\znox_1','*.ras'))
sourcefiles = []
for file_path in file_list: #this forloops reads all files
with open(file_path) as f_input:
sourcefiles.append(f_input.readlines())
Now all the files have been imported into the list sourcefiles.
data = []
alldata = []
cutdata = []
length = 118#len(sourcefiles)
for i in range(0,length):
l = len(sourcefiles[i])
cdata = sourcefiles[i][320:l-2]
cutdata.append(cdata)
This for loop removes headliners and the last two lines in each file.
fig = plt.figure()
ax = fig.gca(projection='3d')
verts = []
zs = list(range(length))
print zs
for j in range(length):
lines = cutdata[j][:]
x = []
y = []
z = []
for line in lines:
a, b, c = line.split()[0:3]
x.append(a)
y.append(b)
y[0], y[-1] = 0, 0
verts.append(list(zip(x, y)))
poly = PolyCollection(verts, facecolors=['r', 'g', 'b','y'])
ax.add_collection3d(poly, zs=zs, zdir='y')
This bit of code splits each line into the three values that needs plotting. Then it adds the the data to a plot. I suspect the above code is taking quite long.
poly.set_alpha(0.7)
ax.set_xlim3d(0, 100)
ax.set_ylabel('Y')
ax.set_ylim3d(-1, 120)
ax.set_zlabel('Z')
ax.set_zlim3d(0, 120000)
plt.xlabel('2$ \theta$')
plt.show()
Standard plotting things.
I'm very beginner at Python and matplotlib but trying to learn! I would like to use matplotlib to plot some simple data from a CSV containing dates with a frequency. The X axis containing dates and Y containing the frequency. Example data from CSV:
2011/12/15,5
2011/12/11,4
2011/12/19,2
I checked the "matplotlib.sf.net/examples" out but appears all the test data is downloaded from a http get. I would really appreciate if someone could guide me with some example code of how to read in (presumably using CSV reader) and display data in chart.
Thank you!!
Maybe you look for something like:
import csv
import datetime as dt
import matplotlib.pyplot as plt
arch = 'C:\\Python26\\programas\\test.csv'
data = csv.reader(open(arch))
data = [(dt.datetime.strptime(item, "%Y/%m/%d"), float(value)) for item, value in data]
data.sort()
[x, y] = zip(*data)
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(x, y)
ax.grid(True)
fig.autofmt_xdate()
plt.show()
I've tried to keep my code as simple as possible and this is by no means elegant, but here you go:
import csv
import matplotlib.pyplot as plt
### Making test CSV file ###
data = [['2011/12/15,5'],['2011/12/11,4'],['2011/12/19,2'],['2011/12/16,3'],['2011/12/20,8'],['2011/12/14,4'],['2011/12/10,10'],['2011/12/9,7']]
with open('test.csv', 'wb') as f:
writer = csv.writer(f)
for i in data:
writer.writerow(i)
### Extract data from CSV ###
with open('test.csv', 'rb') as n:
reader = csv.reader(n)
dates = []
freq = []
for row in reader:
values = row[0].split(',')
dates.append(values[0])
freq.append(values[1])
### Do plot ###
false_x = [x for x in range(len(dates))]
plt.plot(false_x,freq, 'o-')
plt.xticks(range(len(dates)), (dates), rotation=45)
# plt.axis([xmin, xmax, ymin, ymax]) - sets axes limits on graph
plt.axis([-1, 8, 0, 11])
plt.show()
This makes: