Decimal error in list - python

I'm trying to input a csv file that I can then use to do calculations:
import csv
data=[]
file=input ("Enter file name: ")
with open(file,"r") as f:
reader=csv.reader(f)
for row in reader:
data.append([x.strip(";") for x in row])
print(data)
print("Calculate COV")
lst= data
spl= [x.split(";") for y in lst for x in y]
flattened = [float(x) for y in spl for x in y if x]
print (flattened)
But I keep getting this when I use decimal numbers:
[['13', '25;12', '97;13', '12;13', '47;13', '44;13', '09;12', '86;12', '78;12', '91;12', '93;12', '91;13', '11'], ['12', '92;13', '42;13', '58;13', '7;13', '62;13', '7;13', '31;12', '86;12', '59;12', '81;13', '46;12', '9'], ['13', '39;13', '5;13', '29;13', '26;13', '38;13', '45;13', '46;11', '95;12;12', '57;13', '22;12', '88'], ['12', '48;13', '76;13', '7;13', '77;13', '08;13', '48;13', '25;12', '31;12', '56;12', '56;12', '95;13', '38'], ['12', '52;14', '07;14', '46;14', '13;13', '98;14', '07;13', '92;12', '7;13', '01;12', '79;13;13', '13']]
When I should have this:
[13.25, 12.97, 13.12, 13.47, 13.44, 13.09, 12.86, 12.78, 12.91, 12.93, 12.91, 13.11, 12.92, 13.42, 13.58, 13.7, 13.62, 13.7, 13.31, 12.86, 12.59, 12.81, 13.46, 12.9, 13.39, 13.5, 13.29, 13.26, 13.38, 13.45, 13.46, 11.95, 12.57, 13.22, 12.88, 12.48, 13.76, 13.7, 13.77, 13.08, 13.48, 13.25, 12.31, 12.56, 12.56, 12.95, 13.38, 12.52, 14.07, 14.46, 14.13, 13.98, 14.07, 13.92, 12.7, 13.01, 12.79, 13.0, 13.13]


It's really not clear how you are trying to use the CSV module. First, by default, the csv module is going to use comma separated values, not semicolon separated values.
But either way, let's try writing some code:
import csv
class MyDialect(csv.excel):
delimiter = ';'
with open('in.csv', 'r') as f:
reader = csv.reader(f, MyDialect())
data = list(reader)
data = [[float(elem.replace(',', '.')) for elem in line] for line in data]
for line in data:
print line
in.csv
13,25;12,97;13,12;13,47;13,44;13,09;12,86;12,78;12,91;12,93;12,91;13,11
12,92;13,42;13,58;13,7;13,62;13,7;13,31;12,86;12,59;12,81;13,46;12,9
13,39;13,5;13,29;13,26;13,38;13,45;13,46;11,95;12;12,57;13,22;12,88
12,48;13,76;13,7;13,77;13,08;13,48;13,25;12,31;12,56;12,56;12,95;13,38
12,52;14,07;14,46;14,13;13,98;14,07;13,92;12,7;13,01;12,79;13;13,13
stdout
[13.25, 12.97, 13.12, 13.47, 13.44, 13.09, 12.86, 12.78, 12.91, 12.93, 12.91, 13.11]
[12.92, 13.42, 13.58, 13.7, 13.62, 13.7, 13.31, 12.86, 12.59, 12.81, 13.46, 12.9]
[13.39, 13.5, 13.29, 13.26, 13.38, 13.45, 13.46, 11.95, 12.0, 12.57, 13.22, 12.88]
[12.48, 13.76, 13.7, 13.77, 13.08, 13.48, 13.25, 12.31, 12.56, 12.56, 12.95, 13.38]
[12.52, 14.07, 14.46, 14.13, 13.98, 14.07, 13.92, 12.7, 13.01, 12.79, 13.0, 13.13]


Try this:
import csv
data=[]
import re
with open("out.csv","r") as f:
reader=csv.reader(f,delimiter=";")
for row in reader:
print row
data+=[x.split(",") for x in row]
print(data)
print("Calculate COV")
flattened = [float(x) for y in data for x in y]

Related

Find local maxima in data from dataframe

Is there a way to find the local maxima from data I get from CSV file and put its value on the plot?
The x and y values that are in a pandas dataframe look something like this
x = 1598.78, 1596.85, 1594.92, 1592.99, 1591.07, 1589.14, 1587.21, 1585.28, 1583.35, 1581.42, 1579.49, 1577.57, 1575.64, 1573.71, 1571.78, 1569.85, 1567.92, 1565.99, 1564.07, 1562.14, 1560.21, 1558.28, 1556.35, 1554.42, 1552.49, 1550.57, 1548.64, 1546.71, 1544.78, 1542.85, 1540.92, 1538.99, 1537.07, 1535.14, 1533.21, 1531.28, 1529.35, 1527.42, 1525.49, 1523.57, 1521.64, 1519.71, 1517.78, 1515.85, 1513.92, 1511.99, 1510.07, 1508.14, 1506.21, 1504.28, 1502.35, 1500.42, 1498.49, 1496.57, 1494.64, 1492.71, 1490.78, 1488.85, 1486.92, 1484.99, 1483.07, 1481.14, 1479.21, 1477.28, 1475.35, 1473.42, 1471.49, 1469.57, 1467.64, 1465.71, 1463.78, 1461.85, 1459.92, 1457.99, 1456.07, 1454.14, 1452.21, 1450.28, 1448.35, 1446.42, 1444.49, 1442.57, 1440.64, 1438.71, 1436.78, 1434.85, 1432.92, 1430.99, 1429.07, 1427.14, 1425.21, 1423.28, 1421.35, 1419.42, 1417.49, 1415.57, 1413.64, 1411.71, 1409.78, 1407.85, 1405.92, 1403.99, 1402.07, 1400.14
y = 0.640, 0.624, 0.609, 0.594, 0.581, 0.569, 0.558, 0.547, 0.537, 0.530, 0.523, 0.516, 0.508, 0.502, 0.497, 0.491, 0.487, 0.484, 0.481, 0.480, 0.479, 0.482, 0.490, 0.503, 0.520, 0.542, 0.566, 0.586, 0.600, 0.606, 0.593, 0.569, 0.557, 0.548, 0.538, 0.531, 0.527, 0.524, 0.522, 0.522, 0.523, 0.525, 0.526, 0.527, 0.530, 0.534, 0.536, 0.539, 0.547, 0.553, 0.557, 0.563, 0.573, 0.599, 0.654, 0.738, 0.852, 0.891, 0.810, 0.744, 0.711, 0.694, 0.686, 0.683, 0.683, 0.690, 0.700, 0.706, 0.713, 0.723, 0.731, 0.732, 0.737, 0.756, 0.779, 0.786, 0.790, 0.794, 0.802, 0.815, 0.827, 0.832, 0.831, 0.826, 0.823, 0.828, 0.834, 0.834, 0.832, 0.832, 0.831, 0.825, 0.816, 0.804, 0.798, 0.794, 0.786, 0.775, 0.764, 0.752, 0.739, 0.722, 0.708, 0.697
and I'm trying to get something like this.
P.S. Note that numeric values were added with the plt.text function just to exemplify what I want.

x = [1598.78, 1596.85, 1594.92, 1592.99, 1591.07, 1589.14, 1587.21, 1585.28, 1583.35, 1581.42, 1579.49, 1577.57, 1575.64, 1573.71, 1571.78, 1569.85, 1567.92, 1565.99, 1564.07, 1562.14, 1560.21, 1558.28, 1556.35, 1554.42, 1552.49, 1550.57, 1548.64, 1546.71, 1544.78, 1542.85, 1540.92, 1538.99, 1537.07, 1535.14, 1533.21, 1531.28, 1529.35, 1527.42, 1525.49, 1523.57, 1521.64, 1519.71, 1517.78, 1515.85, 1513.92, 1511.99, 1510.07, 1508.14, 1506.21, 1504.28, 1502.35, 1500.42, 1498.49, 1496.57, 1494.64, 1492.71, 1490.78, 1488.85, 1486.92, 1484.99, 1483.07, 1481.14, 1479.21, 1477.28, 1475.35, 1473.42, 1471.49, 1469.57, 1467.64, 1465.71, 1463.78, 1461.85, 1459.92, 1457.99, 1456.07, 1454.14, 1452.21, 1450.28, 1448.35, 1446.42, 1444.49, 1442.57, 1440.64, 1438.71, 1436.78, 1434.85, 1432.92, 1430.99, 1429.07, 1427.14, 1425.21, 1423.28, 1421.35, 1419.42, 1417.49, 1415.57, 1413.64, 1411.71, 1409.78, 1407.85, 1405.92, 1403.99, 1402.07, 1400.14]
y = [0.640, 0.624, 0.609, 0.594, 0.581, 0.569, 0.558, 0.547, 0.537, 0.530, 0.523, 0.516, 0.508, 0.502, 0.497, 0.491, 0.487, 0.484, 0.481, 0.480, 0.479, 0.482, 0.490, 0.503, 0.520, 0.542, 0.566, 0.586, 0.600, 0.606, 0.593, 0.569, 0.557, 0.548, 0.538, 0.531, 0.527, 0.524, 0.522, 0.522, 0.523, 0.525, 0.526, 0.527, 0.530, 0.534, 0.536, 0.539, 0.547, 0.553, 0.557, 0.563, 0.573, 0.599, 0.654, 0.738, 0.852, 0.891, 0.810, 0.744, 0.711, 0.694, 0.686, 0.683, 0.683, 0.690, 0.700, 0.706, 0.713, 0.723, 0.731, 0.732, 0.737, 0.756, 0.779, 0.786, 0.790, 0.794, 0.802, 0.815, 0.827, 0.832, 0.831, 0.826, 0.823, 0.828, 0.834, 0.834, 0.832, 0.832, 0.831, 0.825, 0.816, 0.804, 0.798, 0.794, 0.786, 0.775, 0.764, 0.752, 0.739, 0.722, 0.708, 0.697]
import matplotlib.pyplot as plt
# The slope of a line is a measure of its steepness. Mathematically, slope is calculated as "rise over run" (change in y divided by change in x).
slope = [np.sign((y[i] - y[i-1]) / (x[i] - x[i-1])) for i in range(1, len(y))]
x_prev = slope[0]
optima_dic={'minima':[], 'maxima':[]}
for i in range(1, len(slope)):
if slope[i] * x_prev == -1: #slope changed
if x_prev == 1: # slope changed from 1 to -1
optima_dic['maxima'].append(i)
else: # slope changed from -1 to 1
optima_dic['minima'].append(i)
x_prev=-x_prev
from matplotlib.pyplot import text
plt.rcParams["figure.figsize"] = (20,10)
ix = 0
for x_, y_ in zip(x, y):
plt.plot(x_, y_, 'o--', color='grey')
if(ix in optima_dic['minima']):
plt.text(x_, y_, s = x_, fontsize=10)
ix += 1

how can i smooth the graph values or extract main signals only

when i try to run the code below i get this graph
my code:
from numpy import nan
import json
import os
import numpy as np
import subprocess
import math
import matplotlib.pyplot as plt
from statistics import mean, stdev
def smooth(t):
new_t = []
for i, x in enumerate(t):
neighbourhood = t[max(i-2,0): i+3]
m = mean(neighbourhood)
s = stdev(neighbourhood, xbar=m)
if abs(x - m) > s:
x = ( t[i - 1 + (i==0)*2] + t[i + 1 - (i+1==len(t))*2] ) / 2
new_t.append(x)
return new_t
def outLiersFN(*U):
outliers=[] # after preprocessing list
#preprocessing Fc =| 2*LF1 prev by 1 - LF2 prev by 2 |
c0 = -2 #(previous) by 2 #from original
c1 =-1 #(previous) #from original
c2 =0 #(current) #from original
c3 = 1 #(next) #from original
preP = U[0] # original list
if c2 == 0:
outliers.append(preP[0])
c1+=1
c2+=1
c0+=1
c3+=1
oldlen = len(preP)
M_RangeOfMotion = 90
while oldlen > c2 :
if c3 == oldlen:
outliers.insert(c2, preP[c2]) #preP[c2] >> last element in old list
break
if (preP[c2] > M_RangeOfMotion and preP[c2] < (preP[c1] + preP[c3])/2) or (preP[c2] < M_RangeOfMotion and preP[c2] > (preP[c1] + preP[c3])/2): #Check Paper 3.3.1
Equ = (preP[c1] + preP[c3])/2 #fn of preprocessing # From third index # ==== inserting current frame
formatted_float = "{:.2f}".format(Equ) #with .2 number only
equu = float(formatted_float) #from string float to float
outliers.insert(c2,equu) # insert the preprocessed value to the List
c1+=1
c2+=1
c0+=1
c3+=1
else :
Equ = preP[c2] # fn of preprocessing #put same element (do nothing)
formatted_float = "{:.2f}".format(Equ) # with .2 number only
equu = float(formatted_float) # from string float to float
outliers.insert(c2, equu) # insert the preprocessed value to the List
c1 += 1
c2 += 1
c0 += 1
c3 += 1
return outliers
def remove_nan(list):
newlist = [x for x in list if math.isnan(x) == False]
return newlist
the_angel = [176.04, 173.82, 170.09, 165.3, 171.8, 178.3, 178.77, 179.24, 179.93, 180.0, 173.39, 166.78, 166.03, 165.28, 165.72, 166.17, 166.71, 167.26, 168.04, 167.22, 166.68, 166.13, 161.53, 165.81, 170.1, 170.05, 170.5, 173.01, 176.02, 174.53, 160.09, 146.33, 146.38, 146.71, 150.33, 153.95, 154.32, 154.69, 134.52, 114.34, 115.6, 116.86, 134.99, 153.12, 152.28, 151.43, 151.36, 152.32, 158.9, 166.52, 177.74, 178.61, 179.47, 167.44, 155.4, 161.54, 167.68, 163.96, 160.24, 137.45, 114.66, 117.78, 120.89, 139.95, 139.62, 125.51, 111.79, 112.07, 112.74, 110.22, 107.7, 107.3, 106.52, 105.73, 103.07, 101.35, 102.5, 104.59, 104.6, 104.49, 104.38, 102.81, 101.25, 100.62, 100.25, 100.15, 100.32, 99.84, 99.36, 100.04, 100.31, 99.14, 98.3, 97.92, 97.41, 96.9, 96.39, 95.88, 95.9, 95.9, 96.02, 96.14, 96.39, 95.2, 94.56, 94.02, 93.88, 93.8, 93.77, 93.88, 94.04, 93.77, 93.65, 93.53, 94.2, 94.88, 92.59, 90.29, 27.01, 32.9, 38.78, 50.19, 61.59, 61.95, 62.31, 97.46, 97.38, 97.04, 96.46, 96.02, 96.1, 96.33, 95.61, 89.47, 89.34, 89.22, 89.48, 89.75, 90.02, 90.28, 88.16, 88.22, 88.29, 88.17, 88.17, 94.98, 94.84, 94.69, 94.94, 94.74, 94.54, 94.69, 94.71, 94.64, 94.58, 94.19, 94.52, 94.85, 87.7, 87.54, 87.38, 95.71, 96.57, 97.11, 97.05, 96.56, 96.07, 95.76, 95.56, 95.35, 95.28, 95.74, 96.2, 96.32, 96.33, 96.2, 96.14, 96.07, 96.07, 96.12, 96.17, 96.28, 96.31, 96.33, 96.16, 96.05, 95.94, 95.33, 88.96, 95.0, 95.78, 88.19, 88.19, 88.19, 87.92, 87.93, 88.03, 87.94, 87.86, 87.85, 87.89, 88.08, 88.01, 87.88, 88.02, 88.15, 88.15, 88.66, 88.73, 88.81, 88.41, 88.55, 88.68, 88.69, 88.02, 87.35, 95.19, 95.39, 95.38, 95.37, 95.27, 95.17, 95.33, 95.32, 95.31, 95.37, 95.42, 95.34, 95.44, 95.53, 95.47, 95.41, 95.13, 94.15, 94.78, 97.64, 97.1, 96.87, 97.03, 96.76, 35.44, 23.63, 23.27, 24.71, 26.16, 96.36, 113.13, 129.9, 96.82, 63.74, 34.25, 33.42, 32.6, 30.69, 31.06, 31.43, 97.14, 97.51, 97.23, 98.54, 100.13, 100.95, 28.82, 33.81, 66.81, 99.82, 102.63, 101.9, 101.44, 102.19, 103.22, 103.67, 104.13, 104.07, 104.73, 105.46, 103.74, 102.02, 103.32, 102.59, 29.54, 28.08, 28.76, 29.79, 30.82, 113.51, 129.34, 145.16, 143.18, 148.29, 153.67, 166.14, 161.16, 151.64, 149.27, 146.9, 151.67, 153.02, 149.28, 145.53, 149.1, 152.67, 158.78, 164.89, 164.84, 164.8, 162.11, 159.42, 156.73, 156.28, 155.83, 156.4, 161.0, 165.59, 164.44, 159.73, 155.76, 156.97, 158.92, 159.15, 159.39, 159.99, 160.44, 160.88, 163.89, 166.9, 167.71, 167.11, 167.0, 167.44, 168.38, 153.16, 137.94, 137.65, 152.09, 169.49, 171.36, 173.22, 174.01, 174.0, 174.2, 174.41, 157.74, 141.09, 149.32, 157.57, 156.4, 148.4, 140.78, 141.06, 141.73, 143.05, 143.91, 156.59, 169.29, 172.17, 175.05, 175.29, 175.27, 175.15, 175.02, 174.81, 174.59, 174.76, 174.94, 175.18, 175.41, 175.23, 174.51, 174.64, 174.77, 174.56, 173.25, 172.38, 174.17, 176.4, 177.27, 177.29, 177.33, 178.64, 179.98, 179.99, 176.0, 172.88, 173.77, 173.8, 173.97, 174.72, 175.24, 176.89, 179.07, 179.27, 178.78, 178.29, 175.61, 174.21, 172.8, 173.05, 173.41, 173.77, 174.65, 175.52, 175.58, 176.15, 176.71, 159.12, 141.54, 141.12, 155.62, 170.53, 165.54, 160.71, 158.22, 156.35, 156.82, 158.55, 160.27, 161.33, 162.39, 162.37, 159.48, 156.59, 156.77, 158.05, 159.32, 158.49, 157.66, 157.7, 157.74, 158.44, 159.14, 150.13, 143.06, 136.0, 125.7, 115.41, 111.19, 106.97, 107.1, 107.24, 107.45, 107.67, 113.34, 119.01, 144.87, 170.73, 174.31, 177.89, 174.78, 171.67, 163.26, 134.58, 105.9, 102.98, 100.77, 101.05, 101.39, 101.73, 99.79, 98.71, 97.64, 97.8, 97.89, 96.67, 95.45, 94.33, 93.38, 92.44, 48.53, 91.4, 91.35, 91.34, 91.33, 90.92, 90.51, 88.63, 87.0, 86.74, 86.48, 96.79, 96.09, 95.46, 95.39, 94.32, 93.25, 93.31, 93.37, 93.11, 92.57, 93.41, 94.25, 96.48, 92.71, 88.94, 90.07, 90.43, 78.06, 77.69, 77.32, 90.1, 89.15, 89.14, 88.85, 88.38, 87.63, 121.2, 120.66, 86.89, 86.42, 85.69, 84.86, 84.86, 85.34, 85.82, 86.07, 86.32, 85.82, 85.32, 86.23, 86.69, 87.15, 87.04, 86.87, 86.58, 86.0, 85.41, 85.41, 85.53, 85.66, 85.7, 85.72, 85.75, 85.92, 86.09, 85.77, 85.45, 84.94, 85.55, 86.16, 86.21, 86.1, 85.77, 85.27, 84.56, 84.99, 85.38, 85.42, 85.98, 86.54, 86.5, 86.45, 86.56, 86.63, 86.35, 86.08, 85.82, 85.51, 85.21, 84.6, 84.84, 84.97, 85.1, 86.12, 86.88, 86.8, 86.46, 86.47, 87.23, 87.8, 88.0, 88.08, 88.16, 87.72, 87.63, 87.37, 86.42, 86.48, 87.24, 87.97, 88.09, 88.19, 88.32, 88.44, 87.82, 87.2, 86.03, 85.78, 91.5, 93.0, 88.2, 88.52, 88.42, 87.28, 85.73, 85.62, 85.5, 85.5, 87.06, 87.6, 88.1, 88.31, 88.53, 88.77, 89.14, 89.52, 89.46, 89.4, 90.28, 89.74, 91.28, 92.17, 92.16, 92.15, 93.08, 94.0, 94.66, 95.32, 94.13, 93.7, 93.32, 93.69, 94.58, 95.47, 97.25, 99.03, 99.63, 99.67, 99.71, 100.33, 101.58, 103.36, 103.49, 103.41, 106.31, 109.34, 109.28, 109.21, 107.76, 106.31, 105.43, 104.94, 104.44, 111.19, 117.93, 115.59, 113.24, 116.15, 119.06, 125.43, 140.72, 156.0, 161.7, 143.52, 135.33, 127.13, 127.68, 148.68, 169.68, 172.2, 174.72, 174.75, 174.66, 158.57, 142.63, 145.13, 153.29, 161.45, 163.34, 165.24, 162.25, 159.89, 159.07, 156.39, 155.21, 156.04, 159.29, 160.07, 160.85, 163.45, 162.93, 161.71, 160.06, 158.4, 144.74, 132.64, 134.57, 150.22, 165.86, 172.95, 174.12, 175.3, 175.5, 176.31, 177.71, 179.72, 168.13, 156.55, 146.24, 155.75, 176.0, 175.99, 175.98, 176.0, 176.02, 176.25, 175.13, 174.26, 173.38, 173.37, 173.46, 176.34, 174.55, 172.77, 168.45, 166.35, 166.47, 168.81, 167.43, 166.79, 167.35, 168.65, 168.51, 168.37, 168.88, 169.74, 171.19, 171.33, 169.91, 168.49, 167.11, 166.83, 167.01, 168.68, 170.34, 170.43, 172.15, 173.86, 177.62, 177.61, 175.34, 173.06, 176.47, 179.87, 179.9, 177.67, 175.67, 175.39, 175.36, 177.03, 176.0, 174.98, 174.96, 174.94, 175.76, 176.57, 169.05, 162.99, 164.97, 168.74, 172.51, 167.38, 165.08, 163.03, 163.81, 164.83, 164.81, 164.8, 165.88, 165.36, 159.61, 153.86, 153.57, 153.61, 153.65, 154.62, 155.58, 157.97, 156.35, 155.66, 154.98, 156.11, 157.24, 159.25, 159.6, 160.43, 161.26, 164.71, 168.17, 147.46, 126.92, 106.38, 105.23, 104.4, 105.37, 106.65, 109.21, 107.44, 104.65, 101.86, 102.35, 102.84, 102.79, 102.19, 101.59, 100.98, 100.38, 98.72, 97.73, 97.32, 96.9, 95.11, 93.97, 94.12, 94.12, 93.1, 92.08, 89.29, 90.35, 90.35, 90.35, 90.35, 86.95, 86.37, 86.06, 85.74, 94.56, 93.16, 92.46, 91.76, 88.55, 85.33, 87.52, 92.18, 93.68, 95.18, 94.4, 92.17, 89.94, 89.4, 89.37, 99.44, 100.98, 102.52, 103.18, 88.96, 88.23, 87.5, 85.2, 85.19, 86.87, 121.42, 155.96, 155.97, 155.97, 86.2, 86.5, 86.8, 87.22, 87.36, 87.34, 87.03, 87.04, 87.05, 86.36, 85.68, 85.71, 85.84, 85.93, 86.01, 86.04, 86.08, 85.92, 86.05, 86.18, 86.17, 86.19, 86.23, 86.22, 86.09, 85.92, 85.66, 85.69, 85.69, 85.31, 84.91, 84.93, 84.95, 84.93, 84.91, 84.9, 84.9, 84.9, 84.9, 85.38, 85.52, 85.66, 85.66, 85.4, 85.14, 85.47, 85.8, 85.72, 85.64, 86.09, 85.84, 85.27, 85.47, 85.66, 85.59, 85.52, 85.38, 85.39, 85.28, 85.17, 85.39, 85.7, 85.98, 86.26, 86.61, 92.97, 93.15, 86.58, 86.58, 86.53, 86.47, 98.55, 99.41, 100.16, 100.9, 89.19, 90.28, 91.38, 91.39, 91.4, 91.44, 92.05, 131.05, 170.63, 170.13, 162.43, 125.64, 88.85, 88.85, 99.08, 100.38, 101.69, 100.74, 99.79, 96.33, 93.31, 93.73, 94.87, 96.01, 96.93, 97.85, 98.97, 97.85, 98.14, 99.37, 102.01, 103.8, 105.58, 108.52, 108.12, 107.72, 106.75, 106.82, 109.08, 112.37, 112.52, 112.66, 112.97, 114.12, 115.64, 117.1, 118.57, 126.13, 133.69, 149.27, 163.96, 166.62, 169.27, 164.94, 160.61, 149.35, 141.18, 143.41, 143.57, 149.26, 157.49, 159.94, 151.93, 147.47, 145.97, 145.56, 145.15, 143.85, 142.54, 142.18, 142.43, 143.12, 144.41, 144.38, 151.99, 159.59, 174.81, 174.94, 175.84, 176.87, 162.41, 152.94, 151.59, 155.24, 155.22, 155.19, 155.04]
p0 = outLiersFN(smooth(remove_nan(the_angel)))
the_angel = p0
plt.plot(the_angel) #list(filter(fun, L1))
plt.show()
print((the_angel))
how can i smooth the values in (the_angel) to get graph like this (red line)
i mean ignoring all unnecessary and noisy values and get only main line instead
you can edit my code or suggest me new filter or algorithm

pandas has a rolling() method for dataframes that you can use to calculate the mean over a window of values, e.g. the 70 closest ones:
import pandas as pd
import matplotlib.pyplot as plt
WINDOW_SIZE = 70
the_angel = [176.04, 173.82, 170.09, 165.3, 171.8, # ...
]
df = pd.DataFrame({'the angel': the_angel})
df[f'mean of {WINDOW_SIZE}'] = df['the angel'].rolling(
window=WINDOW_SIZE, center=True).mean()
df.plot(color=['blue', 'red']);

How to fix python google form and selenium submit error?

I've been trying to test out this google form filler but keep running into an issue with the submit button. This has never happened before and I am not sure why, every time I run the code it fills in all the boxes, however, it sometimes doesn't fill out the gender question and rarely or never hits submit or submit another response. The google form link is in the code below.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
import random
from time import sleep
from random import randint
import random
s=Service('/Users) #have the current service installed here
driver = webdriver.Chrome(service=s)
url="https://forms.gle/sNCNuebKtpdF6j7t6"
driver.get(url)
datas = [
]
for i in range(10):
sampleList = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']
val1 = random.choices(sampleList, weights=(5.8, 6.72, 7.5, 7.24, 6.8, 3.16, 3.5, 4.5, 1.2, 2.3), k=10) # 25,25,50
val2 = random.choices(sampleList, weights=(5.8, 6.7, 7.5, 7.2, 6.83, 9.1, 9.5, 8.5, 8.2, 8.3), k=10) # 25,25,50
val3 = random.choices(sampleList, weights=(5.8, 6.7, 7.52, 7.2, 6.8, 3.1, 4.5, 3.53, 1.2, 0.3), k=10) # 25,25,50
val4 = random.choices(sampleList, weights=(0.8, 2.7, 3.53, 3.2, 5.8, 4.12, 9.5, 9.5, 9.2, 9.3), k=10) # 25,25,50
val5 = random.choices(sampleList, weights=(1.78, 3.7, 5.5, 3.2, 6.8, 5.1, 9.51, 8.5, 8.2, 9.3), k=10) # 25,25,50
valx = random.randint(0,9)
datas.append([val1[valx],val2[valx],val3[valx],val4[valx],val5[valx]]) #this list prints out alright and gets the values needed
radiobuttons = driver.find_elements(By.CLASS_NAME, "appsMaterialWizToggleRadiogroupElContainer")
print(datas)
for data in datas: #for each mini list in the big list
print(data)
radiolist = [0, 1, 2, 3]
val0 = random.choices(radiolist, weights=(9.8,11.5,1.2,1.8), k=10)
print(val0)
rand = random.randint(0,9)
print(rand)
#radiobuttons[val0[rand]].click()
radiobuttons[1].click()
print(radiobuttons[val0[rand]])
count = 0 #count is 0
# contain input boxes
textboxes = driver.find_elements(By.CLASS_NAME, 'quantumWizTextinputPaperinputInput') #these textboxes and elements seem to work and get filled out
for value in textboxes:
# enter value
value.send_keys(data[count])
print(value)
# increment count value
count += 1
submit = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "span[class*='appsMaterialWizButtonPaperbuttonContent ']"))).click() #this sometimes works but if so only once
# fill another response
print('hi')
another_response = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, 'Submit another response'))).click()
#this does not work at all
#textboxes = browser.find_elements_by_class_name("quantumWizTextinputPaperinputInput")
#radiobuttons = browser.find_elements_by_class_name("docssharedWizToggleLabeledLabelWrapper")
#checkboxes = browser.find_elements_by_class_name("quantumWizTogglePapercheckboxInnerBox")
#submitbutton = browser.find_element_by_class_name("appsMaterialWizButtonPaperbuttonContent")
# close the window
driver.close()

url="https://forms.gle/sNCNuebKtpdF6j7t6"
driver.get(url)
datas = []
for i in range(10):
sampleList = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']
val1 = random.choices(sampleList, weights=(5.8, 6.72, 7.5, 7.24, 6.8, 3.16, 3.5, 4.5, 1.2, 2.3), k=10) # 25,25,50
val2 = random.choices(sampleList, weights=(5.8, 6.7, 7.5, 7.2, 6.83, 9.1, 9.5, 8.5, 8.2, 8.3), k=10) # 25,25,50
val3 = random.choices(sampleList, weights=(5.8, 6.7, 7.52, 7.2, 6.8, 3.1, 4.5, 3.53, 1.2, 0.3), k=10) # 25,25,50
val4 = random.choices(sampleList, weights=(0.8, 2.7, 3.53, 3.2, 5.8, 4.12, 9.5, 9.5, 9.2, 9.3), k=10) # 25,25,50
val5 = random.choices(sampleList, weights=(1.78, 3.7, 5.5, 3.2, 6.8, 5.1, 9.51, 8.5, 8.2, 9.3), k=10) # 25,25,50
valx = random.randint(0,9)
datas.append([val1[valx],val2[valx],val3[valx],val4[valx],val5[valx]]) #this list prints out alright and gets the values needed
print(datas)
for data in datas: #for each mini list in the big list
radiobuttons = driver.find_elements(By.CLASS_NAME, "appsMaterialWizToggleRadiogroupElContainer")
print(data)
radiolist = [0, 1, 2, 3]
val0 = random.choices(radiolist, weights=(9.8,11.5,1.2,1.8), k=10)
print(val0)
rand = random.randint(0,9)
print(rand)
#radiobuttons[val0[rand]].click()
radiobuttons[1].click()
print(radiobuttons[val0[rand]])
count = 0 #count is 0
# contain input boxes
textboxes = driver.find_elements(By.CLASS_NAME, 'quantumWizTextinputPaperinputInput') #these textboxes and elements seem to work and get filled out
for value in textboxes:
# enter value
value.send_keys(data[count])
print(value)
# increment count value
count += 1
try:
submit = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[text()='Submit']")))
submit.click() #this sometimes works but if so only once
except Exception as e:
print(str(e))
# fill another response
print('hi')
try:
another_response = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, 'Submit another response')))
another_response.click()
except Exception as e:
print(str(e))
Your issues were in the radiobuttons as you need to regrab that element every loop due to page redirection so you ended up with stale elements and using the wrong selector for the submit button. With the following fixes it runs.

separate lines according to a string in them in python

I have a file containing (~1000) lines of data in the following -simplified- form:
movej(p[-0.2875, -0.4, 0.05, -0.0004, 3.1415, 0.0002],a=5.000000,v=5.000000,r=0.000000)
movej([-1.3615, -2.0244, -1.5691, -2.6895, 0.2092, 1.2197],a=5.000000,v=5.000000,r=0.050000)
movel(p[-0.2875, -0.375, 0.0, -0.0004, 3.1415, 0.0002],a=0.500000,v=0.300000,r=0.000000)
movel([-1.2437, -1.9458, -2.0485, -2.2887, 0.0128, 1.2197],a=5.000000,v=5.000000,r=0.050000)
# Various comments
## Various comments
### Various comments
And I need to separate the lines in two sets (for later conversion to pandas dataframes), one set consisting of the lines with the p -starting with "movej(p[" or "movel(p[" and the other consisting of the lines without the p -starting with "movej([" or "movel([".
movej(p[-0.2875, -0.4, 0.05, -0.0004, 3.1415, 0.0002],a=5.000000,v=5.000000,r=0.000000)
movel(p[-0.2875, -0.375, 0.0, -0.0004, 3.1415, 0.0002],a=0.500000,v=0.300000,r=0.000000)
and
movej([-1.3615, -2.0244, -1.5691, -2.6895, 0.2092, 1.2197],a=5.000000,v=5.000000,r=0.050000)
movel([-1.2437, -1.9458, -2.0485, -2.2887, 0.0128, 1.2197],a=5.000000,v=5.000000,r=0.050000)
To do so, I have created the following code that almost does the job.
The issue I am experiencing is that it stops or doesn't store everything in the TCP_lines and joint_lines strings.
I have also tried using list with the same result.
with open("SRP_Assemble all BUSH_BEARING onto HPC_FRONT_BOTTOM.script", "r") as f:
lines = f.readlines()
TCP_lines = ""
joint_lines = ""
for line in lines:
if line.startswith(("movej(p" ,"movel(p")):
TCP_lines += line
elif line.startswith(("movej(" ,"movel(")):
joint_lines += line
print (TCP_lines)
print (joint_lines)
EDIT:
Does the console have a limited space to print results?
So it only prints the first couple of rows?
I used a redirector to print the results in a text box and it looks fine...

I tried to simulate what you are doing. The below outputs the correct lines. What output do you get?
lines = ["movej(p[-0.2875, -0.4, 0.05, -0.0004, 3.1415, 0.0002],a=5.000000,v=5.000000,r=0.000000)",
"movej([-1.3615, -2.0244, -1.5691, -2.6895, 0.2092, 1.2197],a=5.000000,v=5.000000,r=0.050000)",
"movel(p[-0.2875, -0.375, 0.0, -0.0004, 3.1415, 0.0002],a=0.500000,v=0.300000,r=0.000000)",
"movel([-1.2437, -1.9458, -2.0485, -2.2887, 0.0128, 1.2197],a=5.000000,v=5.000000,r=0.050000)",]
TCP_lines = ""
joint_lines = ""
for line in lines:
if line.startswith(("movej(p" ,"movel(p")):
TCP_lines += line
elif line.startswith(("movej(" ,"movel(")):
joint_lines += line
print (TCP_lines)
print (joint_lines)
OUT:
movej(p[-0.2875, -0.4, 0.05, -0.0004, 3.1415,
0.0002],a=5.000000,v=5.000000,r=0.000000)movel(p[-0.2875, -0.375, 0.0, -0.0004, 3.1415, 0.0002],a=0.500000,v=0.300000,r=0.000000)
movej([-1.3615, -2.0244, -1.5691, -2.6895, 0.2092,
1.2197],a=5.000000,v=5.000000,r=0.050000)movel([-1.2437, -1.9458, -2.0485, -2.2887, 0.0128, 1.2197],a=5.000000,v=5.000000,r=0.050000)

Move the print statements inside the context manager and add a debugging line, Like so:
with open("SRP_Assemble all BUSH_BEARING onto HPC_FRONT_BOTTOM.script", "r") as f:
lines = f.readlines()
TCP_lines = ""
joint_lines = ""
for line in lines:
if line.startswith(("movej(p", "movel(p")):
TCP_lines += line
elif line.startswith(("movej(", "movel(")):
joint_lines += line
else:
print(line) # Check all other lines if they failed to pass the check
print(TCP_lines)
print(joint_lines)
This outputs:
movej(p[-0.2875, -0.4, 0.05, -0.0004, 3.1415, 0.0002],a=5.000000,v=5.000000,r=0.000000)
movel(p[-0.2875, -0.375, 0.0, -0.0004, 3.1415, 0.0002],a=0.500000,v=0.300000,r=0.000000)
movej([-1.3615, -2.0244, -1.5691, -2.6895, 0.2092, 1.2197],a=5.000000,v=5.000000,r=0.050000)
movel([-1.2437, -1.9458, -2.0485, -2.2887, 0.0128, 1.2197],a=5.000000,v=5.000000,r=0.050000)
...

Max of values from 3 columns in a dataframe?

I would like to calculate the maximum of 3 columns value.
import pandas as pd
import pandas_datareader.data as web
data = web.DataReader("^NSEI" , data_source="yahoo",start = "1/4/2016",end ="6/4/2018")
data=pd.DataFrame(data)
data["High-Low"] = data["High"] - data["Low"]
data["Close-low"] = abs(data["Close"].shift(1) - data["Low"])
data["Close-High"] = abs(data["Close"].shift(1) - data["High"])
data["True_Range"] = data[[data["High-Low"], data["Close-low"],data["Close-High"]]].max(axis=1)
In data["True _Range"] column, I want to take the maximum of value in data["High-low"],data["Close-low"] and data["Close-high"] columns. On this, it is giving a name error.
KeyError: "None of [Index([(156.44970703125, 67.9501953125, 79.75, 118.35009765625, 53.05029296875, 110.75, 100.5, 165.150390625, 161.0, 139.2001953125, 127.25, 98.60009765625, 229.39990234375, 148.7001953125, 105.7998046875, 65.94970703125, 58.19970703125, 59.25, 172.85009765625, 59.2001953125, 148.25, 69.10009765625, 91.099609375, 96.5, 149.349609375, 48.30029296875, 94.10009765625, 248.69970703125, 165.7998046875, 126.0, 166.94970703125, 163.05029296875, 87.25, 80.89990234375, 51.69970703125, 151.0, 81.0498046875, 72.80029296875, 67.7998046875, 268.80029296875, 200.39990234375, 72.2001953125, 77.900390625, 61.7998046875, 85.0, 114.7001953125, 99.7001953125, 83.35009765625, 68.650390625, 92.400390625, 102.85009765625, 105.89990234375, 95.7001953125, 95.849609375, 84.400390625, 56.25, 161.69970703125, 70.64990234375, 98.5, 75.60009765625, 74.0498046875, 60.05029296875, 147.64990234375, 46.89990234375, 94.89990234375, 42.64990234375, 161.94970703125, 54.0498046875, 92.599609375, 77.85009765625, 72.85009765625, 94.35009765625, 50.0, 84.0, 151.9501953125, 50.4501953125, 157.5498046875, 100.349609375, 52.5, 155.10009765625, 51.75, 70.69970703125, 60.5498046875, 120.10009765625, 59.19970703125, 112.2001953125, 66.39990234375, 96.7998046875, 101.75, 60.39990234375, 71.2998046875, 109.400390625, 76.64990234375, 98.39990234375, 45.75, 131.900390625, 134.5, 87.150390625, 49.2001953125, 79.2998046875, ...), (nan, 28.0498046875, 63.44970703125, 184.39990234375, 12.75, 107.0, 76.05029296875, 84.5, 118.60009765625, 109.5, 101.39990234375, 13.14990234375, 193.60009765625, 59.2998046875, 50.80029296875, 1.25, 16.44970703125, 28.14990234375, 21.85009765625, 22.2998046875, 127.900390625, 105.25, 4.150390625, 2.64990234375, 125.89990234375, 112.10009765625, 120.4501953125, 255.75, 107.35009765625, 75.849609375, 125.25, 87.60009765625, 19.39990234375, 45.7998046875, 10.0498046875, 143.849609375, 99.7998046875, 57.30029296875, 14.5, 203.9501953125, 48.05029296875, 85.85009765625, 37.19970703125, 31.5, 43.2001953125, 61.0, 84.39990234375, 25.5498046875, 4.849609375, 85.9501953125, 55.4501953125, 19.35009765625, 5.35009765625, 13.35009765625, 60.4501953125, 44.2998046875, 128.7998046875, 32.85009765625, 46.4501953125, 33.2001953125, 72.2998046875, 8.64990234375, 170.14990234375, 11.4501953125, 78.5, 19.75, 38.35009765625, 8.0498046875, 63.25, 7.7001953125, 37.150390625, 30.64990234375, 38.69970703125, 72.2998046875, 32.5, 22.10009765625, 145.44970703125, 58.5498046875, 72.5, 70.75, 49.75, 0.30029296875, 57.14990234375, 20.099609375, 28.349609375, 106.89990234375, 0.7998046875, 116.19970703125, 42.75, 18.9501953125, 80.0, 103.35009765625, 47.64990234375, 27.5, 15.25, 60.44970703125, 13.60009765625, 7.39990234375, 5.85009765625, 44.2001953125, ...)], dtype='object')] are in the [columns]"
​

Try:
data["True_Range"] = data[["High-Low","Close-low","Close-High"]].max(axis=1)
Instead of:
data["True_Range"] = data[[data["High-Low"], data["Close-low"],data["Close-High"]]].max(axis=1)

Your error signifies that you have input the data instead of the nested columnnames into your view of the dataframe, which produced the error.
Since the data from the columns themselves are not in the columnnames - you received the error.
One way of avoiding the error is:
data["True_Range"] = data[["High-Low", "Close-low", "Close-High"]].max(axis=1)

Categories

Resources