I have two functions which both create a diagramm. But when I run those 2 functions, in the second one is the data which should be in the first one. Here are the diagramms:
This diagramm shows the temerature
And this one should only show the humidity data. Not the humidity and the temperature data.
Here is my source code:
from pandas import DataFrame
import sqlite3
import matplotlib.pyplot as plt
import pandas as pd
from datetime import date, datetime
datum = str(date.today())
date = [datum]
con = sqlite3.connect("/home/pi/test2.db")
sql = "SELECT * from data4 WHERE date in (?)"
df3 = pd.read_sql_query(sql,con, params=[datum])
def daily_hum():
df3 = pd.read_sql_query(sql,con, params=[datum])
df3['datetime'] = pd.to_datetime((df3.date + ' ' + df3.time))
df3.groupby([df3.datetime]).hum.mean().plot()
plt.savefig('/home/pi/flask/static/daily_hum.jpg')
def daily_temp1():
df4 = pd.read_sql_query(sql,con, params=[datum])
df4['datetime'] = pd.to_datetime((df4.date + ' ' + df4.time))
df4.groupby([df4.datetime]).temp.mean().plot()
plt.savefig('/home/pi/flask/static/daily_temp.jpg')
daily_temp()
daily_hum()
The database/ the DataFrame looks like this:
id,hum,temp,zeit,date
721,60,21,11:04:23,2020-06-21
722,64,22,11:04:24,2020-06-21
723,68,22,11:04:27,2020-06-21
724,70,22,11:07:20,2020-06-21
725,63,22,11:08:20,2020-06-21
726,63,22,11:09:21,2020-06-21
727,63,22,11:10:22,2020-06-21
728,63,22,11:11:22,2020-06-21
729,69,22,11:12:24,2020-06-21
730,64,22,11:13:29,2020-06-21
731,70,22,11:14:32,2020-06-21
732,64,22,11:15:33,2020-06-21
733,64,22,11:16:34,2020-06-21
734,64,22,11:17:34,2020-06-21
735,64,22,11:18:35,2020-06-21
736,64,22,11:19:35,2020-06-21
737,64,22,11:20:36,2020-06-21
738,64,22,11:21:37,2020-06-21
739,64,22,11:22:37,2020-06-21
740,64,22,11:23:38,2020-06-21
741,65,22,11:24:38,2020-06-21
742,65,22,11:25:39,2020-06-21
743,65,22,11:26:40,2020-06-21
744,65,22,11:27:40,2020-06-21
I hope you can help me
You could try this. Matplotlib needs to know, if you want a new figure for each plot or not.
from pandas import DataFrame
import sqlite3
import matplotlib.pyplot as plt
import pandas as pd
from datetime import date, datetime
datum = str(date.today())
date = [datum]
con = sqlite3.connect("/home/pi/test2.db")
sql = "SELECT * from data4 WHERE date in (?)"
df3 = pd.read_sql_query(sql,con, params=[datum])
df3['datetime'] = pd.to_datetime((df3.date + ' ' + df3.time))
# new figure
fig, ax = plt.subplots()
# Some figure modifying code
fig.suptitle('Titel of Figure')
ax.set_xlabel('X-Label')
ax.set_ylabel('Y-Label')
df3.groupby([df3.datetime]).hum.mean().plot(ax=ax)
plt.savefig('/home/pi/flask/static/daily_hum.jpg')
# new figure
fig, ax = plt.subplots()
# Some figure modifying code
fig.suptitle('Titel of Figure')
ax.set_xlabel('X-Label')
ax.set_ylabel('Y-Label')
df3.groupby([df3.datetime]).temp.mean().plot(ax=ax)
plt.savefig('/home/pi/flask/static/daily_temp.jpg')
I am trying to plot a real-time data getting loaded in dataframe. But the attempts have led to printing of multiple blank graph frames in response to dynamic data feed, instead of plotting the data in single frame of graph.
I am implementing a solution to perform sentiment analysis on live twitter stream. I am able to stream the tweets, put them into a DataFrame and apply the required sentiment analysis algorithm on them one by one. I created a column in the DataFrame which holds the compound value generated by that algorithm for an individual tweet.
This DataFrame is getting dynamically updated as the tweets stream and the intent is to plot this real time updated compound value against time.
I have tried plotting the graph as per mentioned advises of using plt.ion(), plt.draw() instead of plt.show() functions etc. But instead of plotting one frame which gets updated with the values, the program starts printing multiple frames one after another as the data gets updated in the DataFrame.
import pandas as pd
import csv
from bs4 import BeautifulSoup
import re
import tweepy
import ast
from pytz import timezone
from datetime import datetime
import matplotlib.pyplot as plt
import time
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
import json
from textblob import TextBlob
from unidecode import unidecode
import nltk
from nltk.sentiment.vader import SentimentIntensityAnalyzer
ckey= '#######'
csecret= '#######'
atoken= '#########'
asecret= '#########'
class listener(StreamListener):
def on_data(self,data):
try:
global df
data=json.loads(data)
time = data["created_at"]
tweet = unidecode(data["text"])
tweet1 = BeautifulSoup(tweet,"lxml").get_text()
df = pd.DataFrame(columns = ['time','tweet'])
df['time'] = pd.Series(time)
df['tweet'] = pd.Series(tweet1)
def convert_time(time):
eastern = timezone('US/Eastern')
utc = timezone('UTC')
created_at = datetime.strptime(time, '%a %b %d %H:%M:%S %z %Y')
est_created_at = created_at.astimezone(eastern)
return (est_created_at)
df['time'] = df['time'].apply(convert_time)
def hour(time):
hour = pd.DatetimeIndex(time).hour
return hour
df['hour'] = df['time'].apply(hour)
def sentiment_analysis(tweet):
sid = SentimentIntensityAnalyzer()
return (sid.polarity_scores(tweet)['compound'])
df['compound'] = df['tweet'].apply(sentiment_analysis)
#print(df['compound'])
#print(df['time'])
plt.ion()
fig, ax = plt.subplots()
df.plot(y=df'compound', ax=ax)
ax.clear()
ax.axis([ 0, 24, -5,5])
plt.xlabel('Time')
plt.ylabel('Sentiment')
plt.draw()
plt.pause(0.2)
except KeyError as e:
print(str(e))
return (True)
auth=OAuthHandler(ckey,csecret)
auth.set_access_token(atoken,asecret)
twitterStream = Stream(auth, listener())
twitterStream.filter(track=["######"])
Expected Result - One frame of graph getting updated and plotting the real-time data.
Actual Result - Multiple blank graphs
I apologize if i have missed on any information/point.
I am getting this error only sometimes when I try to plot a bar chart with the value_counts() of the multinomialnb_label values in the dataframe.
Other times it is fine. I can't figure out how to fix the error.
It is part of a flask application and everything works fine until it hits the line:
my_plot = multinomialnb_count.plot(kind='bar', legend=None)
Here is the multinomialnb_count details:
[1395 rows x 15 columns]
bug report 721
question 492
praise 112
noise 49
other feedback 21
Name: multinomialnb_label, dtype: int64
Any ideas?
import matplotlib
matplotlib.use('Agg')
import pandas as pd
import numpy as np
import pymysql.cursors
import sqlalchemy
import datetime
import sys
import re
from wordcloud import WordCloud, STOPWORDS
import preprocessor as p
import nltk
from nltk.tokenize import sent_tokenize, word_tokenize
from nltk.corpus import stopwords
from nltk.collocations import *
from nltk.stem import WordNetLemmatizer
import matplotlib.pyplot as plt, mpld3
import matplotlib.dates as dates
import collections
import contractions
from matplotlib import rcParams
def make_charts(username):
rcParams.update({'figure.autolayout': True})
username = username
new_db_name = username + "_predicted_tweets"
# Get tweets from MYSQL database
dbServerName = "localhost"
dbUser = "root"
dbPassword = "woodycool123"
dbName = "azure_support_tweets"
engine = sqlalchemy.create_engine('mysql+pymysql://root:woodycool123#localhost:3306/azure_support_tweets')
df = pd.read_sql_table(new_db_name, engine)
data = pd.DataFrame(df)
multinomialnb_count = data['multinomialnb_label'].value_counts()
my_plot = multinomialnb_count.plot(kind='bar', legend=None)
Error:
ValueError: view limit minimum -0.5 is less than 1 and is an invalid Matplotlib date value. This often happens if you pass a non-datetime value to an axis that has datetime units
Solved it! It was as simple as just closing the plt at the end of the function with plt.close().
It was because the plt was configured for a time series further down and then when the script was rerun it was using that. I assumed it would clear all the values but you have to close that plt to then use a new one! Thanks
Continuation from: Getting date/time and data out of csv into matplotlib
import datetime
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import matplotlib.cbook as cbook
import pandas
import StringIO
f = open(r'clean data.csv')
#Make a string buffer and read in the CSV file while stripping \x00's
output = StringIO.StringIO()
for x in f.readlines():
output.write(x.replace('\x00',''))
#Close out input file
f.close()
#Set position back to start for pandas to read
output.seek(0)
df = pandas.read_csv(output, skiprows=38, parse_dates=['Time'], index_col="Time")
fig, ax = plt.subplots()
ax.plot(df.index,df['108 <Air> (C)'])
#ax.xaxis.set_major_locator(mdates.DayLocator())
#ax.format_xdata = mdates.DateFormatter('%Y-%m-%d')
#fig.autofmt_xdate()
plt.show()
So I can actually plot this data with this current code, the problem occurs when I try to continue on with this example: https://matplotlib.org/gallery/api/date.html#sphx-glr-gallery-api-date-py
If you uncomment out
ax.xaxis.set_major_locator(mdates.DayLocator())
I get
OverflowError: Python int too large to convert to C long
Whats up with that?
Here is some input data: https://pastebin.com/SSZyaSJ4
I am using the Yahoo Finance Library in Python to pull data of a stock.
import yahoo_finance
ticker = 'GLD'
begdate = '2014-11-11'
enddate = '2016-11-11'
data = yahoo_finance.Share('GLD')
data1 = data.get_historical(begdate,enddate)
gld_df = pd.DataFrame(data1)
date_df = (list(gld_df["Date"]))
adj_close_df = list(gld_df["Adj_Close"])
print(adj_close_df)
plt.plot(adj_close_df,date_df)
I would like to plot this Adjusted Close Price on Y-Axis and the corresponding Dates on the X Axis, but my above code is giving an error when I try to do that.
I am using Python 3.x, Anaconda
You could generate the list as below:
l = [ x['Close'] for x in data1]
And the plot:
import matplotlib.pyplot as plt
plt.plot(l)
plt.show()
I got it.
import yahoo_finance
from pylab import *
import numpy as np
import scipy.signal as sc
import matplotlib.pyplot as plt
import pandas as pd
import datetime as dt
ticker = 'GLD'
begdate = '2014-11-11'
enddate = '2016-11-11'
data = yahoo_finance.Share('GLD')
data1 = data.get_historical(begdate,enddate)
gld_df = pd.DataFrame(data1)
date_df = pd.to_datetime((list(gld_df["Date"])))
adj_close_df = list(gld_df["Adj_Close"])
plt.plot(date_df,adj_close_df)