I would like to insert data to a new table where the AmountSpent in tblFinance for CategoryID'?' is < the budget in tblCategory which is linked to the ID
e.g.
If AmountSpent = 1000, CategoryID = 1 CategoryBudget = 900, insert the amountspent into tblExpRptOver and if it's less than the budget, insert to tblExpRptUnder.
I've tried what I thought should work but it gives me this error:
line 613, in test
'CategoryID = ? AND AmountSpent < ? ', (category_input, budget_input, ))
sqlite3.InterfaceError: Error binding parameter 1 - probably unsupported type.
The code I'm trying to execute is:
view_avg = input("Would you like to view the averages for under/over expenses for a specific category? Y/N \n")
if view_avg == "Y":
category_input = input("Please input the CategoryID for the category you'd like to view the averages for:"
"\n")
budget_input = ('SELECT CategoryBudget FROM tblCategory WHERE CategoryID = ?', (category_input, ))
c.execute('INSERT INTO tblExpOver SELECT CategoryID, AmountSpent FROM tblExpRptMonth WHERE '
'CategoryID = ? AND AmountSpent < ? ', (category_input, budget_input, ))
c.execute('INSERT INTO tblExpUnder SELECT CategoryID, AmountSpent FROM tblExpRptMonth WHERE '
'CategoryID = ? AND AmountSpent > ? ', (category_input, budget_input,))
df_avg_over = pd.read_sql_query("SELECT * FROM tblExpOver", conn)
df_avg_under = pd.read_sql_query("SELECT * FROM tblExpUnder", conn)
with PdfPages('GraphByMonth.pdf') as pdf:
firstPage = plt.figure(figsize=(11.69, 8.27))
firstPage.clf()
txt = 'Average Expense on Over Expenses for Category ', category_input, ':',\
df_avg_over["AmountSpent"].mean()
txt1 = 'Average Expense on Under Expenses for Category ', category_input, ':', \
df_avg_under["AmountSpent"].mean()
firstPage.text(0.5, 0.5, txt, txt1, transform=firstPage.transFigure, size=24, ha="center")
pdf.savefig()
plt.close()
fig = plt.figure(figsize=(11.69, 8.27))
df2.plot(kind='bar')
plt.title('Expense Graph by Month')
txt = 'Month Expense Graph'
plt.text(0.05, 0.95, txt, transform=fig.transFigure, size=24)
plt.xlabel("Expense Number")
plt.ylabel("Amount Spent")
pdf.savefig()
plt.close()
menu()
Any help is appreciated!
EDIT: CategoryBudget is stored in a different table for category information - tblCategory.
You're never executing the budget_input query. You can't use a placeholder for a SELECT query.
Instead of doing a separate query, you should join the tblCategory table in the INSERT ... SELECT query.
c.execute('''INSERT INTO tblExpOver
SELECT r.CategoryID, r.AmountSpent
FROM tblExpRptMonth AS r
JOIN tblCategory AS c ON r.CategoryId = c.CategoryId AND r.AmountSpent < c.CategoryBudget
WHERE r.CategoryID = ?''', (category_input,))
c.execute('''INSERT INTO tblExpUnder
SELECT r.CategoryID, r.AmountSpent
FROM tblExpRptMonth AS r
JOIN tblCategory AS c ON r.CategoryId = c.CategoryId AND r.AmountSpent > c.CategoryBudget
WHERE r.CategoryID = ?''', (category_input,))
Note that you never insert anything for rows that are exactly equal to the budgeted amount. Unless that's intentional, you should change either < or > to <= or >=, depending on which category you want them to be in.
Also, it seems like you may have < and > backwards, if the Over and Under table names mean that the spending is more and less than the budgeted amount.
Related
i try to put some orders at a specific price.
For example i would like to put 20 dollars to buy some ETHUSDT at 800 usdt but it gives me this error:
binance.exceptions.BinanceAPIException: APIError(code=-1106): Parameter 'quoteOrderQty' sent when not required.
there is my call function:
buy_order = client.create_order(
symbol = "ETHUSDT",
price = 800,
quoteOrderQty = 25,
side = client.SIDE_BUY,
type = client.ORDER_TYPE_LIMIT,
timeInForce = client.TIME_IN_FORCE_GTC)
but i dont have any error when i put this :
buy_order = client.create_test_order(symbol="ETHUSDT", side='BUY', type='MARKET', quoteOrderQty=10)
to be honnest actually i do that:
def putOrderBuy_at_price(self, symbol, amount, price):
monney_price = self.client.get_symbol_ticker(symbol=symbol)
# Calculate how much ETH $200 can buy
print(monney_price['price'])
i = 10
while i != 0:
try:
buy_quantity = round(amount / float(price), i)
print("-----------", buy_quantity)
#ETHUSDT
buy_order = self.client.create_order(
symbol = symbol,
price = price,
quantity = buy_quantity,
side = self.client.SIDE_BUY,
type = self.client.ORDER_TYPE_LIMIT,
timeInForce = self.client.TIME_IN_FORCE_GTC)
break
except Exception:
print(i)
i -= 1
And i think there is a better way for that.
Thanks for yours answers
quoteOrderQty is only for MARKET order.
I have this query that I use on a weekly basis to create a report using pd.read_sql. I want to be able to update the case statement of store, update the Date Add, and store IN at the end of the statement without having to manually change the store numbers and the dates. Is there any way that I can edit the query to make the updates?
This is the query
dataframe = pd.read_sql("""
SELECT Top(10)
CAST( Store as VARCHAR) + 'þ' as Store,
CONVERT( VARCHAR, Tran_Dt2, 101 ) + 'þ' as Tran_Dt,
CONVERT(char(5), Start_Time, 108) + 'þ' as Start_Time,
[Count]
FROM
(
SELECT
CASE
WHEN [Store] = 313 THEN 3174
WHEN [Store] = 126 THEN 3191
END AS Store
, DATEADD (YEAR, +2, DATEADD(DAY, +4, Tran_Dt2)) as Tran_Dt2
,[Start_Time]
,[Count]
,Store as Sister_Store
FROM
(
SELECT
Store,
CONVERT(datetime, Tran_Dt) as Tran_Dt2,
Start_Time,
Count
FROM [VolumeDrivers].[dbo].[SALES_DRIVERS_ITC_Signup_65wks]
WHERE CONVERT(datetime, Tran_Dt) between CONVERT(datetime,'2/8/2019') and CONVERT(datetime,'3/15/2019')
AND
Store IN (313, 126)
--Single Store: Store = Store #
) AS A
) AS B
ORDER BY Tran_Dt2, Store
""", con = conn)
I would want to be able to do something like declare a variable and have it populate in the code such as something like:
oldstore1 = 313
newstore1 = 3174
oldstore2 = 126
newstore2 = 3191
daframe = pd.ready_sql("""...
...
SELECT
CASE
WHEN [Store] = oldstore1 THEN newstore1
WHEN [Store] = oldstore2 THEN newstore2
...
UPDATE----
I am currently at this point and had the query working until my kernel restarted and I lost my code. Any tips on why it isn't working anymore?
#Declare variables for queries
old_store1 = 313
new_store1 = 3157
old_store2 = 126
new_store2 = 3196
datefrom = '2/8/2019'
dateto = '3/15/2019'
yearadd = '+2'
dayadd = '+4'
ITC = pd.read_sql("""SELECT
CAST( Store as VARCHAR) + 'þ' as Store,
CONVERT( VARCHAR, Tran_Dt2, 101 ) + 'þ' as Tran_Dt,
CONVERT(char(5), Start_Time, 108) + 'þ' as Start_Time,
[Count]
FROM
(
SELECT
CASE
WHEN [Store] = {old_store1} THEN {new_store1}
WHEN [Store] = {old_store2} THEN {new_store2}
END AS Store
, DATEADD (YEAR, {yearadd}, DATEADD(DAY, {dayadd}, Tran_Dt2)) as Tran_Dt2
,[Start_Time]
,[Count]
,Store as Sister_Store
FROM
(
SELECT
Store,
CONVERT(datetime, Tran_Dt) as Tran_Dt2,
Start_Time,
Count
FROM [VolumeDrivers].[dbo].[SALES_DRIVERS_ITC_Signup_65wks]
WHERE CONVERT(datetime, Tran_Dt) between CONVERT(datetime,{datefrom}) and CONVERT(datetime,{dateto})
AND
Store IN ({old_store1}, {old_store2})
--Single Store: Store = Store #
) AS A
) AS B
ORDER BY Tran_Dt2, Store
""", con = conn)
Was able to figure out why it wasn't working. I guess python 3 and beyond has a built in function that allows you to place "f" in front of the query and will let you pass the variables you created. I know this isn't the most secure way of executing the script but I'll look into creating a for loop in the future that will allow it to be more secure. Thanks for all the help!
#Declare variables for queries
old_store1 = 313
new_store1 = 3157
old_store2 = 126
new_store2 = 3196
datefrom = '2/15/2019'
dateto = '3/22/2019'
yearadd = '+2'
dayadd = '+4'
ITC = pd.read_sql(f"""SELECT
CAST( Store as VARCHAR) + 'þ' as Store,
CONVERT( VARCHAR, Tran_Dt2, 101 ) + 'þ' as Tran_Dt,
CONVERT(char(5), Start_Time, 108) + 'þ' as Start_Time,
[Count]
FROM
(
SELECT
CASE
WHEN [Store] = {old_store1} THEN {new_store1}
WHEN [Store] = {old_store2} THEN {new_store2}
.....
#run code and verify it works
Sales_Drivers_ITCSignup = pd.read_sql(ITCQuery, con = conn, index_col='Store')
Sales_Drivers_ITCSignup.head()
I've got this netcdf of weather data (one of thousands that require postgresql ingestion). I'm currently capable of inserting each band into a postgis-enabled table at a rate of about 20-23 seconds per band. (for monthly data, there is also daily data that i have yet to test.)
I've heard of different ways of speeding this up using COPY FROM, removing the gid, using ssds, etc... but I'm new to python and have no idea how to store the netcdf data to something I could use COPY FROM or what the best route might be.
If anyone has any other ideas on how to speed this up, please share!
Here is the ingestion script
import netCDF4, psycopg2, time
# Establish connection
db1 = psycopg2.connect("host=localhost dbname=postgis_test user=********** password=********")
cur = db1.cursor()
# Create Table in postgis
print(str(time.ctime()) + " CREATING TABLE")
try:
cur.execute("DROP TABLE IF EXISTS table_name;")
db1.commit()
cur.execute(
"CREATE TABLE table_name (gid serial PRIMARY KEY not null, thedate DATE, thepoint geometry, lon decimal, lat decimal, thevalue decimal);")
db1.commit()
print("TABLE CREATED")
except:
print(psycopg2.DatabaseError)
print("TABLE CREATION FAILED")
rawvalue_nc_file = 'netcdf_file.nc'
nc = netCDF4.Dataset(rawvalue_nc_file, mode='r')
nc.variables.keys()
lat = nc.variables['lat'][:]
lon = nc.variables['lon'][:]
time_var = nc.variables['time']
dtime = netCDF4.num2date(time_var[:], time_var.units)
newtime = [fdate.strftime('%Y-%m-%d') for fdate in dtime]
rawvalue = nc.variables['tx_max'][:]
lathash = {}
lonhash = {}
entry1 = 0
entry2 = 0
lattemp = nc.variables['lat'][:].tolist()
for entry1 in range(lat.size):
lathash[entry1] = lattemp[entry1]
lontemp = nc.variables['lon'][:].tolist()
for entry2 in range(lon.size):
lonhash[entry2] = lontemp[entry2]
for timestep in range(dtime.size):
print(str(time.ctime()) + " " + str(timestep + 1) + "/180")
for _lon in range(lon.size):
for _lat in range(lat.size):
latitude = round(lathash[_lat], 6)
longitude = round(lonhash[_lon], 6)
thedate = newtime[timestep]
thevalue = round(float(rawvalue.data[timestep, _lat, _lon] - 273.15), 3)
if (thevalue > -100):
cur.execute("INSERT INTO table_name (thedate, thepoint, thevalue) VALUES (%s, ST_MakePoint(%s,%s,0), %s)",(thedate, longitude, latitude, thevalue))
db1.commit()
cur.close()
db1.close()
print(" Done!")
If you're certain most of the time is spent in PostgreSQL, and not in any other code of your own, you may want to look at the fast execution helpers, namely cur.execute_values() in your case.
Also, you may want to make sure you're in a transaction, so the database doesn't fall back to an autocommit mode. ("If you do not issue a BEGIN command, then each individual statement has an implicit BEGIN and (if successful) COMMIT wrapped around it.")
Something like this could do the trick -- not tested though.
for timestep in range(dtime.size):
print(str(time.ctime()) + " " + str(timestep + 1) + "/180")
values = []
cur.execute("BEGIN")
for _lon in range(lon.size):
for _lat in range(lat.size):
latitude = round(lathash[_lat], 6)
longitude = round(lonhash[_lon], 6)
thedate = newtime[timestep]
thevalue = round(
float(rawvalue.data[timestep, _lat, _lon] - 273.15), 3
)
if thevalue > -100:
values.append((thedate, longitude, latitude, thevalue))
psycopg2.extras.execute_values(
cur,
"INSERT INTO table_name (thedate, thepoint, thevalue) VALUES %s",
values,
template="(%s, ST_MakePoint(%s,%s,0), %s)"
)
db1.commit()
I have the following code:
def create_table():
c.execute('CREATE TABLE IF NOT EXISTS TEST(SITE TEXT, SPORT TEXT, TOURNAMENT TEXT, TEAM_1 TEXT, TEAM_2 TEXT, DOUBLE_CHANCE_1X TEXT, DOUBLE_CHANCE_X2 TEXT, DOUBLE_CHANCE_12 TEXT, DRAW_1 TEXT, DRAW_2 TEXT DATE_ODDS TEXT, TIME_ODDS TEXT)')
create_table()
def data_entry():
c.execute("INSERT INTO TEST(SITE, SPORT, TOURNAMENT, TEAM_1, TEAM_2, DOUBLE_CHANCE_1X, DOUBLE_CHANCE_X2, DOUBLE_CHANCE_12, DATE_ODDS, TIME_ODDS) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(Site, sport.strip(), tournament.strip(), team_1.strip(), team_2.strip(), x_odd.strip(), y_odd.strip(), z_odd.strip(), Date_odds, Time_odds))
conn.commit()
def double_chance():
c.execute("UPDATE TEST SET DOUBLE_CHANCE_1X = x_odd, DOUBLE_CHANCE_X2 = y_odd, DOUBLE_CHANCE_12 = z_odd WHERE TOURNAMENT = tournament and TEAM_1 = team_1 and TEAM_2 = team_2 and DATE_ODDS = Date_odds and TIME_ODDS = Time_odds")
conn.commit()
driver.get(link)
Date_odds = time.strftime('%Y-%m-%d')
Time_odds = time.strftime('%H:%M')
sport = (driver.find_element_by_xpath(".//*[#id='breadcrumb']/li[2]/a")).text #example Footbal
tournament = (driver.find_element_by_xpath(".//*[#id='breadcrumb']/li[4]/a")).text #example Premier League
try:
div = (driver.find_element_by_xpath(".//*[#id='breadcrumb']/li[5]/a")).text #to find any division if exists
except NoSuchElementException:
div = ""
market = driver.find_element_by_xpath(".//*[contains(#id,'ip_market_name_')]")
market_name = market.text
market_num = market.get_attribute('id')[-9:]
print market_num
team_1 = (driver.find_element_by_xpath(".//*[#id='ip_marketBody" + market_num + "']/tr/td[1]//*[contains(#id,'name')]")).text
team_2 = (driver.find_element_by_xpath(".//*[#id='ip_marketBody" + market_num + "']/tr/td[3]//*[contains(#id,'name')]")).text
print sport, tournament, market_name, team_1, team_2
data_entry() #first SQL call
for ip in driver.find_elements_by_xpath(".//*[contains(#id,'ip_market3')]"):
num = ip.get_attribute('id')[-9:]
type = (driver.find_element_by_xpath(".//*[contains(#id,'ip_market_name_" + num + "')]")).text
if type == 'Double Chance':
print type
print num
x_odd = (driver.find_element_by_xpath(".//*[#id='ip_market" + num + "']/table/tbody/tr/td[1]//*[contains(#id,'price')]")).text
y_odd = (driver.find_element_by_xpath(".//*[#id='ip_market" + num + "']/table/tbody/tr/td[2]//*[contains(#id,'price')]")).text
z_odd = (driver.find_element_by_xpath(".//*[#id='ip_market" + num + "']/table/tbody/tr/td[3]//*[contains(#id,'price')]")).text
print x_odd, y_odd, z_odd
double_chance() #second SQL call
c.close()
conn.close()
Update:
Based on the answer below I updated the code, but I can't make it work.
When I run it, I get the following error:
sqlite3.OperationalError: no such column: x_odd
What should I do?
Update 2:
I found the solution:
I created an unique ID in order to be able to select exactly the row I want when I run the second SQL query. In this case it doesn't modify any other rows:
def double_chance():
c.execute("UPDATE TEST SET DOUBLE_CHANCE_1X = (?), DOUBLE_CHANCE_X2 = (?), DOUBLE_CHANCE_12 = (?) WHERE ID = (?)",(x_odd, y_odd, z_odd, ID_unique))
conn.commit()
Now it works perfectly.
Use the UPDATE statement to update columns in an existing row.
UPDATE TEST SET DRAW_1=value1,DRAW_2=value2 WHERE column3=value3;
If data_entry(1) is always called first, then change the statement in data_entry_2() to UPDATE. If not you will need to check if the row exists in both cases and INSERT or UPDATE accordingly.
I'm having trouble inserting data into my table. I have a list of stocks that I pass to the function getStockData.
I use a for loop to iterate through the list and get the data for each ticker symbol. At the end I put all the information into a dictionary. My final step is to insert the data into a table. I've been unsuccessful at inserting the data in the dictionary into my table.
def getStockData(x):
nowdate = raw_input("What Is Todays Date?: ")
print "Todays list has %d stocks on it\n" % len(x)
for stock in x:
stockPrice = ystockquote.get_price(stock)
stockPriceChange = ystockquote.get_change(stock)
originalPrice = float(stockPrice) + (float(stockPriceChange) * -1)
changePercentage = (float(stockPriceChange) / originalPrice) * 100
stockDict = {'Date': nowdate, 'Ticker Symbol': stock, 'Closing Price': stockPrice,
'Price Change': stockPriceChange, 'Percentage Changed': changePercentage}
conn = db.connect('stocks.db')
cursor = conn.cursor()
cursor.execute('insert into losers values (?, ?, ?, ?, ?)', (stockDict['Date'], stockDict['Ticker Symbol'], stockDict['Price Change'],
stockDict['Percentage Changed'], stockDict['Closing Price']) )
conn.close()
I think you forget to commit your data to your DB before close.
Try
conn.commit()