i'm trying to schedule a task every 5 seconds, here what i did:
conn = connect('mydatabase.db')
c = conn.cursor()
c.execute('CREATE TABLE IF NOT EXISTS RSSEntries (entry_id INTEGER PRIMARY KEY AUTOINCREMENT, title , url , date );')
def checkLink(linko):
c.execute("SELECT entry_id FROM RSSEntries WHERE url = ?", (linko,))
datas=c.fetchall()
if len(datas)==0:
return True
else:
return False
def storeData():
data = feedparser.parse("http://www...")
for i in range(len(data['entries'])):
if checkLink(data.entries[i].link) is True:
print "doesn't exist"
c.execute("insert into RSSEntries VALUES\
(NULL,'%s', '%s', '%s')" % (data.entries[i].title,data.entries[i].link, data.feed.updated))
else:
print "exist"
schedule.every(5).seconds.do(storeData)
conn.commit()
but the storeData method is not reachable..
if i run storeData() instead of schedule.every(5).seconds.do(storeData) the code work perfectly, what i'm doing wrong
any suggesting or other ways to do this task are welcome.
I think you are missing the scheduler loop at the end of your script:
while True:
schedule.run_pending()
time.sleep(1)
https://pypi.python.org/pypi/schedule
Related
I need to set some user meta in my wordpress through local python script. Hence I can't use the WP update_user_meta for it - it has to be done manually.
import mysql.connector as mysql
cnx = mysql.connect(host=HOST, database=DATABASE, user=USER, password=PASSWORD)
cursor = cnx.cursor()
get_meta = ("SELECT * FROM `ff_usermeta` WHERE `user_id`= 1 AND (`meta_key`='nickname' OR `meta_key`='info' OR `meta_key`='bg' OR `meta_key`='avatar' OR `meta_key`='profile_updated')")
cursor.execute(get_meta)
meta = cursor.fetchall()
#some processing of the result
cursor.execute(q, (...))
cnx.commit()
cursor.close()
cnx.close()
Now I need to check if the result has meta with each of the keys.
If the key already exists for this user, it needs to run UPDATE for this meta.
If this user still has no meta of this key, it has to INSERT new row.
if(there's no 'nickname' in meta_key on either of 5 or less rows):
q = ("INSERT INTO `ff_usermeta` ...")
else:
q = ("UPDATE `ff_usermeta` ...")
...and 4 more times like that?.. Seems like a good place for a cycle, but I don't really like the idea to make it 5x queues, especially since there might be more fields in the future.
I was thinking along the lines of searching the fetchall result for matches in meta_key, and if found, adding required data to one array, if not - to another. And then just running one update and one insert at the end, assuming both are not empty. If I were to write it in semi-php style, it would look roughly like this:
if(in_array("nickname", meta))
for_update .= "`nickname`='"+data[0]+"', "
else:
fields .= "`nickname`, "
vals .= "'"+data[0]+"', "
if(in_array("bg", meta)):
for_update .= "`bg`='"+data[1]+"', "
else:
fields .= "`bg`, "
vals .= "'"+data[1]+"', "
if(for_update):
update = ("UPDATE `ff_usermeta` SET "+for_update+" WHERE 1")
if(fields):
insert = ("INSERT INTO `ff_usermeta`("+fields+") VALUES ("+vals+")")
But absolutely no clue how to translate it correctly to python. I had to google it up to things like "why dot not working to add one string to another". Any advice? Or perhaps there is a better way? Thanks!
It is not complete, you can not update your rows in that way.
But with this you can start to make your query
The frist select gets exactly 1 row, if the user_id exists.
The user_id doesn't seem the right choice for this, but to get what you can do it is enough.
If the query doesn't have an entry, the it will insert some data you get from anywhere
The update as the insert are in that form wrong as you have to insert 5 new orws or update max 5 rows, but that is more for you to programm
import mysql.connector as mysql
HOST = "localhost"
DATABASE = ""
USER = "root"
PASSWORD = "mypassword"
cnx = mysql.connect(host=HOST, database=DATABASE, user=USER, password=PASSWORD)
cnx = mysql.connect(host=HOST, database=DATABASE, user=USER, password=PASSWORD)
cursor = cnx.cursor()
user_id = 1
get_meta = ("""SELECT umeta_id, user_id , MAX(IF( `meta_key`='nickname', meta_value,'')) AS 'nickname' , MAX(IF( `meta_key`='info', meta_value,'')) AS 'info' , MAX(IF( `meta_key`='bg', meta_value,'')) AS 'bg' , MAX(IF( `meta_key`='avatar', meta_value,''NULL'')) AS 'avatar' , MAX(IF (`meta_key`='profile_updated', meta_value,'')) AS 'profile_updated' FROM `ff_usermeta` WHERE `user_id`= %s GROUP BY umeta_id, user_id:""")
result = cursor.execute(get_meta,(user_id,))
if result > 0:
data = cursor.fetchone()
for_update = "";
#some processing of the result
if not data["nickname"]:
for_update += "`nickname`='"+data["nickname"]+"', "
if not data["bg"]:
for_update += "`bg`='"+data["bg"]+"', "
query = ("UPDATE `ff_usermeta` SET "+for_update+" WHERE user_id = " + user_id)
else:
#here are no data to be gathered as there is no user_id present add new user
nickname = ""
bg= ""
info = ""
avatar = ""
profile_updated = ""
fields= ""
vals = ""
fields += "`nickname`,`info`, `bg`,`avatar`,`profile_updated`"
vals += "'"+nickname+"', "+"'"+info+"', "+"'"+bg+"', "+"'"+avatar+"', "+"'"+profile_updatedfo+"'"
query = ("INSERT INTO `ff_usermeta`("+fields+") VALUES ("+vals+")")
cursor.execute(query)
cnx.commit()
cursor.close()
cnx.close()
I tried my best to adapt the suggestion above, but couldn't figure out how to make it work. Eventually I went another way, and it seems to work somehow, so I'll post the full code in case anyone would find it useful.
What it does: checks the queue in table with validation request, then parses a page (separate function) and updates user profile accodringly.
import mysql.connector as mysql
import time
from datetime import datetime
cnx = mysql.connect(host=HOST, database=DATABASE, user=USER, password=PASSWORD)
while True: #endless loop as a temporary scheduler
cursor = cnx.cursor()
#getting first request in the queue - 0: id, 1: url, 2: parse, 3: status, 4: user, 5: user_page, 6: req_date, 7: action
cursor.execute("SELECT * FROM `ff_qq` WHERE status = 0 LIMIT 1")
row = cursor.fetchone()
if row:
status = 1 #processed
if row[7] == "verify":
get_user = ("SELECT * FROM `ff_users` WHERE ID = %s LIMIT 1")
cursor.execute(get_user, (row[4],))
user = cursor.fetchone() #0 - ID, 5 - user_url, 8 - user_status, 9 - display_name
#separate function that returns data to insert into mysql
udata = verify(row) #0 - nickname, 1 - fb_av, 2 - fb_bg, 3 - fb_info, 4 - owner
ustat = row[1].split("/authors/")
if udata['owned'] or user[8] == ustat[1]:
update_user = ("UPDATE `ff_users` SET user_status = %s, display_name = %s, user_url = %s WHERE ID = %s LIMIT 1")
cursor.execute(update_user, (ustat[1], udata['nickname'], row[1], user[0]))
status = 2 #success
get = ("SELECT `meta_value` FROM `ff_usermeta` WHERE `user_id`= %s AND `meta_key`='ff_capabilities' LIMIT 1")
cursor.execute(get, (row[4],))
rights = cursor.fetchone()
if rights == 'a:1:{s:10:"subscriber";b:1;}':
promote = ("UPDATE `ff_usermeta` SET `meta_value` = 'a:1:{s:6:\"author\";b:1;}' "
"WHERE `user_id` = %s AND `meta_key`='ff_capabilities' LIMIT 1")
cursor.execute(promote, (row[0],))
#list of meta_key values in same order as returned data
ff = ['nickname', 'fb_av', 'fb_bg', 'fb_info']
for x in range(0,3): #goes through each one of the above list
if udata[ff[x]]: #yes this actually works, who would've thought?..
#current meta_key added directly into the string
get = ("SELECT `meta_value` FROM `ff_usermeta` WHERE `user_id`= %s AND `meta_key`='" + ff[x] + "' LIMIT 1")
cursor.execute(get, (row[4],))
meta = cursor.fetchone()
if(meta): #update if it exists, otherwise insert new row
qq = ("UPDATE `ff_usermeta` SET `meta_value` = %s "
"WHERE `user_id` = %s AND `meta_key`='" + ff[x] + "' LIMIT 1")
else:
qq = ("INSERT INTO `ff_usermeta`(`meta_value`, `meta_key`, `user_id`) "
"VALUES ('%s','" + ff[x] + "','%s'")
cursor.execute(qq, (udata[ff[x]], row[0])) #same execute works for both
else:
status = 3 #verification failed
#update queue to reflect its status
update = ("UPDATE `ff_qq` SET status = %s WHERE id = %s LIMIT 1")
cursor.execute(update, (status, row[0]))
cnx.commit()
cursor.close()
now = datetime.now()
print(now.strftime("%d.%m.%Y %H:%M:%S"))
time.sleep(180) #sleep until it's time to re-check the queue
cnx.close()
This is my first project with SQLite.
The code runs perfect I checked and the lines look perfect.
I supposed that due to lack of knowledge of SQLite I'm making a mistake.
Problem: The code runs perfect no problem. But when I finish it doesn't print the values or even save the values in the .db file.
Full Code:
import sqlite3
import datetime
import time
conn = sqlite3.connect('covid.db')
c = conn.cursor()
def create_table():
c.execute('''CREATE TABLE IF NOT EXISTS
covidTrack(
name TEXT,
email TEXT,
ph_number INTEGER,
datestamp TEXT,
keyword TEXT)''')
i_name = input('Please insert FULL NAME : \n ...')
i_email = input('Please insert EMAIL : \n ...')
i_number = input('Please insert PHONE NUMBER : \n ...')
print('Your data has been saved for acelerated contact, thank you.')
time.sleep(3)
def data_entry():
c.execute('INSERT INTO covidTrack VALUES(?,?,?)',
(i_name, i_email, i_number))
conn.commit()
def dynamic_data_entry():
keyword = nameofvenue
date = str(datetime.datetime.fromtimestamp(unix).strftime('%Y-%M-%D %h:%m:%s'))
c.execute('INSERT INTO covidTrack VALUES(date, keyword)')
conn.commit()
def read_from_db():
c.execute('''SELECT * FROM covidTrack
WHERE datestamp
BETWEEN "2021-02-06 14:50:00" AND "2021-02-06 15:00:00"''')
conn.commit()
for row in c.fetchall():
print(row)
create_table()
data_entry()
dynamic_data_entry()
read_from_db()
c.close()
conn.close()
I suppose if something wrong with the way I use conn.commit().
import sqlite3
import datetime
import time
conn = sqlite3.connect('covid.db')
c = conn.cursor()
def create_table():
c.execute('''CREATE TABLE IF NOT EXISTS
covidTrack(
name TEXT,
email TEXT,
ph_number INTEGER,
datestamp TEXT,
keyword TEXT)''')
i_name = input('Please insert FULL NAME : \n ...')
i_email = input('Please insert EMAIL : \n ...')
i_number = input('Please insert PHONE NUMBER : \n ...')
print('Your data has been saved for acelerated contact, thank you.')
time.sleep(3)
def data_entry():
date, keyword = dynamic_data_entry()
c.execute('INSERT INTO covidTrack VALUES(?, ?, ?, ?, ?)', (i_name, i_email, i_number, date, keyword))
conn.commit()
def dynamic_data_entry():
keyword = 'nameofvenue'
date = str(datetime.datetime.fromtimestamp(time.time()).strftime('%Y-%M-%D %h:%m:%s'))
return date, keyword
def read_from_db():
c.execute('''SELECT * FROM covidTrack''')
conn.commit()
create_table()
data_entry()
read_from_db()
for row in c.fetchall():
print(row)
c.close()
conn.close()
change the code below (make the commit call part of the function that insert the data). Do it in dynamic_data_entry as well
def dynamic_data_entry():
keyword = nameofvenue
date = str(datetime.datetime.fromtimestamp(unix).strftime('%Y-%M-%D %h:%m:%s'))
c.execute('INSERT INTO covidTrack VALUES(date, keyword)')
conn.commit()
to
def dynamic_data_entry():
keyword = nameofvenue
date = str(datetime.datetime.fromtimestamp(unix).strftime('%Y-%M-%D %h:%m:%s'))
c.execute('INSERT INTO covidTrack VALUES(date, keyword)')
conn.commit()
You do not actually commiting your executes. Move conn.commit after actual executes.
hi i am looking to insert these 3 values into my SQL database table that has columns: email, cardnumber, dateandtime
here is my code:
email = input("Email: ")
cardnumber = int(input("Enter card number:"))
now = datetime.now()
now = now.strftime('%Y-%m-%d %H:%M:%S')
newrowforsql()
my code for the query is:
def newrowforsql():
query = """\
insert into table1 (email,cardnumber,dateandtime)
values(email,cardnumber,now)"""
insertnewrow = execute_query_commit(conn, query)
I cant seem to insert the values
my code for executing the query and committing it is:
def execute_query_commit(connection, query):
cursor = connection.cursor()
try:
cursor.execute(query)
connection.commit()
print("Query executed and committed")
except pyodbc.Error as e:
print(f"The error '{e}' occurred")
As "azro" mentioned correctly you didn't put in the variable content to the query, you just put in the name of the variable which contains the information you want. What you need to change is the following:
def newrowforsql():
query = """\
insert into table1 (email,cardnumber,dateandtime)
values(email,cardnumber,now)"""
insertnewrow = execute_query_commit(conn, query)
to
def newrowforsql():
query = """\
insert into table1 (email,cardnumber,dateandtime)
values({theEmail},{theCardnumber},{now})""".format(theEmail=email, theCardnumber=cardnumber, now=now)
insertnewrow = execute_query_commit(conn, query)
This is one of the most used options to manipulate strings in python. But if you are using python3.7+ (maybe from Python3.6 and up, but I'm not sure) there is a much better and faster option to manipulate strings, it's name is "f-strings".
Here is the same solution but with f-strings instead of the method str.format
def newrowforsql():
query = f"""\
insert into table1 (email,cardnumber,dateandtime)
values({email},{cardnumber},{now})"""
insertnewrow = execute_query_commit(conn, query)
Good luck!
Hello
I have a question about SQLite functions, maybe.
So, question:
How to check if name I set in Python is in certain column?
Example:
name = 'John'
Table name = my_table
Column name = users
Code details:
C = conn.cursor()
Please
Use parameter in the query as required. See the attached example for better understanding.
Sample SQLite code for searching value in tables
import sqlite3 as sqlite
import sys
conn = sqlite.connect("test.db")
def insert_single_row(name, age):
try:
age = str(age)
with conn:
cursor = conn.cursor()
cursor.execute("CREATE TABLE IF NOT EXISTS USER_TABLE(NAME TEXT, AGE INTEGER);")
cursor.execute("INSERT INTO USER_TABLE(NAME, AGE) VALUES ('"+name+"',"+age+")")
return cursor.lastrowid
except:
raise ValueError('Error occurred in insert_single_row(name, age)')
def get_parameterized_row(name):
try:
with conn:
cursor = conn.cursor()
cursor.execute("SELECT * FROM USER_TABLE WHERE NAME = :NAME",
{"NAME":name})
conn.commit()
return cursor.fetchall()
except:
raise ValueError('Error occurred in get_parameterized_row(name)')
if __name__ == '__main__':
try:
return_id = insert_single_row("Shovon", 24)
return_id = insert_single_row("Shovon", 23)
return_id = insert_single_row("Sho", 24)
all_row = get_parameterized_row("Shovon")
for row in all_row:
print(row)
except Exception as e:
print(str(e))
Output:
('Shovon', 24)
('Shovon', 23)
Here I have created a table called USER_TABLE with two attributes: NAME and AGE. Then I inserted several values in the table and searched for a specific NAME. Hope it gives a way to start using SQLite in the project.
I was told that writting directly to a cursor is a serious SQL vunlerability and anyone could easily dump my DB... How can I securely do SQL stuff?
import psycopg2
import web
urls = (
"/", "Index",
"/questlist", "Questlist"
)
web.config.debug = True
app = web.application(urls, globals())
render = web.template.render("templates/", base="layout")
con = psycopg2.connect(
database = "postgres",
user = "postgres",
password = "balloons",
port = "55210502147432"
)
class Index(object):
def __init__(self):
pass
def GET(self):
return render.index()
class Questlist(object):
def __init__(self):
pass
def GET(self):
try:
c = con.cursor()
c.execute("SELECT quest_title, quest_difficulty, quest_post FROM quest_list")
questlist = c.fetchall()
return render.quest(Quests = questlist)
except psycopg2.InternalError as e:
con.rollback()
print e
return "Session error"
return "wtf did u do,? u really busted her"
def POST(self):
form = web.input(quest_title="", quest_difficulty="", quest_post="")
if len(form.quest_title) + len(form.quest_difficulty) + len(form.quest_post) > 50:
return "Too many characters submitted"
try:
c = con.cursor()
c.execute("INSERT INTO quest_list (quest_title, quest_difficulty, quest_post) \
VALUES (%s, %s, %s)", (form.quest_title, form.quest_difficulty, form.quest_post))
con.commit()
except psycopg2.InternalError as e:
con.rollback()
print e
except psycopg2.DataError as e:
con.rollback()
print e
return "invalid data, you turkey"
return render.index()
if __name__ == "__main__":
app.run()
Here's the SQL im worried about:
c.execute("INSERT INTO quest_list (quest_title, quest_difficulty, quest_post) \
VALUES (%s, %s, %s)", (form.quest_title, form.quest_difficulty, form.quest_post))
here's the site right now that im using this on:
http://rpg.jeffk.org/questlist
feel free to try and break it
c.execute("INSERT INTO quest_list (quest_title, quest_difficulty, quest_post) \
VALUES (%s, %s, %s)", (form.quest_title, form.quest_difficulty, form.quest_post))
this is fine ... you are using the format strings that are built into python SQL libraries to avoid injection issues
c.execute("INSERT INTO quest_list(quest_title, quest_difficulty, quest_post)\
VALUES (%s, %s, %s)"%(form.quest_title, form.quest_difficulty, form.quest_post))
would be a potential security flaw as you are just using standard string formatting instead of the SQL mechanisms
when using standard string formatting consider the following user input
form.quest_post = "1);SELECT * FROM USERS;//"
this would allow them to dump your whole user table as it would get passed as
c.execute("INSERT INTO quest_list(quest_title,quest_dificulty,quest_post)\
VALUES (something_benign,something_else,1);SELECT * FROM USERS;//)")
which hopefully you can recognize as being a problematic statement ... or they could change your admin password or whatever ...