SELECT_NO_PROCESSED_QRCODES = "SELECT id, visit_date, added_in_monitoring FROM locations_qrcode " \
"WHERE DATE_FORMAT(visit_date, '%Y%m%d') = %(value)s"
This SQL works perfectly outside a python (def) function.
Is this because of MySQL's DATE_FORMAT that stops working because it is inside a Python function?
My visit_date field is a datetime field, I need to use strftime within SQL to convert to ('% Y-% m-%d), how can I do this?
like: strftime(visit_date,'% Y-% m-% d')
1370 (42000): execute command denied to user 'erico'#'%' for routine 'visit_date.srftime'
user - password and database is ok
def process_qrcodes():
now = datetime.now().strftime('%Y-%m-%d')
while True:
try:
cur = conn.cursor()
#QRCODES = "SELECT id, added_in_monitoring, string FROM locations_qrcode WHERE added_in_monitoring='0'"
now = datetime.now().strftime("%Y-%m-%d")
NON_PROCESSED_QRCODES = "SELECT id, added_in_monitoring, string FROM locations_qrcode " \
"WHERE visit_date.srftime('%%Y-%%m-%%d') =%s" % (now,)
cur.execute(NON_PROCESSED_QRCODES)
results = cur.fetchall()
for result in results:
idqrcode = result[0]
string = result[2]
process_status_qrcode(idqrcode, string)
print("OI")
except Exception as e:
print(e)
time.sleep(10)
Related
I need to set some user meta in my wordpress through local python script. Hence I can't use the WP update_user_meta for it - it has to be done manually.
import mysql.connector as mysql
cnx = mysql.connect(host=HOST, database=DATABASE, user=USER, password=PASSWORD)
cursor = cnx.cursor()
get_meta = ("SELECT * FROM `ff_usermeta` WHERE `user_id`= 1 AND (`meta_key`='nickname' OR `meta_key`='info' OR `meta_key`='bg' OR `meta_key`='avatar' OR `meta_key`='profile_updated')")
cursor.execute(get_meta)
meta = cursor.fetchall()
#some processing of the result
cursor.execute(q, (...))
cnx.commit()
cursor.close()
cnx.close()
Now I need to check if the result has meta with each of the keys.
If the key already exists for this user, it needs to run UPDATE for this meta.
If this user still has no meta of this key, it has to INSERT new row.
if(there's no 'nickname' in meta_key on either of 5 or less rows):
q = ("INSERT INTO `ff_usermeta` ...")
else:
q = ("UPDATE `ff_usermeta` ...")
...and 4 more times like that?.. Seems like a good place for a cycle, but I don't really like the idea to make it 5x queues, especially since there might be more fields in the future.
I was thinking along the lines of searching the fetchall result for matches in meta_key, and if found, adding required data to one array, if not - to another. And then just running one update and one insert at the end, assuming both are not empty. If I were to write it in semi-php style, it would look roughly like this:
if(in_array("nickname", meta))
for_update .= "`nickname`='"+data[0]+"', "
else:
fields .= "`nickname`, "
vals .= "'"+data[0]+"', "
if(in_array("bg", meta)):
for_update .= "`bg`='"+data[1]+"', "
else:
fields .= "`bg`, "
vals .= "'"+data[1]+"', "
if(for_update):
update = ("UPDATE `ff_usermeta` SET "+for_update+" WHERE 1")
if(fields):
insert = ("INSERT INTO `ff_usermeta`("+fields+") VALUES ("+vals+")")
But absolutely no clue how to translate it correctly to python. I had to google it up to things like "why dot not working to add one string to another". Any advice? Or perhaps there is a better way? Thanks!
It is not complete, you can not update your rows in that way.
But with this you can start to make your query
The frist select gets exactly 1 row, if the user_id exists.
The user_id doesn't seem the right choice for this, but to get what you can do it is enough.
If the query doesn't have an entry, the it will insert some data you get from anywhere
The update as the insert are in that form wrong as you have to insert 5 new orws or update max 5 rows, but that is more for you to programm
import mysql.connector as mysql
HOST = "localhost"
DATABASE = ""
USER = "root"
PASSWORD = "mypassword"
cnx = mysql.connect(host=HOST, database=DATABASE, user=USER, password=PASSWORD)
cnx = mysql.connect(host=HOST, database=DATABASE, user=USER, password=PASSWORD)
cursor = cnx.cursor()
user_id = 1
get_meta = ("""SELECT umeta_id, user_id , MAX(IF( `meta_key`='nickname', meta_value,'')) AS 'nickname' , MAX(IF( `meta_key`='info', meta_value,'')) AS 'info' , MAX(IF( `meta_key`='bg', meta_value,'')) AS 'bg' , MAX(IF( `meta_key`='avatar', meta_value,''NULL'')) AS 'avatar' , MAX(IF (`meta_key`='profile_updated', meta_value,'')) AS 'profile_updated' FROM `ff_usermeta` WHERE `user_id`= %s GROUP BY umeta_id, user_id:""")
result = cursor.execute(get_meta,(user_id,))
if result > 0:
data = cursor.fetchone()
for_update = "";
#some processing of the result
if not data["nickname"]:
for_update += "`nickname`='"+data["nickname"]+"', "
if not data["bg"]:
for_update += "`bg`='"+data["bg"]+"', "
query = ("UPDATE `ff_usermeta` SET "+for_update+" WHERE user_id = " + user_id)
else:
#here are no data to be gathered as there is no user_id present add new user
nickname = ""
bg= ""
info = ""
avatar = ""
profile_updated = ""
fields= ""
vals = ""
fields += "`nickname`,`info`, `bg`,`avatar`,`profile_updated`"
vals += "'"+nickname+"', "+"'"+info+"', "+"'"+bg+"', "+"'"+avatar+"', "+"'"+profile_updatedfo+"'"
query = ("INSERT INTO `ff_usermeta`("+fields+") VALUES ("+vals+")")
cursor.execute(query)
cnx.commit()
cursor.close()
cnx.close()
I tried my best to adapt the suggestion above, but couldn't figure out how to make it work. Eventually I went another way, and it seems to work somehow, so I'll post the full code in case anyone would find it useful.
What it does: checks the queue in table with validation request, then parses a page (separate function) and updates user profile accodringly.
import mysql.connector as mysql
import time
from datetime import datetime
cnx = mysql.connect(host=HOST, database=DATABASE, user=USER, password=PASSWORD)
while True: #endless loop as a temporary scheduler
cursor = cnx.cursor()
#getting first request in the queue - 0: id, 1: url, 2: parse, 3: status, 4: user, 5: user_page, 6: req_date, 7: action
cursor.execute("SELECT * FROM `ff_qq` WHERE status = 0 LIMIT 1")
row = cursor.fetchone()
if row:
status = 1 #processed
if row[7] == "verify":
get_user = ("SELECT * FROM `ff_users` WHERE ID = %s LIMIT 1")
cursor.execute(get_user, (row[4],))
user = cursor.fetchone() #0 - ID, 5 - user_url, 8 - user_status, 9 - display_name
#separate function that returns data to insert into mysql
udata = verify(row) #0 - nickname, 1 - fb_av, 2 - fb_bg, 3 - fb_info, 4 - owner
ustat = row[1].split("/authors/")
if udata['owned'] or user[8] == ustat[1]:
update_user = ("UPDATE `ff_users` SET user_status = %s, display_name = %s, user_url = %s WHERE ID = %s LIMIT 1")
cursor.execute(update_user, (ustat[1], udata['nickname'], row[1], user[0]))
status = 2 #success
get = ("SELECT `meta_value` FROM `ff_usermeta` WHERE `user_id`= %s AND `meta_key`='ff_capabilities' LIMIT 1")
cursor.execute(get, (row[4],))
rights = cursor.fetchone()
if rights == 'a:1:{s:10:"subscriber";b:1;}':
promote = ("UPDATE `ff_usermeta` SET `meta_value` = 'a:1:{s:6:\"author\";b:1;}' "
"WHERE `user_id` = %s AND `meta_key`='ff_capabilities' LIMIT 1")
cursor.execute(promote, (row[0],))
#list of meta_key values in same order as returned data
ff = ['nickname', 'fb_av', 'fb_bg', 'fb_info']
for x in range(0,3): #goes through each one of the above list
if udata[ff[x]]: #yes this actually works, who would've thought?..
#current meta_key added directly into the string
get = ("SELECT `meta_value` FROM `ff_usermeta` WHERE `user_id`= %s AND `meta_key`='" + ff[x] + "' LIMIT 1")
cursor.execute(get, (row[4],))
meta = cursor.fetchone()
if(meta): #update if it exists, otherwise insert new row
qq = ("UPDATE `ff_usermeta` SET `meta_value` = %s "
"WHERE `user_id` = %s AND `meta_key`='" + ff[x] + "' LIMIT 1")
else:
qq = ("INSERT INTO `ff_usermeta`(`meta_value`, `meta_key`, `user_id`) "
"VALUES ('%s','" + ff[x] + "','%s'")
cursor.execute(qq, (udata[ff[x]], row[0])) #same execute works for both
else:
status = 3 #verification failed
#update queue to reflect its status
update = ("UPDATE `ff_qq` SET status = %s WHERE id = %s LIMIT 1")
cursor.execute(update, (status, row[0]))
cnx.commit()
cursor.close()
now = datetime.now()
print(now.strftime("%d.%m.%Y %H:%M:%S"))
time.sleep(180) #sleep until it's time to re-check the queue
cnx.close()
hi i am looking to insert these 3 values into my SQL database table that has columns: email, cardnumber, dateandtime
here is my code:
email = input("Email: ")
cardnumber = int(input("Enter card number:"))
now = datetime.now()
now = now.strftime('%Y-%m-%d %H:%M:%S')
newrowforsql()
my code for the query is:
def newrowforsql():
query = """\
insert into table1 (email,cardnumber,dateandtime)
values(email,cardnumber,now)"""
insertnewrow = execute_query_commit(conn, query)
I cant seem to insert the values
my code for executing the query and committing it is:
def execute_query_commit(connection, query):
cursor = connection.cursor()
try:
cursor.execute(query)
connection.commit()
print("Query executed and committed")
except pyodbc.Error as e:
print(f"The error '{e}' occurred")
As "azro" mentioned correctly you didn't put in the variable content to the query, you just put in the name of the variable which contains the information you want. What you need to change is the following:
def newrowforsql():
query = """\
insert into table1 (email,cardnumber,dateandtime)
values(email,cardnumber,now)"""
insertnewrow = execute_query_commit(conn, query)
to
def newrowforsql():
query = """\
insert into table1 (email,cardnumber,dateandtime)
values({theEmail},{theCardnumber},{now})""".format(theEmail=email, theCardnumber=cardnumber, now=now)
insertnewrow = execute_query_commit(conn, query)
This is one of the most used options to manipulate strings in python. But if you are using python3.7+ (maybe from Python3.6 and up, but I'm not sure) there is a much better and faster option to manipulate strings, it's name is "f-strings".
Here is the same solution but with f-strings instead of the method str.format
def newrowforsql():
query = f"""\
insert into table1 (email,cardnumber,dateandtime)
values({email},{cardnumber},{now})"""
insertnewrow = execute_query_commit(conn, query)
Good luck!
I am using a code that is using mysql. I am very new in mysql so I would be thankful if you could help. My input is a huge dumpfile of wikipediapages in xml bz2 format. The input format is some text files extracted from that xml file with this format:
<doc id="12" url="https://en.wikipedia.org/wiki?curid=12" title="Anarchism"> text... </doc>
the only parts that connects the program to sql is as follows:
def read_in_STOP_CATS(f_n = "/media/sscepano/Data/Wiki2015/STOPCAT/STOP_CATS.txt"):
s = []
f = open(f_n, "r")
for line in f:
s.append(line.rstrip().lower())
return s
def connect_2_db():
try:
cnx = mysql.connector.connect(user='test', password='test',
host='127.0.0.1',
database='wiki_category_links')
except mysql.connector.Error as err:
if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
print("Something is wrong with your user name or password")
elif err.errno == errorcode.ER_BAD_DB_ERROR:
print("Database does not exist")
else:
print(err)
return cnx
def articles_selected(aid):
global cnx
global STOP_CATS
cursor = cnx.cursor(buffered=True)
cursor.execute("SELECT * FROM categorylinks where cl_from = " + str(aid))
row = cursor.fetchone()
while row is not None:
#print(row)
cat = row[1].lower()
#print cat
for el in STOP_CATS:
if el in cat:
return False
row = cursor.fetchone()
return True
cnx = connect_2_db()
STOP_CATS = read_in_STOP_CATS()
TITLE_WEIGHT = 4
my problem is that right now I do not know how should I connect to mysql to be able to run the code and the main prob;lem is that I do not know what is categorylinks in the code? That should be the name of my sql table? Does it mean that I need to make an sql table with this name and import all my text file in this one table?
what does 'where' means in this line also????
As RiggsFolly said, you need to get something like WHERE cl_from = 'some string'
You could do it this way:
cursor.execute("SELECT * FROM categorylinks where cl_from ='" + str(aid)+"'")
But it is better to use prepared statements like this one:
select_stmt = "SELECT * FROM categorylinks where cl_from = %(aid)s"
cursor.execute(select_stmt, { 'aid':str(aid) })
So in your code you have:
A database named wiki_category_links
In that database you have a table called categorylinks
And the select you have means that you are going to get, from table categorylinks, all rows that have the column cl_from equal to the value of aid variable.
I am using Python 3.6, pyodbc, and connect to SQL Server.
I am trying make connection to a database, then creating a query with parameters.
Here is the code:
import sys
import pyodbc
# connection parameters
nHost = 'host'
nBase = 'base'
nUser = 'user'
nPasw = 'pass'
# make connection start
def sqlconnect(nHost,nBase,nUser,nPasw):
try:
return pyodbc.connect('DRIVER={SQL Server};SERVER='+nHost+';DATABASE='+nBase+';UID='+nUser+';PWD='+nPasw)
print("connection successfull")
except:
print ("connection failed check authorization parameters")
con = sqlconnect(nHost,nBase,nUser,nPasw)
cursor = con.cursor()
# make connection stop
# if run WITHOUT parameters THEN everything is OK
ask = input ('Go WITHOUT parameters y/n ?')
if ask == 'y':
# SQL without parameters start
res = cursor.execute('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN '2017-03-01T00:00:00.000' AND '2017-03-01T01:00:00.000'
''')
# SQL without parameters stop
# print result to console start
row = res.fetchone()
while row:
print (row)
row = res.fetchone()
# print result to console stop
# if run WITH parameters THEN ERROR
ask = input ('Go WITH parameters y/n ?')
if ask == 'y':
# parameters start
STARTDATE = "'2017-03-01T00:00:00.000'"
ENDDATE = "'2017-03-01T01:00:00.000'"
# parameters end
# SQL with parameters start
res = cursor.execute('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN :STARTDATE AND :ENDDATE
''', {"STARTDATE": STARTDATE, "ENDDATE": ENDDATE})
# SQL with parameters stop
# print result to console start
row = res.fetchone()
while row:
print (row)
row = res.fetchone()
# print result to console stop
When I run the program without parameters in SQL, it works.
When I try running it with parameters, an error occurred.
Parameters in an SQL statement via ODBC are positional, and marked by a ?. Thus:
# SQL with parameters start
res = cursor.execute('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN ? AND ?
''', STARTDATE, ENDDATE)
# SQL with parameters stop
Plus, it's better to avoid passing dates as strings. Let pyodbc take care of that using Python's datetime:
from datetime import datetime
...
STARTDATE = datetime(year=2017, month=3, day=1)
ENDDATE = datetime(year=2017, month=3, day=1, hour=0, minute=0, second=1)
then just pass the parameters as above. If you prefer string parsing, see this answer.
If you're trying to use pd.to_sql() like me I fixed the problem by passing a parameter called chunksize.
df.to_sql("tableName", engine ,if_exists='append', chunksize=50)
hope this helps
i tryied and have a lot of different errors: 42000, 22007, 07002 and others
The work version is bellow:
import sys
import pyodbc
import datetime
# connection parameters
nHost = 'host'
nBase = 'DBname'
nUser = 'user'
nPasw = 'pass'
# make connection start
def sqlconnect(nHost,nBase,nUser,nPasw):
try:
return pyodbc.connect('DRIVER={SQL Server};SERVER='+nHost+';DATABASE='+nBase+';UID='+nUser+';PWD='+nPasw)
except:
print ("connection failed check authorization parameters")
con = sqlconnect(nHost,nBase,nUser,nPasw)
cursor = con.cursor()
# make connection stop
STARTDATE = '11/2/2017'
ENDDATE = '12/2/2017'
params = (STARTDATE, ENDDATE)
# SQL with parameters start
sql = ('''
SELECT * FROM TABLE
WHERE TABLE.TIMESTAMP BETWEEN CAST(? as datetime) AND CAST(? as datetime)
''')
# SQL with parameters stop
# print result to console start
query = cursor.execute(sql, params)
row = query.fetchone()
while row:
print (row)
row = query.fetchone()
# print result to console stop
say = input ('everething is ok, you can close console')
I fixed this issue with code if you are using values through csv.
for i, row in read_csv_data.iterrows():
cursor.execute('INSERT INTO ' + self.schema + '.' + self.table + '(first_name, last_name, email, ssn, mobile) VALUES (?,?,?,?,?)', tuple(row))
I had a similar issue. Saw that downgrading the version of PyODBC to 4.0.6 and SQLAlchemy to 1.2.9 fixed the error,using Python 3.6
Hello
I have a question about SQLite functions, maybe.
So, question:
How to check if name I set in Python is in certain column?
Example:
name = 'John'
Table name = my_table
Column name = users
Code details:
C = conn.cursor()
Please
Use parameter in the query as required. See the attached example for better understanding.
Sample SQLite code for searching value in tables
import sqlite3 as sqlite
import sys
conn = sqlite.connect("test.db")
def insert_single_row(name, age):
try:
age = str(age)
with conn:
cursor = conn.cursor()
cursor.execute("CREATE TABLE IF NOT EXISTS USER_TABLE(NAME TEXT, AGE INTEGER);")
cursor.execute("INSERT INTO USER_TABLE(NAME, AGE) VALUES ('"+name+"',"+age+")")
return cursor.lastrowid
except:
raise ValueError('Error occurred in insert_single_row(name, age)')
def get_parameterized_row(name):
try:
with conn:
cursor = conn.cursor()
cursor.execute("SELECT * FROM USER_TABLE WHERE NAME = :NAME",
{"NAME":name})
conn.commit()
return cursor.fetchall()
except:
raise ValueError('Error occurred in get_parameterized_row(name)')
if __name__ == '__main__':
try:
return_id = insert_single_row("Shovon", 24)
return_id = insert_single_row("Shovon", 23)
return_id = insert_single_row("Sho", 24)
all_row = get_parameterized_row("Shovon")
for row in all_row:
print(row)
except Exception as e:
print(str(e))
Output:
('Shovon', 24)
('Shovon', 23)
Here I have created a table called USER_TABLE with two attributes: NAME and AGE. Then I inserted several values in the table and searched for a specific NAME. Hope it gives a way to start using SQLite in the project.