I'm sure I'm making a simple mistake, but I be darned if I can figure it out. I am making a temperature/humidity monitor using a RPi and a DHT22. Code is in python. Works great. I'd like to dump the variable data collected into a MySQL db, but my insert query keeps failing. I can insert straight strings, but can't seem to figure out how to reference the variables. Here is my code
import time
time.sleep(2)
import MySQLdb
temperature = 60.0
humidity = 30.0
IP_Add = "123.456.78.9"
location = "basement"
name = "home"
while True:
humidity = humidity
temperature = temperature
fTemperature = (temperature * 9 / 5) + 32
name = 'home'
if humidity is not None and temperature is not None:
print('Temp={1:0.1f} Humidity={0:0.1f}%'.format(temperature, humidity))
else:
print('Whoops')
myDB = MySQLdb.connect(host="localhost", port = 3306, user = "root", passwd = "********", db = "PLAYTHING")
cur = myDB.cursor()
try:
#query1 = "INSERT INTO `testdata`(`Name`) VALUES (name)"
#query2 = "INSERT INTO `testdata`(`Name`, `Location`) VALUES (name, location)"
#query3 = "INSERT INTO `testdata`(`Name`, `Location`, `T-Reading`)VALUES (%s, %s, %s)", ('No_ID2', 'basement', 30.5)
query4 = "INSERT INTO `testdata`(`Name`, `Location`, `T-Reading`)VALUES {0} {1} {2}".format ('No_ID2', 'basement', 30.5)
#query5 = "INSERT INTO testdata(`Name`, `Location`, `T-Reading`, `H-Reading`) VALUES ('Friday3', 'location', 72.5, 29.6)"
cur.execute(query)
myDB.commit()
print ("Commit Sucessful")
except (MySQLdb.Error, MySQLdb.Warning) as e:
print(e)
cur.close()
myDB.close()
time.sleep(10)
I have checked the MySQLdb docs at https://mysqlclient.readthedocs.io/user_guide.html#functions-and-attributes
which offers this as a guide
"""INSERT INTO breakfast (name, spam, eggs, sausage, price)
VALUES (%s, %s, %s, %s, %s)""",
[
("Spam and Sausage Lover's Plate", 5, 1, 8, 7.95 ),
("Not So Much Spam Plate", 3, 2, 0, 3.95 ),
("Don't Wany ANY SPAM! Plate", 0, 4, 3, 5.95 )
] )
but that seems not to work for me.
Query 1 and 2 execute but enter no data, col's 3 and 4 are NULL.
Query 3 gives me this message "TypeError: query() argument 1 must be string or read-only buffer, not tuple"
Query 4 enters no data and gives me this: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'No_ID2 basement 30.5' at line 1")
Query 5 is successful, but doesn't solve the problem of getting the variables from the program and inserting them into the db.
If someone would point out my error I would appreciate it.
Issues with the queries:
#1 and #2: name and location in VALUES (name, location) are considered as column names in database, thus no data.
#3: As Ilja pointed out, the tuple should be in execute() call. This should be the way to go.
query3 = ("INSERT INTO `testdata`(`Name`, `Location`, `T-Reading`)"
+ " VALUES (%s, %s, %s)")
cur.execute(query3, ('No_ID2', 'basement', 30.5))
#4: To put value directly into VALUES, string must be quoted. Below is the right format. Note: This is for experimental only as it pose SQL Injection security risk.
query4 = ("INSERT INTO `testdata`(`Name`, `Location`, `T-Reading`)"
+ " VALUES ('{0}', '{1}', {2})".format (
'No_ID2', 'basement', 30.5
))
#5: All values in SQL statement are constant.
Related
So I am inserting data measurements into my MySQL database and sometimes I have null values that need to be inserted. Bu I always get the same error and I don't understand why. If the value isn't None, the measurement is inserted, but not the other way around.
This is the part of the code where a measurement is inserted.
sql = ("INSERT INTO `measurement` (`station_id`, `metric_name`, `raw_value`, `corrected_value`, `quality`, `failure`, `failure_type`, `timestamp`) " + \
"VALUES (%s, %s, %s, %s, %s, %s, %s, %s);")
if str(message["failure"]) == False:
failure = 0
else:
failure = 1
if str(message["true_value"]) == None:
cursor.execute(sql, (str(sensor_id[0][0]), metric, None, str(message["value"]), str(message["quality"]), failure, message["failure_type"], timestamp))
else:
cursor.execute(sql, (str(sensor_id[0][0]), metric, str(message["true_value"]), str(message["value"]), str(message["quality"]), failure, message["failure_type"], timestamp))
This is the error I get:
mysql.connector.errors.DatabaseError: 1265 (01000): Data truncated for column 'raw_value' at row 1
Column raw_value is configured as a double in the database.
I want to insert multiple records to a mysql db using python. I use mysql.connector. but I receive error. when I try to insert a record without formatting it works but multiple line with formatting doesn't!
I've used ? instead of %s but I still receive this error.
my_nodes = []
myconnection = mysql.connector.connect(
host='127.0.0.1', user='root', passwd='1234', db='job_graph1')
mycursor=myconnection.cursor()
for nodes in range(1, 33):
weight = random.randint(0, 100)
my_record = (nodes, weight, False, False)
my_nodes.append(my_record)
sqlqu = "INSERT INTO t_node(n_id,n_weight,is_entry,is_exit) VALUES(%S, %S, %s, %s)"
mycursor.executemany(sqlqu, my_nodes)
the error I receive is:
Failed processing format-parameters; %s" % err)
mysql.connector.errors.ProgrammingError: Failed processing format-parameters; 'tuple' object cannot be interpreted as an integer
So you need to remove %S in your sql request.
Because it causes this error :
print("(%s, %S)"%(1, 2))
ValueError: unsupported format character 'S' (0x53) at index 6
So instead of VALUES(%S, %S, %s, %s) use VALUES(%s, %s, %s, %s)
I'm quite new in Python (Python 3.4.6) :)
I'm trying to insert into a mysql db some lines but with variables.
At the beginning, I've a dictionary list_hosts.
Here is my code :
import mysql.connector
import time
db = mysql.connector.connect(host='localhost', user='xxxxx', passwd='xxxxx', database='xxxxx')
cursor = db.cursor()
now_db = time.strftime('%Y-%m-%d %H:%M:%S')
for key, value in list_hosts
key_db += key+", "
value_ex += "%s, "
value_db += "\""+value+"\", "
key_db = key_db.strip(" ")
key_db = key_db.strip(",")
value_ex = value_ex.strip(" ")
value_ex = value_ex.strip(",")
value_db = value_db.strip(" ")
value_db = value_db.strip(",")
add_host = ("INSERT INTO nagios_hosts (date_created, date_modified, "+key_db+") VALUES ("+value_ex+")")
data_host = ("\""+now_db+"\", \""+now_db+"\", "+value_db)
cursor.execute(add_host, data_host)
db.commit()
db.close()
Example of list_hosts:
OrderedDict([('xxxx1', 'data1'), ('xxxx2', 'data2'), ('xxxx3', 'data3'), ('xxxx4', 'data4'), ('xxxx5', 'data5'), ('xxxx6', 'data6')])
I've simplified the code of course.
I did it like this as I've never have the same amount of items in the dictionnary.
I'm trying to create something like this :
add_host - INSERT INTO TABLE (date_created, date_modified, xxxx1, xxxx2, xxxx3, xxxx4, xxxx5, xxxx6) VALUES (%s, %s, %s, %s, %s, %s)
data_host - now, now, data1, data2, data3, data4, data5, data6
Where there are never the same number of xxxx...
They all exist in the DB, but I don't need to fill each column for each item in the dictionnary.
When I execute I get this error :
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'xxxxxxxxxxxxxxxxxxxxx' at line 1
As I'm beginning with Python, I think there are a lot of things we can clean too... don't hesitate :)
Here's a canonical python3 (python2 compatible) solution:
import time
from collections import OrderedDict
list_hosts = OrderedDict([("field1", "value1"), ("field2", "value2"), ("fieldN", "valueN")])
# builds a comma-separated string of db placeholders for the values:
placeholders = ", ".join(["%s"] * (len(list_hosts) + 2))
# builds a comma-separated string of field names
fields = ", ".join(("date_created","date_modified") + tuple(list_hosts.keys()))
# builds a tuple of values including the dates
now_db = time.strftime('%Y-%m-%d %H:%M:%S')
values = (now_db, now_db) + tuple(list_hosts.values())
# build the SQL query:
sql = "INSERT INTO nagio_hosts({}) VALUES({})".format(fields, placeholders)
# and safely execute it
cursor.execute(sql, values)
db.commit()
As #khelwood mentioned in the comments, you should use parameterized queries.
If the number of columns you're inserting varies, you might prefer to generate a tuple and use it in a parameterized query then.
cursor.execute() accepts two parameters:
a query as a string;
parameters as a tuple.
The idea is to generate the string and the tuple and pass those to cursor.execute().
You'll need something like this:
list_hosts = {'xxxx1': 'data1', 'xxxx2': 'data2', 'xxxx3': 'data3', 'xxxx4': 'data4'}
keys = [] # creating a list for keys
values = () # creating a tuple for values
for key, value in list_hosts.items():
keys.append(key)
values = values + (value,)
keys_str = ', '.join(keys)
ps = ', '.join(['%s'] * len(list_hosts))
query = "INSERT INTO tbl (%s) VALUES (%s)" % (keys_str, ps)
print(query)
# INSERT INTO tbl (data_created, data_modified, xxxx1, xxxx2, xxxx3, xxxx4) VALUES (%s, %s, %s, %s)
cursor.execute(query, values)
Just tried it on a sample data, works fine!
I am trying to use Python to insert into MySQL database, but I have an auto-increment column (TeamID). I am using the lines below and it works like a charm. BUT I would like to not specify the TeamID in my Python script as it is an auto-increment
try:
cur.execute ("INSERT INTO teams values (%d, '%s', %d, '%s')" % (11,"Sevilla", 7, "Jorge Sampaoli"))
db.commit()
this works perfectly
How can I get rid of the first %d and 11 please? I want this value to be added automatically via the script
any help is very much appreciated
EDIT:
#!/usr/bin/python
import MySQLdb
db = MySQLdb.connect(host="localhost", # your host, usually localhost
user="username", # your username
passwd="password", # your password
db="dbname") # name of the data base
cur = db.cursor()
try:
cur.execute ("INSERT INTO teams values ('%s', %d, '%s')" % ("Sevilla", 7, "Jorge Sampaoli"))
db.commit()
except Exception as e:
print("Rolling back")
print(e)
db.rollback()
db.close()
Issue is now resolved
I did specify the column names but didn't notice I need to use %s for all columns including int values. As below:
cur.execute("INSERT INTO teams (TeamName, CountryID, TeamManager) values (%s,%s,%s)", ('Everton', 1, 'Ronald Koeman'))
Try
INSERT INTO teams (name, numb, player) VALUES ('%s', %d, '%s')
I.e.explicitly list columns.
Also PLEASE don't do it like this -- instead of doing '%s' you really need to use prepared statements,I think in Python it is something like:
cursor.execute("INSERT INTO teams (name, numb, player) VALUES (%s, %d, %s)", ['Sevilla', 7, 'John Smith'])
Read up on SQL injections.
import sqlite3
def insert_data(lVideoList, gate_id):
connection = sqlite3.connect('db.sqlite',
detect_types=sqlite3.PARSE_DECLTYPES |
sqlite3.PARSE_COLNAMES)
cursor = connection.cursor()
success = 200
# gateid = 1
default_value = 0
for gate_id in range(no_of_gate):
gate_id = i+1
for videofilename in lVideoList:
print("videofilename: ", videofilename)
insert_query = ("INSERT INTO dailyfootfall(csv_name, video_download, processed, footfall, send_status, "
"male_footfall, send_status_male, female_footfall, send_status_female, gate_id,outsiders, send_status_outsiders) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,?)")
cursor.execute(insert_query,[videofilename, success, success, default_value, success, default_value,
success, default_value, success, gate_id, default_value, success])
print("Data Inserted Successfully !")
connection.commit()
cursor.close()
connection.close()
if __name__ == "__main__":
lVideoList = getCompleteVideoList("2022_01_24", "10:00", "22:00")
no_of_gate = 3
insert_data (lVideoList, gate_id)
print("default_value inserted!!!!")
I am trying to take the data from a dictionary (the example is simplified for readability) and insert it into a mysql database.
I have the following piece of code.
import pymysql
conn = pymysql.connect(server, user , password, "db")
cur = conn.cursor()
ORFs={'E7': '562', 'E6': '83', 'E1': '865', 'E2': '2756 '}
table="genome"
cols = ORFs.keys()
vals = ORFs.values()
sql = "INSERT INTO %s (%s) VALUES(%s)" % (
table, ",".join(cols), ",".join(vals))
print sql
print ORFs.values()
cur.execute(sql, ORFs.values())
cur.close()
conn.close()
the print sql statement returns
INSERT INTO genome (E7,E6,E1,E2) VALUES(562,83,865,2756 )
when I type this directly into the mysql command line, the mysql command works. But when I run the python script I get an error:
<type 'exceptions.TypeError'>: not all arguments converted during string formatting
args = ('not all arguments converted during string formatting',)
message = 'not all arguments converted during string formatting'
As always, any suggestions would be highly appreciated.
The previous answer doesn't work for non string dictionary value. This one is a revised version.
format_string = ','.join(['%s'] * len(dict))
self.db.set("""INSERT IGNORE INTO listings ({0}) VALUES ({1})""".format(", ".join(dict.keys()),format_string),
(dict.values()))
sql = "INSERT INTO %s (%s) VALUES(%s)" % (
table, ",".join(cols), ",".join(vals))
This SQL includes values and cur.execute(sql, ORFs.values()) has values, too.
So, it should be cur.execute(sql).
In my case, I will skip null columns.
data = {'k': 'v'}
fs = ','.join(list(map(lambda x: '`' + x + '`', [*data.keys()])))
vs = ','.join(list(map(lambda x: '%(' + x + ')s', [*data.keys()])))
sql = "INSERT INTO `%s` (%s) VALUES (%s)" % (table, fs, vs)
count = cursor.execute(sql, data)