I have this piece of code that collects data from a HAT connected to a Raspberry.
When run it gives this error:
[51.57, 22.30, 1002.01]
Traceback (most recent call last):
File "dbWriter.py", line 45, in <module>
write2DB(record)
File "dbWriter.py", line 26, in write2DB
assert len(values) == 3
AssertionError
I am by no means a programmer, i just fiddle around. It is meant to save 'record' to a database, which is then read and updated in realtime on an apache2 server. All help appreciated.
import mysql.connector
from itertools import repeat
import sys
import bme680
import time
try:
sensor = bme680.BME680(bme680.I2C_ADDR_PRIMARY)
except IOError:
sensor = bme680.BME680(bme680.I2C_ADDR_SECONDARY)
sensor.set_humidity_oversample(bme680.OS_2X)
sensor.set_pressure_oversample(bme680.OS_4X)
sensor.set_temperature_oversample(bme680.OS_8X)
sensor.set_filter(bme680.FILTER_SIZE_3)
mydb = mysql.connector.connect(
host='localhost',
user='pi',
passwd='pass',
database='weatherDB'
)
mycursor = mydb.cursor()
def write2DB(values):
assert len(values) == 3
sqlText = '''INSERT INTO data(humidity, temperature, pressure) VALUES({},{}, })'''.format(values[0], values[1], values[2])
mycursor.execute(sqlText)
mydb.commit()
for _ in repeat(None):
sensor.get_sensor_data()
output_humi = '{0:.2f}'.format(
sensor.data.humidity)
output_temp = '{0:.2f}'.format(
sensor.data.temperature)
output_pres = '{0:.2f}'.format(
sensor.data.pressure)
record = []
record = ('[' + (output_humi) + ', ' + (output_temp) + ', ' + (output_pres) + ']')
print(record)
write2DB(record)
time.sleep(10)
pass
You have:
record = ('[' + (output_humi) + ', ' + (output_temp) + ', ' + (output_pres) + ']')
record evaluates to a single string, not a list of 3 elements and hence your exception.
Change the above to:
record = [output_humi, output_temp, output_pres]
You are also missing a { in your format specification. It should be:
sqlText = '''INSERT INTO data(humidity, temperature, pressure) VALUES({},{}, {})'''.format(values[0], values[1], values[2])
An alternative would be to use a prepared statement:
sqlText = 'INSERT INTO data(humidity, temperature, pressure) VALUES(%s, %s, %s)'
mycursor.execute(sqlText, values)
In the above case you will be passing actual strings as the values. I don't know how the columns are defined, but no matter. If they are defined as floating point or decimal values, the strings will be converted to the correct type.
Related
I don't know what I'm doing wrong with the leftJoin() function.
I have a connection to a DB2 database and an Oracle database.
Queries return me a result, both in DB2 and Oracle.
I continue to get the primary key value and try to pass it as a variable to the leftJoin() function, but it doesn't work here.
The key consists of two fields. If I manually put the value of 'ID', 'VER' into on in df1 in merge it works.
import ibm_db
import ibm_db as db
import ibm_db_dbi
import pandas as pd
import cx_Oracle
import re
def connectDB2():
arg1 = "DATABASE=...;HOSTNAME=...;PORT=...;UID=...;PWD=...;"
conn = db.connect(arg1, "", "")
if conn:
print ('connection success')
# Run SQL
sql_generic = "SELECT distinct 'select ' || LISTAGG(COLNAME,', ') || ' from ' || trim(TABSCHEMA) || '.' || tabname || ' FETCH FIRST 2 ROWS ONLY' FROM SYSCAT.columns WHERE TABSCHEMA = '...' AND TABNAME = '...' AND COLNAME NOT IN ('CDC_STATUS','CDC_ODS_UPD') GROUP BY TABSCHEMA, TABNAME"
stmt = ibm_db.exec_immediate(conn, sql_generic)
result = ibm_db.fetch_both(stmt)
conn1 = ibm_db_dbi.Connection(conn)
connectDB2.df = pd.read_sql(result['1'], conn1)
print('df', connectDB2.df)
sql_PK = "SELECT COLNAMES FROM syscat.INDEXES WHERE TABSCHEMA='...' AND TABNAME = '...' AND UNIQUERULE='P'"
conn2 = ibm_db_dbi.Connection(conn)
connectDB2.df1 = pd.read_sql(sql_PK, conn2)
print('pk', connectDB2.df1)
d = connectDB2.df1.loc[:, "COLNAMES"]
print('d', d)
print('d0', d[0])
content_new1 = re.sub('$|^', '\'', d[0], flags=re.M)
content_new2 = re.sub('\'\+', '\'', content_new1, flags=re.M)
connectDB2.content_new3 = re.sub('\+', '\',\'', content_new2, flags=re.M)
print('c3', connectDB2.content_new3) --> format: 'ID','VER'
else:
print ('connection failed')
def connectOracle():
con = cx_Oracle.connect('...')
orders_sql = """select ... from ... FETCH FIRST 2 ROWS ONLY""";
connectOracle.df_orders = pd.read_sql(orders_sql, con)
print(connectOracle.df_orders)
def leftJoin():
df1 = pd.merge(connectOracle.df_orders, connectDB2.df, on=connectDB2.content_new3, how='left')
connectDB2()
connectOracle()
leftJoin()
I am adding below what the logs return.
Traceback (most recent call last):
File "C:\Users\PycharmProjects\pythonProject1\testConnection.py", line 68, in <module>
leftJoin()
File "C:\Users\PycharmProjects\pythonProject1\testConnection.py", line 57, in leftJoin
df1 = pd.merge(connectOracle.df_orders, connectDB2.df, on=connectDB2.content_new3, how='left')
File "C:\Users\PycharmProjects\pythonProject1\venv\lib\site-packages\pandas\core\reshape\merge.py", line 106, in merge
op = _MergeOperation(
File "C:\Users\PycharmProjects\pythonProject1\venv\lib\site-packages\pandas\core\reshape\merge.py", line 699, in __init__
) = self._get_merge_keys()
File "C:\Users\PycharmProjects\pythonProject1\venv\lib\site-packages\pandas\core\reshape\merge.py", line 1096, in _get_merge_keys
right_keys.append(right._get_label_or_level_values(rk))
File "C:\Users\PycharmProjects\pythonProject1\venv\lib\site-packages\pandas\core\generic.py", line 1779, in _get_label_or_level_values
raise KeyError(key)
KeyError: "'ID','VER'"
You are using the merge command wrongly,
I dont know what is actually inside your given dfs
connectOracle.df_orders
connectDB2.df
But i know for sure, you are doing a left join. And you are passing a key found on your second df or "right" df per say.
pd.merge(connectOracle.df_orders, connectDB2.df, on = 'guy with the same index found in both your dfs', how='left')
If you dont have that guy, well them you should define your left key or your right that want to 'join', on the parameters
i would like to add values to the database in sqlite3, and they need to be encrypted. And when i want to retrieve values they need to be decrypted. Now I'm getting this error TypeError: token must be bytes. Here is the full list of errors: Traceback (most recent call last): File "C:\Users\d\OneDrive\Bureaublad\Assignment8\CDMS.py", line 75, in <module> get_clients() File "C:\Users\d\OneDrive\Bureaublad\Assignment8\CDMS.py", line 68, in get_clients new += fernet.decrypt(i).decode() + " " File "C:\Users\d\AppData\Local\Programs\Python\Python38\lib\site-packages\cryptography\fernet.py", line 75, in decrypt timestamp, data = Fernet._get_unverified_token_data(token) File "C:\Users\d\AppData\Local\Programs\Python\Python38\lib\site-packages\cryptography\fernet.py", line 100, in _get_unverified_token_data utils._check_bytes("token", token) File "C:\Users\d\AppData\Local\Programs\Python\Python38\lib\site-packages\cryptography\utils.py", line 29, in _check_bytes raise TypeError("{} must be bytes".format(name)) TypeError: token must be bytes
Here is my code:
from Classes import Client
from cryptography.fernet import Fernet
import sqlite3
key = Fernet.generate_key()
fernet = Fernet(key)
conn = sqlite3.connect('database.db')
c = conn.cursor()
def insert_client(client):
fullname = fernet.encrypt(client.fullname.encode())
fullname1 = fullname.decode()
adress = fernet.encrypt(client.adress.encode())
adress1 = adress.decode()
zipcode = fernet.encrypt(client.zipcode.encode())
zipcode1 = zipcode.decode()
city = fernet.encrypt(client.city.encode())
city1 = city.decode()
email = fernet.encrypt(client.email.encode())
email1 = email.decode()
mphone = fernet.encrypt(client.mphone.encode())
mphone1 = mphone.decode()
with conn:
c.execute("INSERT INTO client VALUES (:fullname, :adress, :zipcode, :city, :email, :mphone)",
{'fullname': fullname1, 'adress': adress1, 'zipcode': zipcode1,
'city': city1, 'email': email1, 'mphone': mphone1})
client1 = Client('Name1', 'Street1', 'Zipcode', 'NewYork', '123#gmail.com', '12345678')
def get_clients():
arr = []
new = ""
c.execute("SELECT * FROM client")
arr = c.fetchall()
for i in arr:
new += fernet.decrypt(i).decode() + " "
return new
insert_client(client1)
get_clients()
conn.close()
The first function is for adding clients, by a class object named Client. And the second function is for retrieving all values decrypted.
fetchall returns an iterable of rows (most commonly tuples). If you want to only process one record, you should instead use fetchone:
...
arr = c.fetchone()
for i in arr:
new += fernet.decrypt(i).decode() + " "
...
Alternatively you could iterate the cursor and return a list:
def get_clients():
arr = []
c.execute("SELECT * FROM client")
for row in c:
new = ""
for i in row:
new += fernet.decrypt(i).decode() + " "
arr.append(new)
return arr
But the Pythonic way would be to use comprehensions:
def get_clients():
c.execute("SELECT * FROM client")
return [' '.join(fernet.decrypt(i).decode() for i in row)
for row in c]
Slightly shorter, isn't it?
I'm trying to create a simple script for changing some mysql data, but I'm little bit confused about parametrized queries. The code look like this:
reader_id = sys.argv[1]
cmd_id = sys.argv[2]
conn = mysql.connect(user='...', passwd='...', host='...', db='...')
curs = conn.cursor()
mysql_data = {
"reader_id": int(reader_id),
"cmd_id": int(cmd_id)
}
curs.execute("UPDATE `reader_config_hw` SET `changed_` = '1' "
"WHERE `object_id` = %(reader_id)d AND `id` = %(cmd_id)d;", mysql_data)
curs.execute("UPDATE `reader` SET `check_for_command` = '1' WHERE `id` = %(reader_id)d", mysql_data)
and the error result is
Traceback (most recent call last):
File "./main.py", line 25, in <module>
"WHERE `object_id` = %(reader_id)d AND `id` = %(cmd_id)d;", mysql_data)
File "/usr/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 210, in execute
query = query % args
TypeError: %d format: a number is required, not str
It says that number is required but I already changed the type of the variable by int(reader_id), is it right?
Try changing
"WHERE 'object_id' = %(reader_id)d AND 'id' = %(cmd_id)d;"
to
"WHERE 'object_id' = %(reader_id)s AND 'id' = %(cmd_id)s;"
I think the 'd' after your variable calls indicates a "digit" whereas you need to use a string.
EDIT: to clarify, strings can be used in place of integers in MySQL queries
I have a sensor type DHT22 connected to a raspberry.
I have written a script in python but when I run it I get errors
#!/usr/bin/python
import MySQLdb
import subprocess
import re
import sys
import time
import datetime
import Adafruit_DHT
conn = MySQLdb.connect("localhost","zeus","gee3g673r","logi")
while(True):
date = time.strftime("%d/%m/%Y")
clock = time.strftime("%H:%M")
#output = subprocess.check_output(["/usr/bin/AdafruitDHT.py 2302", "4"]);
output = Adafruit_DHT.read_retry(Adafruit_DHT.AM2302, 4)
matches = re.search("Temp =\s+([0-9.]+)", output)
if (not matches):
time.sleep(0)
continue
temp = float(matches.group(1))
matches = re.search("Hum =\s+([0-9.]+)", output)
if (not matches):
time.sleep(0)
continue
humidity = float(matches.group(1))
# MYSQL DATA Processing
c = conn.cursor()
c.execute("INSERT INTO data_th (date, clock, temp, hum) VALUES (%s, %s,%s, %s)",(date, clock, temp, humidity))
#print "DB Loaded"
time.sleep(360)
This is the error encountered on running the script:
root#raspberrypi:/home# ./hdt.py
Traceback (most recent call last):
File "./dht.py", line 22, in <module>
matches = re.search("Temp =\s+([0-9.]+)", output)
File "/usr/lib/python2.7/re.py", line 142, in search
return _compile(pattern, flags).search(string)
TypeError: expected string or buffer
Adafruit_DHT.read_retry() does not return string. re.search expects string as second parameter.
Please have a look at code below (taken from Adafruit_Python_DHT/examples):
# Try to grab a sensor reading. Use the read_retry method which will retry up
# to 15 times to get a sensor reading (waiting 2 seconds between each retry).
humidity, temperature = Adafruit_DHT.read_retry(sensor, pin)
# Un-comment the line below to convert the temperature to Fahrenheit.
# temperature = temperature * 9/5.0 + 32
# Note that sometimes you won't get a reading and
# the results will be null (because Linux can't
# guarantee the timing of calls to read the sensor).
# If this happens try again!
if humidity is not None and temperature is not None:
print 'Temp={0:0.1f}* Humidity={1:0.1f}%'.format(temperature, humidity)
else:
print 'Failed to get reading. Try again!'
sys.exit(1)
Original Code:
import sys
import os
import latexmake
import mysql.connector
conn = mysql.connector.connect(user='root',password='oilwell',host='localhost',database='sqlpush1')
with conn:
mycursor = conn.cursor()
mycursor=execute("SELECT DATE,oil,gas,oilprice,gasprice,totrev FROM results WHERE DATE BETWEEN '2011-01-01' AND '2027-12-01'")
rows = mycursor.fetchall()
a.write("\\documentclass{standalone}\\usepackage{booktabs}\n\n\\usepackage{siunitx}\r \n\
\r\n\\begin{document}\r\n\\begin{tabular}{ccS[table-format = 5.2]} \\\\ \\toprule\r")
a.write("Date & Oil & Gas & Oil price & Gas price & Total Revenue \\\\ \\midrule \r")
for row in rows:
a = open("testtest.tex", "w")
a.write("" + str(row[0]) + " & " + str(row[1]) + " & " + str(row[2]) + " & " + str(row[3]) + " & " + str(row[4]) + " & " + str(row[5]) + " \\\\ \r")
a.write("\\bottomrule \\end{tabular}\r\\end{document}")
a.close
print (os.path.getsize("testtest.tex"))
os.system('latexmk.py -q testtest.tex')
mycursor.close()
conn.close()
a.close()
After run by IDLE, and red error pop up like
Traceback (most recent call last):
File "C:\Users\Cheng XXXX\Desktop\tabletest.py", line 8, in <module>
with conn:
AttributeError: __exit__
I checked the file and cannot file mistake, need help.
You are trying to use the connection as a context manager:
with conn:
This object doesn't implement the necessary methods to be used like that; it is not a context manager, as it is missing (at least) the __exit__ method.
If you are reading a tutorial or documentation that uses a different MySQL library, be aware that this feature may be supported by some libraries, just not this one. The MySQLdb project does support it, for example.
For your specific case, you don't even need to use the with conn: line at all; you are not making any changes to the database, no commit is required anywhere. You can safely remove the with conn: line (unindent everything under it one step). Otherwise you can replace the context manager with a manual conn.commit() elsewhere.
Alternatively, you can create your own context manager for this use-case, using the #contextlib.contextmanager() decorator:
from contextlib import contextmanager
#contextmanager
def manage_transaction(conn, *args, **kw):
exc = False
try:
try:
conn.start_transaction(*args, **kw)
yield conn.cursor()
except:
exc = True
conn.rollback()
finally:
if not exc:
conn.commit()
and use this as:
with manage_transaction(conn) as cursor:
# do things, including creating extra cursors
where you can pass in extra arguments for the connection.start_transaction() call.