While calling the stored procedure of mysql using python, I am getting a syntax error.
The code for the stored procedure is as follows,
while True:
try:
date = time.strftime("%d/%m/%Y")
temp,humidity,light = main.reading()
args= (192.168.1.145, b8:27:eb:06:e4:4b, Temp_PI, temp)
cursor.callproc('SPR_IU_Sensor_Data',args)
conn.commit()
time.sleep(interval2)
except:
MySQLdb.Error, e:
conn.rollback()
print "Transaction aborted: %d: %s" % (e.args[0], e.args[1])
The error is as follows;
File "procedure.py", line 53
args= (192.168.1.145, b8:27:eb:06:e4:4b, Temp_PI, temp)
^
SyntaxError: invalid syntax
You need to quote the ip addresses, pass them in as strings:
args = ('192.168.1.145', 'b8:27:eb:06:e4:4b', Temp_PI, temp)
Python has no notion of an IP address literal notation.
Related
python code using nmap to scan a range of ip's and then store those ip's individually in mysql table.
how to scan when we need to scan a range of ip, e.g. 172.27.20.130-140. and store all ips in table of mysql. now a range is needed to be searched and each ip should be stored in table.
#NMAP SCANNING CODE
nmScan=nmap.PortScanner()
host=self._firstname.value # taking input from a pyforms gui input field
result=nmScan.scan(hosts=host, arguments='-sV -v -p 1-1024')
print('Host : %s (%s)' % (host, nmScan[host].hostname()))
print('State : %s' % nmScan[host].state())
for proto in nmScan[host].all_protocols():
print('----------')
print('Protocol : %s' % proto)
lport = nmScan[host][proto].keys()
#lport.sort()
for port in lport:
thisDict = nmScan[host][proto][port]
print ('port : %s\tstate : %s\tVersion:%s,v%s'% (port, nmScan[host][proto][port]['state'],thisDict['product'] ,thisDict['version']))
#INSERT into INPUT table CODE
try:
connection = mysql.connector.connect(host='localhost',
database='testdb',
user='root',
password='Pooja#123')
cursor = connection.cursor()
sql_insert_query = """ INSERT INTO input (sno,ip) VALUES (%s,%s)"""
so=cursor.lastrowid
sno=so
ip=self._firstname.value # taking input from a pyforms gui input field
insert_tuple = (sno,ip)
result = cursor.execute(sql_insert_query, insert_tuple)
print("inserted")
connection.commit()
except mysql.connector.Error as error :
connection.rollback() #rollback if any exception occured
print("Failed inserting record into input table {}".format(error))
finally:
#closing database connection.
if(connection.is_connected()):
cursor.close()
connection.close()
print("MySQL connection is closed")
The ipaddress module is useful for this type of thing:
In [1]: import ipaddress
In [2]: addr = ipaddress.IPv4Address('172.27.20.130')
In [3]: print(addr)
172.27.20.130
In [4]: print(addr + 1)
172.27.20.131
I'm trying to get data from raspberry pi 3 to Azure
The script reads data from Raspberry Pi3, which is connect via bluetooth to sensors and takes several values.
Unfortunately i am getting an error, when I run it
"local variable 'temperature' referenced before assignment"
def iothub_client_sample_run():
msgs=[]
for address, name in list(devices.items()):
try:
client = iothub_client_init()
if client.protocol == IoTHubTransportProvider.MQTT & (name == "Flower care"):
msg_txt_formatted = MSG_TXT % (
temperature,
sunlight,
moisture,
fertility)
message = IoTHubMessage(msg_txt_formatted)
# optional: assign ids
message.temperature_id = "%d" % temperature
client.send_event_async(message, send_confirmation_callback, devices.items())
print ( "IoTHubClient.send_event_async accepted message {} for transmission to IoT Hub.".format(devices.items()) )
return msgs
while 1:
msgs=iothub_client_sample_run()
for msg in msgs:
print msg['topic']
print msg['payload']
(result, mid)=mqttc.publish(msg['topic'],msg['payload'])
print ( "Send status: %s" % status )
time.sleep(10)
mqttc.disconnect()
except IoTHubError as iothub_error:
print ( "Unexpected error %s from IoTHub" % iothub_error )
return
except KeyboardInterrupt:
print ( "IoTHubClient sample stopped" )
print_last_message_time(client)
The error message here is pretty clear.
Remember that Python reads and executes code one line at a time so if you have a variable stated after the function that uses it then it is going to throw an error. Juggle your code around to put your variables before they are called and you shouldn't have any problems with this error again.
I am trying to make two calls to the database, calls are being multithreaded. However sometimes the first call data may be empty, and therefore it gets an error: ProgrammingError: (2014, “Commands out of sync; you can't run this command now”)
when I call the second query. Is there a way to clear the cursor without closing the database connection?
# CHECK IF NO RECORD IN DATABASE OF SKU
try:
cursor.execute('SELECT * FROM `amazon` inner join `ebay` on amazon.ASIN=ebay.SKU WHERE `ASIN` = "%s"' % (sku))
data = cursor.fetchall()
except Exception as ee:
conn.rollback()
data = None
print "Ebay error", ee
# IF NO RECORD ADD EVERYTHING
if not data:
try:
print "Inserting into ebay table: " + str((sku, float(price), int(quantity), itemID, float(cp_price)))
cursor.execute('INSERT INTO ebay VALUES ("%s", %.2f, %.2f, %s, %.2f, %s)' % (sku, float(price), int(quantity), itemID, float(cp_price), True))
conn.commit()
except Exception as ee:
print('DB insert error', ee)
i += 1
continue
I am working on a python script to take an incoming String of data from several arduinos to read, split and insert that data into a database.
The problem is the sensor data varies in the number of values depending on what kind of sensor is used. I cannot figure out the proper way to loop through the separated pieces and insert them properly.
A '1' specifies 10HS sensor and need 1 space for the incoming value
The String comes in like this for 10HS sensors:
"Cucumber2015,Arduino01,1,20150918124200,25.3,75.5,1,12 .....
A '2' specifies 10HS sensor and need 1 space for the incoming value
"Cucumber2015,Arduino01,1,20150918124200,25.3,75.5,2,12,24,23 ......
The for loop should repeat until all the sensor values have an insert statement for their respective tables.
I have tried the code shown below and keep getting errors.
How can I accomplish this? Besides the syntax problem, am I going about this correctly?
My current error
File "serialToDbV3.py", line 50
index =index+1
^
SyntaxError: invalid syntax
Python code
#!/usr/bin/python
import serial
import MySQLdb
#establish connection to MySQL. You'll have to change this for your database.
dbConn = MySQLdb.connect("localhost","python_user","password","IrrigationDB") or die ("could not connect to database")
#open a cursor to the database
cursor = dbConn.cursor()
device = '/dev/ttyUSB0' #this will have to be changed to the serial port you are using
baudrate = 9600
def getSerialData():
try:
print "Trying...",device
arduino = serial.Serial(device, baudrate)
except:
print "Failed to connect on",device
try:
print "Trying to get data"
next(arduino)
data = arduino.readline() #read the data from the Arduino
pieces = data.split(",") #split the data by the tab
print "Data: %s" % data
print "Piece 0: ProjectID %s" % pieces[0]
print "Piece 1: ArduinoID %s" % pieces[1]
print "Piece 2: Plot# %s" % pieces[2]
print "Piece 3: SQLTime %s" % pieces[3]
print "Piece 4: AirTemp %s" % pieces[4]
print "Piece 5: Humidity %s" % pieces[5]
print "Piece 6: SensType %s" % pieces[6]
print "Piece 7: SensData %s" % pieces[7]
print "Piece 8: %s" % pieces[8]
print "Piece 9: %s" % pieces[9]
print "Piece 10: %s" % pieces[10]
#Here we are going to insert the data into the Database
try:
print "Trying insertion..."
cursor.execute("INSERT IGNORE INTO `IrrigationDB`.`Project`(`idProject`)VALUES (%s)", (pieces[0]))
cursor.execute("INSERT IGNORE INTO `IrrigationDB`.`Arduino`(`idArduino`,`FK_ProjectID`)VALUES (%s,%s)", (pieces[1],pieces[0]))
cursor.execute("INSERT IGNORE INTO `IrrigationDB`.`Plot`(`idPlot`,`FK_ArduinoID`)VALUES (%s,%s)", (pieces[2],pieces[1]))
cursor.execute("INSERT INTO `IrrigationDB`.`Reading`(`DateAndTime`,`airTempC`,`relativeHumidity`,`FK_PlotID`)VALUES (%s,%s,%s,%s)", (pieces[3],pieces[4],pieces[5],pieces[2]))
startingPosition = 6
for index in xrange(startingPosition, len(pieces), step):
if pieces[6] == 1 :
cursor.execute("INSERT INTO `IrrigationDB`.`10HS_Sensor`(`id10HS_Sensor`,`DielectricPermittivity`)VALUES (%s,%s,%s)", (pieces[2],pieces[i+1])
index =index+1
if pieces[6] == 2 :
cursor.execute("INSERT INTO `IrrigationDB`.`GS3_Sensor`(`idGS3_Sensor`,`DielectricPermittivity`,`soilTempC`,`electricalConductivity`)VALUES (%s,%s,%s)", (pieces[2],pieces[i+1],pieces[i+2],pieces[i+3]))
index=index+3
except MySQLdb.IntegrityError:
print "failed to insert data"
except:
print "Failed to get data from Arduino!"
val = 0
while val == 0 :
getSerialData()
Use proper exception reporting, at least until you get to end-user land.
try:
cursor.execute("INSERT IGNORE INTO `IrrigationDB`.`Project`(`idProject`)VALUES (%s)", (pieces[0]))
except Exception, e:
print "this is sql #6", e
raise
This will tell you more clearly what is going on, which you can then post here.
I have a problem where I can upload CSV files to MySQL, but then something happens and I get an encoding error. Can some one please review my code and tell what is wrong? I'm new to enconding.
The following snippet is how I write the CSV files that will be uploaded, the data is extracted from an MDB file using the MDN tools (mdb-export):
tableIndex = 1
for tName in tableNames:
fileName = os.path.join(csvPath, os.path.basename(mdb).split('.')[0] + '_' + tName + '.csv')
try:
p = subprocess.Popen(["mdb-export", "-H", mdb, tName], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
tableContent, error = p.communicate()
if(p.returncode != 0):
_logger.error('[%3d] Export Subprocess %d %s' % (tID, p.returncode, tableContent))
SendMdbError(tID, mdb, _logger, 'ALERT: Export Subprocess')
return(['', False])
if(error):
_logger.error('[%3d] Export Communicate %d %s' % (tID, p.returncode, error.strip()))
SendMdbError(tID, mdb, _logger, 'ALERT: Export Communicate')
return(['', False])
except Exception as ex:
_logger.exception('[%3d] Export Error' % tID)
SendMdbError(tID, mdb, _logger, 'ALERT: Export Exception')
return(['', False])
except:
_logger.exception('[%3d] Export Unexpected' % tID)
SendMdbError(tID, mdb, _logger, 'ALERT: Export Unexpected')
return(['', False])
# If no data, no need for corresponding SQL
if(len(tableContent) == 0):
emptyTables.append(tName)
# If data exists, dump data
else:
# Add the 'DriveTest' to the data to upload
tableContent = tableContent.split('\n')
tableContent = [dt + ',' + line for line in tableContent if(line)]
tableContent = '\n'.join(tableContent)
try:
with open(fileName, 'wb') as f:
f.write(tableContent)
if(_VERBOSITY):
_logger.debug('[%3d] %3d - Write CSV SIZE[%8d] FILE: %s' %(tID, tableIndex, len(tableContent.split('\n')), fileName))
tableIndex += 1
except IOError as err:
_logger.exception('[%3d] Write IOError: %s' % (tID, str(err)))
SendMdbError(tID, mdb, _logger, 'ALERT: Write IOError')
return(['', False])
except Exception as ex:
_logger.exception('[%3d] Write Exception' % tID)
SendMdbError(tID, mdb, _logger, 'ALERT: Write Exception')
return(['', False])
except:
_logger.exception('[%3d] Write Unexpected: %s' % tID)
SendMdbError(tID, mdb, _logger, 'ALERT: Write Unexpected')
return(['', False])
The following is where I upload the CSV file, and here is where I get the error:
# Upload the data
tableIndex = 0
for table in tableDDL:
try:
with warnings.catch_warnings(record=True) as war:
_logger.info('[%3d] %3d Going up... %s' %(tID, tableIndex+1, os.path.basename(mdb).split('.')[0] + '_' + table))
_sqlLock[tableIndex].acquire()
#self.cursor.execute(tableDDL[table])
self.cursor.execute(tableULD[table])
self.conn.commit()
_sqlLock[tableIndex].release()
if(war):
#if(_VERBOSITY): print('[%3d] %3d WARNINGS[%3d] %s' % (tID, tableIndex+1, len(war), os.path.basename(mdb).split('.')[0] + '_' + table))
_logger.warning('[%3d] %3d WARNINGS[%3d] %s' % (tID, tableIndex+1, len(war), os.path.basename(mdb).split('.')[0] + '_' + table))
for w in war:
_logger.warning('[%3d] %s' % (tID, w.message))
#if(_VERBOSITY): print('[%3d] %3d Uploaded %s' % (tID, tableIndex+1, os.path.basename(mdb).split('.')[0] + '_' + table))
_logger.info('[%3d] %3d Uploaded %s' % (tID, tableIndex+1, os.path.basename(mdb).split('.')[0] + '_' + table))
tableIndex += 1
# Remove the uploaded CSV file
try:
os.remove(csvFiles[table]+'.csv')
_logger.info('[%3d] Removed CVS file: %s' % (tID, csvFiles[table]+'.csv'))
except OSError:
pass
except (MySQLdb.InternalError, MySQLdb.NotSupportedError) as err:
_logger.error('[%3d] %3d Internal: %s %s' % (tID, tableIndex+1, err, sys.exc_info()[0]))
self.conn.rollback()
self.Disconnect(tID, _logger, _VERBOSITY, _DEBUG)
return(False)
except MySQLdb.OperationalError as err:
_logger.error('[%3d] %3d OperationalError: %s' % (tID, tableIndex+1, sys.exc_info()[0]))
_logger.error(err)
self.conn.rollback()
self.Disconnect(tID, _logger, _VERBOSITY, _DEBUG)
return(False)
except MySQLdb.ProgrammingError as err:
_logger.error('[%3d] %3d ProgrammingError: %s' % (tID, tableIndex+1, sys.exc_info()[0]))
_logger.error(err)
self.conn.rollback()
self.Disconnect(tID, _logger, _VERBOSITY, _DEBUG)
return(False)
except MySQLdb.Error as err:
_logger.error('[%3d] %3d QUERY: %s %s' % (tID, tableIndex+1, err, sys.exc_info()[0]))
self.conn.rollback()
self.Disconnect(tID, _logger, _VERBOSITY, _DEBUG)
return(False)
except Exception as err:
_logger.error('[%3d] %3d Exception: %s %s' % (tID, tableIndex+1, err, sys.exc_info()[0]))
#self.conn.rollback()
#self.Disconnect(tID, _logger, _VERBOSITY, _DEBUG)
#return(False)
pass
except:
_logger.error('[%3d] %3d Other: %s' % (tID, tableIndex+1, sys.exc_info()[0]))
self.conn.rollback()
self.Disconnect(tID, _logger, _VERBOSITY, _DEBUG)
return(False)
The error I get is the following:
2015-06-13 19:42:21,743 __main__ - ERROR - [ 1] 1 Exception: 'ascii' codec can't encode character u'\xb4' in position 40: ordinal not in range(128) <type 'exceptions.UnicodeEncodeError'>
2015-06-13 19:42:30,962 __main__ - ERROR - [ 1] 1 Exception: 'ascii' codec can't encode character u'\xb4' in position 27: ordinal not in range(128) <type 'exceptions.UnicodeEncodeError'>
I noticed that the given data gets uploaded, but not sure if all rows are uploaded.
Thanks!
Try before putting csv into DB s.decode('UTF-8') and after getting it out of the DB s.encode('UTF-8')
I did it for SQLite and it worked OK.
Getting this to work should not be too difficult, but you have to understand what you're doing. Don't just try all possible combinations of s.encode("UTF-8").decode("UTF-8") and stuff like that.
First, understand the difference between a string and bytes. See https://docs.python.org/3/howto/unicode.html. You can encode a string to bytes: bytes = text.encode("UTF-8"), and you can decode bytes to a string: text = bytes.decode("UTF-8")
Second since a CSV file is a text file, you should open the CSV file in text mode. open(fileName, 'w', encoding="utf-8"). There's no need to encode or decode text in your code when writing the file.
Third, it is perfectly OK to write Unicode text to a TEXT field. No need for BINARYs or BLOBs. But make sure your database has a collation setting that can deal with it, usually that would be one of the utf-8 collations. Then to put Unicode in your database, use python strings and don't decode them to bytes.
The error message implies that the column definition in MySQL is CHARACTER SET ascii; is that correct?
B4 sounds like the latin1 (not utf8) encoding for ´, which could be coming from a Microsoft Word document in a context such as it´s.
So, even changing the column to be CHARACTER SET utf8 won't fix the problem.
BINARY and BLOB are essentially the same type of field -- any byte is allowed. VARCHAR and TEXT validate the bytes during INSERT to make sure they match the CHARACTER SET.