I have this:
import pymysql
import pymysql.cursors
host = "localhost"
port=3306
user = "db"
password='pass'
db='test'
charset='utf8mb4'
cursorclass=pymysql.cursors.DictCursor
try:
connection= pymysql.connect(host=host,port=port,user=user,password=passw,db=db,charset=charset,cursorclass=cursorclass)
Executor=connection.cursor()
except Exception as e:
print(e)
sys.exit()
I tried using the pandas to_sql(), but it is replacing the values in the table with the latest one. I want to insert the values into the table using the Pandas, but I want to avoid the duplicate entries and if any then it should get passed.
It might be possible to pickle the dataframe, and insert it into a table under a column of type BLOB. If you go this way, you'd have to depickle the result returned by mysqld
EDIT: I see what you are trying to do now. Here is a possible solution. Let me know if it works!
# assume you have declared df and connection
records = df.to_dict(orient = 'records')
for record in records:
sql = "INSERT INTO mytable ({0}) \
VALUES ({1})".format(record.keys(), record.values())
curs = connection.cursor()
try:
curs.execute(sql)
curs.close()
except:
break #handle/research the error
Related
I have a code like this:
import mysql.connector as mysql
from generate_records import generateRecords
devicesQuery = "CALL iot.sp_sensors_overview()"
try:
db = mysql.connect(
user = "username",
password = "password",
host = "hostname",
database="iot"
)
cursor = db.cursor(dictionary=True, buffered=True)
cursor.execute(devicesQuery)
for sensor in cursor:
generateRecords(sensor, db)
cursor.close()
except mysql.connector.Error as error:
print("Error:")
print(error)
else:
db.close()
The purpose of generateRecords function is obviously to generate records and run the INSERT query against the different table.
Seems like I do something wrong, because no matter what I trying, I getting different errors here, like mysql.connector.errors.OperationalError: MySQL Connection not available..
(upd) I also tried to change the code like it was suggested (see example bellow), with no luck - I still receiving the MySQL connection not available. error.
rows = cursor.fetchall()
cursor.close()
for sensor in rows:
cursor2 = db.cursor()
generateRecords(sensor, cursor2)
So, should I create a new connection within generateRecords function, or pass something different within it, or use some kind of different approach here?
Thank you!
Finally I found what was wrong. I'm used the query to call the stored procedure. Using the cursor.callproc("sp_sensors_overview") instead fixed my issue, and now I'm able to create the next cursor without errors.
I would like to ask how can I declare the variable "mobile" on the sql query?
I can print the mobile using .format() but it ain't working on the query cur.execute(). How should I declare the "mobile" variable for the query to update my database?
Also for your information I have a text file which named currentNum.txt and the only value inside is 639662146331(please take note that this should a string)
Thank you in advance!
import psycopg2
import os
try:
conn = psycopg2.connect("dbname='database' user='postgres' password='password'")
print("Connected to database.")
except:
print("Unable to connect to database.")
cur = conn.cursor()
try:
list=open("currentNum.txt", "r")
mobile=list.readline()
# It doesn't update the database
cur.execute("UPDATE notif_counter SET status='Notify' WHERE mobile='{0}'".format(mobile))
conn.commit();
# This is working
print("Table notif_counter where mobile {0} successfully updated.".format(mobile))
list.close()
except:
print("Failed to update the table for notif_counter")
finally:
if(conn):
cur.close()
conn.close()
print("PostgreSQL connection is closed.")
list = open("input.txt",'r')
mobile = list.readline().strip()
cur.execute("UPDATE notif_counter SET status='Notify' WHERE mobile='{}'".format(mobile))
it looks like you have forgot to remove trailing whitespace, use .strip()
I am quite new to programming. I have written the following code by researching from StackOverflow and other sites. I am trying to upload a csv file to the MS SQL Server. Every time I run this it connects and then a message pops up 'Previous SQL was not a query'. I am not sure how to actually tackle this. Any suggestions and help will be appreciated
import pyodbc import _csv
source_path= r'C:\Users\user\Documents\QA Canvas\module2\Module 2 Challenge\UFO_Merged.csv'
source_expand= open(source_path, 'r')
details= source_expand.readlines
print('Connecting...')
try:
conn = pyodbc.connect(r'DRIVER={ODBC Driver 13 for SQL Server};'r'SERVER=FAHIM\SQLEXPRESS;'r'DATABASE=Ash;'r'Trusted_Connection=yes')
print('Connected')
cur = conn.cursor()
print('Cursor established')
sqlquery ="""
IF EXISTS
(
SELECT TABLE_NAME ,TABLE_SCHEMA FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 'UFO_MERGED' AND TABLE_SCHEMA = 'dbo')
BEGIN
DROP TABLE [dbo].[UFO_MERGED]
END
CREATE TABLE [dbo].[UFO_MERGED]
( [ID] smallint
,[COMMENTS] varchar(max)
,[FIRST OCCURANCE] datetime
,[CITY] varchar(60)
,[COUNTRY] varchar(20)
,[SHAPE] varchar(20)
,[SPEED] smallint
,[SECOND OCCURANCE] datetime
PRIMARY KEY(id)
) ON [PRIMARY]
"""
result = cur.execute(sqlquery).fetchall()
for row in result:
print(row)
print("{} rows returned".format(len(result)))
sqlstr= """
Insert into [dbo].[UFO_Merged] values ('()','()','()','()','()','()','()','()')
"""
for row in details[1:]:
row_data =row.split(',')
sqlquery=sqlstr.format(row_data[0],row_data[1],row_data[2],row_data[3],row_data[4],row_data[5],row_data[6],row_data[7])
result=cur.execute(sqlquery)
conn.commit()
conn.close()
except Exception as inst:
if inst.args[0]== '08001':
print("Cannot connect to the server")
elif inst.args[0] == '28000':
print("Login failed - check connection string")
else:
print(inst)
Well, make sure the SQL works first, before you try to introduce other technologies (Python, R, C#, etc.) on top of it. The SQL looks a little funky, but I'm not a SQL expert, so I can't say for sure, and I don't have time to recreate your setup on my machine. Maybe you can try with something a bit less complex, get that working, and then graduate to something more advanced. Does the following work for you?
import pyodbc
user='sa'
password='PC#1234'
database='climate'
port='1433'
TDS_Version='8.0'
server='192.168.1.103'
driver='FreeTDS'
con_string='UID=%s;PWD=%s;DATABASE=%s;PORT=%s;TDS=%s;SERVER=%s;driver=%s' % (user,password, database,port,TDS_Version,server,driver)
cnxn=pyodbc.connect(con_string)
cursor=cnxn.cursor()
cursor.execute("INSERT INTO mytable(name,address) VALUES (?,?)",('thavasi','mumbai'))
cnxn.commit()
I'm having some troubles with this python method.
I don't have any problem getting the select results but when I've tried to execute the update I don't get any results.
I have tried to generate another cursor object, redefine the cursor, generate another connection, use a different sql query (without the use of the %s) and I didn't have any results.
If you could give me any help i would be really appreciate.
def getTarea():
conn = db.connect('url','user','pass','dbInstance')
with conn:
try:
cursor = conn.cursor(db.cursors.DictCursor)
sql = "SELECT CMD, ID_TAREA FROM TAREAS WHERE OBTENIDA = '0' AND DEVICE_ID = '1001' ORDER BY FECHA_TAREA DESC LIMIT 1"
cursor.execute(sql)
f.write(sql+"\n")
# fetch all of the rows from the query
data = cursor.fetchone()
# print the rows
f.write("CMD: "+data["CMD"]+"\n")
f.write("ID_TAREA: "+ str(data["ID_TAREA"])+"\n")
idTarea = str(data["ID_TAREA"])
obtenido = 1
cursor.execute("""UPDATE TAREAS SET OBTENIDA=%s WHERE ID_TAREA =%s""", (obtenido, idTarea))
cursor.close()
conn.close()
except Exception as e:
f.write("error \n"+e)
return cmd
conn.commit() will commit the changes, as documented in this similar post: Database does not update automatically with MySQL and Python
I've got psycopg2 running and I can successfully query tables of my database. This is a working example of querying the table my_table:
import psycopg2
try:
conn_string="dbname='my_dbname' user='user' host='localhost' password='password'"
print "Connecting to database\n->%s" % (conn_string)
conn = psycopg2.connect(conn_string)
print "connection succeeded"
except:
print "no connection to db"
cur = conn.cursor()
try:
cur.execute(""" SELECT * from my_table; """)
records = cur.fetchall()
cur.close()
except:
print "Query not possible"
Question: How can I query a view, let it be called my_view, within the same database my_dbname?
The same way you'd query a table. From a SELECT point of view, a VIEW is the exact same thing as a TABLE:
cur.execute("SELECT * from my_view")
Note that you generally do not want to use a black except:. Catch a specific exception if you have to, but you are usually better off not catching the exception at all rather than block all feedback on errors as you've done here.