I'm trying to export a python dataframe to a SQL Server table.
Is there a better way to do this? I'm getting errors.
Dataframe - results_out
Output SQL table - FraudCheckOutput
cnn_out = pyodbc.connect('driver={SQL Server};server=XYZ;database=BulkLog;uid=sa;pwd=test')
results_out.to_sql(con=cnn_out, name='FraudCheckOutput', if_exists='replace', flavor='sqlite_master')
Thanks.
Ok, this supposed to make the work done:
import pypyodbc
def database_insert(query, params=())
conn_params = 'driver={SQL Server};server=XYZ;database=BulkLog;uid=sa;pwd=test'
try:
conn = pypyodbc.connect(conn_params)
except pypyodbc.Error, e:
print str(e)
else:
if conn.connected:
db = conn.cursor()
db.execute(query, params).commit()
finally:
if conn:
conn.close()
SQL_INSERT_QUERY = """
INSERT INTO table_name (
[field_name1],
[field_name2]
)
VALUES (
1,
'vale string'
)
WHERE
field_name3 = ?
"""
database_insert(SQL_INSERT_QUERY, ('field_name3_value',))
in pyodbc usage is very similar
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=localhost;DATABASE=testdb;UID=me;PWD=pass')
cursor = cnxn.cursor()
cursor.execute("insert into products(id, name) values ('pyodbc', 'awesome library')")
cnxn.commit()
more on http://code.google.com/p/pyodbc/wiki/GettingStarted
Related
I encountered error message (INVALID SYNTAX) and have no idea why it happens.
Please help to figure it out.
import pymysql
db = pymysql.connect(host='localhost', port=3306, user='root', passwd='xxxxxxxx', db='ecommerce', charset='utf8')
ecommerce = db.cursor()
for index in range(10):
product_code = 215673140 + index +1
sql=INSERT INTO product VALUES(str(product_code),'sample data1','sample date2','sample data3'); ----> here's error point.
ecommerce.execute(sql)
db.commit()
db.close()
Try this. You have to enclose INSERT INTO ... inside " ". Also, don't use ; after the statement.
I suggest you should use %s. That way, you can insert any value inside that column
import pymysql
db = pymysql.connect(host='localhost', port=3306, user='root', passwd='xxxxxxxx', db='ecommerce', charset='utf8')
ecommerce = db.cursor()
for index in range(10):
product_code = 215673140 + index +1
sql = "INSERT INTO product VALUES (%s, %s,%s,%s,%s)"
val = (str(product_code),'sample data1','sample date2','sample data3')
ecommerce.execute(sql, val)
db.commit()
db.close()
im pretty new with python,im sorry if my question and my english doesnt sound right at all.
ive been trying to integrated my scraping output to mysql,but unfortunately i got stuck.can you good people help me?
the output/error
*<pymysql.cursors.Cursor object at 0x000000E81FF095E0>
<pymysql.connections.Connection object at 0x000000E81FEF9070>
KeyError Traceback (most recent call last)
in
9 data = csv.DictReader(scrapeddata)
10 for row in data:
---> 11 sql = "INSERT INTO emas ( tanggal, terakhir, pembukaan, tertinggi, terendah, vol, perubahan%) VALUES ('%s','%s','%s','%s','%s','%s','%s')" %(str(row["tanggal"]),str(row["terakhir"]),str(row["pembukaan"]),str(row["tertinggi"]),str(row["terendah"]),str(row["vol"]),str(row["perubahan%"]))
12 print(sql)
13 cur.execute(sql)
KeyError: 'tanggal'*
this is my code
import csv
import pymysql
#Connecting to MySQL in Windows
conn = pymysql.connect(host="127.0.0.1", port = 3306, user = "root", passwd = '', database = "mysql",
charset = "utf8")
cur = conn.cursor()
cur.execute("USE historis")
print(cur)
print(conn)
with open(r'C:\Users\shulhan\output_emas.csv') as scrapeddata:
data = csv.DictReader(scrapeddata)
for row in data:
sql = "INSERT INTO emas ( tanggal, terakhir, pembukaan, tertinggi, terendah, vol, perubahan%)
VALUES ('%s','%s','%s','%s','%s','%s','%s')"
%(str(row["tanggal"]),str(row["terakhir"]),str(row["pembukaan"]),str(row["tertinggi"]),str(row["terendah"]),str(row["vol"]),str(row["perubahan%"]))
print(sql)
cur.execute(sql)
conn.commit()
i think your connection setup should look like this:
connection = pymysql.connect(host='localhost',
user='user',
password='passwd',
db='db',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
you can find more information here:
https://pymysql.readthedocs.io/en/latest/user/examples.html
So here is my code. I don't know why insert isn't working. A select statement works. It doesn't fail the try catch either leading me to believe the query is executing. Also entering the insert query manually into MySQL Workbench seems to work fine.
def runQuery(query):
try:
conn = mysql.connector.connect(host='localhost',
database='optionsdata',
user='python',
passwd='python')
cursor = conn.cursor()
cursor.execute(query)
conn.close()
cursor.close()
print(query)
except Error as e:
print("Error", e)
def convertDate(date_str):
date_object = datetime.datetime.strptime(date_str, '%m/%d/%Y').date()
return date_object
ticker = "MSFT"
html = urlopen("https://api.nasdaq.com/api/quote/" + ticker + "/option-chain?assetclass=stocks&todate=2020-05-08&fromdate=2020-04-07&limit=0").read().decode('utf-8')
optionsData = json.loads(html)
rows = optionsData["data"]["optionChainList"]["rows"]
for row in rows:
call = row["call"]
expiryDate = convertDate(call["expiryDate"])
query = "INSERT INTO `optionsdata`.`call` (`ticker`, `symbol`, `last`, `change`, `bid`, `ask`, `volume`, `openinterest`, `strike`, `expiryDate`, `grabTime`) VALUES ('{0}', '{1}', '{2}', '{3}', '{4}', '{5}', '{6}', '{7}', '{8}', '{9}', '{10}');".format(ticker, call["symbol"], call["last"], call["change"], call["bid"], call["ask"], call["volume"], call["openinterest"], call["strike"], expiryDate, datetime.datetime.now())
runQuery(query)
A sample of what an insert query looks like
INSERT INTO `optionsdata`.`call` (`ticker`, `symbol`, `last`, `change`, `bid`, `ask`, `volume`, `openinterest`, `strike`, `expiryDate`, `grabTime`) VALUES ('MSFT', '#MSFT 200508C00175000', '3.21', '-0.29', '2.80', '4.25', '54', '228', '175.00', '2020-05-08', '2020-04-09 19:39:22.554538');
This is a great question! I spent hours trying to figure this out a few weeks ago. It's tricky because after executing the query, you have to call
conn.commit()
to actually update the data. So change your runQuery function like this:
def runQuery(query):
try:
conn = mysql.connector.connect(host='localhost',
database='optionsdata',
user='python',
passwd='python')
cursor = conn.cursor()
cursor.execute(query)
conn.commit() # Added commit line
conn.close()
cursor.close()
print(query)
except Error as e:
print("Error", e)
See this doc page for more info.
output is like
i displayed result from database in a tabular form using BeautifulTable python. but the result is unorganised.
try:
connection = mysql.connector.connect(host='localhost',
database='testdb',
user='root',
password='Pooja#123')
cur= connection.cursor()
sql = "select ip,ports,services from output where date=now()"
number_of_rows = cur.execute(sql)
table=BeautifulTable()
table.set_style(BeautifulTable.STYLE_DOTTED)
table.column_headers=["ip","ports","services"]
table.column_alignments['ip'] = BeautifulTable.ALIGN_LEFT
table.column_alignments['ports'] = BeautifulTable.ALIGN_LEFT
table.column_alignments['services'] = BeautifulTable.ALIGN_RIGHT
row=cur.fetchall()
for r in row:
table.append_row(r)
print(table)
I am trying to execute the following script. but I don't get neither the desired results nor a error message ,and I can't figure out where I'm doing wrong.
import pyodbc
cnxn = pyodbc.connect("Driver={SQL Server Native Client 11.0};"
"Server=mySRVERNAME;"
"Database=MYDB;"
"uid=sa;pwd=MYPWD;"
"Trusted_Connection=yes;")
cursor = cnxn.cursor()
cursor.execute('select DISTINCT firstname,lastname,coalesce(middlename,\' \') as middlename from Person.Person')
for row in cursor:
print('row = %r' % (row,))
any ideas ? any help is appreciated :)
You have to use a fetch method along with cursor. For Example
for row in cursor.fetchall():
print('row = %r' % (row,))
EDIT :
The fetchall function returns all remaining rows in a list.
If there are no rows, an empty list is returned.
If there are a lot of rows, *this will use a lot of memory.*
Unread rows are stored by the database driver in a compact format and are often sent in batches from the database server.
Reading in only the rows you need at one time will save a lot of memory.
If we are going to process the rows one at a time, we can use the cursor itself as an interator
Moreover we can simplify it since cursor.execute() always returns a cursor :
for row in cursor.execute("select bla, anotherbla from blabla"):
print row.bla, row.anotherbla
Documentation
I found this information useful to retrieve data from SQL database to python as a data frame.
import pandas as pd
import pymssql
con = pymssql.connect(server='use-et-aiml-cloudforte-aiops- db.database.windows.net',user='login_username',password='login_password',database='database_name')
cursor = con.cursor()
query = "SELECT * FROM <TABLE_NAME>"
cursor.execute(query)
df = pd.read_sql(query, con)
con.close()
df
import mysql.connector as mc
connection creation
conn = mc.connect(host='localhost', user='root', passwd='password')
print(conn)
#create cursor object
cur = conn.cursor()
print(cur)
cur.execute('show databases')
for i in cur:
print(i)
query = "Select * from employee_performance.employ_mod_recent"
emp_data = pd.read_sql(query, conn)
emp_data