I'm new to Python and I wanted to ask you for help.
I want to put the data of a view in SQL Server into a table of my database in MySQL, when I try to give the following error:
Execution failed on sql: SELECT name FROM sqlite_master WHERE
type='table' AND name=?; not all arguments converted during string
formatting unable to rollback
Using Python version 3.7
Below is the code I use:
import pymysql.cursors
import pyodbc
import pandas as pd
# SQL Server Connection
connection = pyodbc.connect("DSN=SQLServer") #autocommit=True
try:
with connection.cursor() as cursor:
result = "SELECT * FROM dw.dbo.vW_sale"
df = pd.read_sql_query("SELECT * FROM dw.dbo.vW_sale", connection)
cursor.execute(result)
table = cursor.fetchall()
print(table)
finally:
connection.close()
# MySQL connection
cnx = pymysql.connect(host='test',
user='test',
password='test',
db='dw')
try:
with cnx.cursor() as cursor:
mysql = "select *from ft_sale_test"
cursor.execute(mysql)
result = cursor.fetchall()
#print(result)
finally:
cnx.close()
# using if_exists to handle the table that already exists
The error happens right here
df.to_sql(con=cnx, name= 'ft_sale_test', if_exists= 'replace')
Related
I have already searched for several solutions here and tried to get a working code. Everything works except for the where query.
In the where query I search for the highest value (numeric). However, this does not really work...
Here is my code and the structure of the MySQL database.
Thanks!
import pymysql
conn = pymysql.connect(host='localhost', unix_socket='', user='root', passwd='pw', db='database')
cur = conn.cursor()
cur.execute("SELECT * FROM dose")
for r in cur:
curr = conn.cursor()
sql = """UPDATE dose
SET status = "printed"
WHERE id = SELECT GREATEST (status) FROM dose (status);"""
# print(sql)
try:
# Execute the SQL command
curr.execute(sql)
# Commit your changes in the database
conn.commit()
except:
# Rollback in case there is any error
conn.rollback()
curr.close()
cur.close()
conn.close()
My SQL Database
You have a lot of things wrong in your code.
You don´t use the results of your first select query, and the only thing that you do is iterate over the results to execute an UPDATE
Your update query is wrong
You should change it to:
import pymysql
conn = pymysql.connect(host='localhost', unix_socket='', user='root', passwd='pw', db='database')
curr = conn.cursor()
sql = """UPDATE dose
SET status = 'printed'
WHERE id = (SELECT max(status) FROM dose) """
try:
# Execute the SQL command
curr.execute(sql)
# Commit your changes in the database
conn.commit()
except:
# Rollback in case there is any error
conn.rollback()
curr.close()
conn.close()
I am getting below error
query = command % processed_params TypeError: not all arguments
converted during string formatting
I am trying to pull data from SQL server and then inserting it into Snowflake
my below code
import pyodbc
import sqlalchemy
import snowflake.connector
driver = 'SQL Server'
server = 'tanmay'
db1 = 'testing'
tcon = 'no'
uname = 'sa'
pword = '123'
cnxn = pyodbc.connect(driver='{SQL Server}',
host=server, database=db1, trusted_connection=tcon,
user=uname, password=pword)
cursor = cnxn.cursor()
cursor.execute("select * from Admin_tbldbbackupdetails")
rows = cursor.fetchall()
#for row in rows:
# #data = [(row[0], row[1],row[2], row[3],row[4], row[5],row[6], row[7])]
print (rows[0])
cnxn.commit()
cnxn.close()
connection = snowflake.connector.connect(user='****',password='****',account='*****')
cursor2 = connection.cursor()
cursor2.execute("USE WAREHOUSE FOOD_WH")
cursor2.execute("USE DATABASE Test")
sql1="INSERT INTO CN_RND.Admin_tbldbbackupdetails_ip"
"(id,dbname, dbpath, backupdate, backuptime, backupStatus, FaildMsg, Backupsource)"
"values (?,?,?,?,?,?,?,?)"
cursor2.execute(sql1,*rows[0])
It's obviously string parsing error.
You missed to provide parameter to %s printout.
If you cannot fix it step back and try another approach.
Use another script to achieve the same and get back to you bug tomorrow :-)
My script is doing pretty much the same:
1. Connect to SQL Server
-> fetchmany
-> multipart upload to s3
-> COPY INTO Snowflake table
Details are here: Snowpipe-for-SQLServer
I am trying to connect to Oracle through Python and trying to execute a few DDL & DML statements. Please help how it can be done
a simple query
import cx_Oracle
con = cx_Oracle.connect('pythonhol/welcome#127.0.0.1/orcl')
cur = con.cursor()
cur.execute('select * from departments order by department_id')
for result in cur:
print result
cur.close()
con.close()
You can do something like this:
import cx_Oracle
connection = cx_Oracle.connect("username", "password", "ip of your server"/"database name")
cursor = connection.cursor()
cursor.execute("select * from table_name")
for row in cursor:
print(row)
connection.close()
I am attempting to great a temporary table in an SQL database and populate the table from a pandas dataframe. I am receiving an error when using the df.to_sql to populate the temp table. Thank you for the assistance.
import pandas as pd
from sqlalchemy import create_engine
import pandas.io.sql as psql
import urllib
params = urllib.quote_plus("DRIVER={SQL Server};SERVER=ServerAddressHere;DATABASE=DatabaseNameHere;Trusted_Connection=yes")
engine = create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
connection = engine.connect()
resoverall = connection.execute('''SELECT DISTINCT
a.CountryRegionID AS ISO_Short,
b.Name
FROM
CustTable AS a
LEFT JOIN AddressCountryRegion AS b
ON b.CountryRegionID = a.CountryRegionID''')
Countries= pd.DataFrame(resoverall.fetchall())
Countries.columns = resoverall.keys()
Countries= pd.Countries['ISO_Short'].str.upper()
Countries= pd.DataFrame(data=Countries)
temp = connection.execute('''
create table #tempTable
(
ISO_Short varchar(5)
)
''')
Countries.to_sql('Countries',engine)
The error I'm receiving is:
ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]CREATE TABLE permission denied in database 'databasename'. (262) (SQLExecDirectW)") [SQL: u'\nCREATE TABLE [Countries] (\n\t[index] BIGINT NULL, \n\t[ISO_Short] VARCHAR(max) NULL\n)\n\n'
UPDATE:
The other option I thought of is to use Pyodbc and convert Countries to a dictionary and then pass the dictionary values into the temporary table. Using this method, everything works until I try and pass the dictionary to the temp table. I have the following code using this approach:
import pandas as pd
import pyodbc
import pandas.io.sql as psql
cnxn = pyodbc.connect('''DRIVER={SQL Server};SERVER=telsmith;
DATABASE=DatabaseNameHere;Trusted_Connection=yes;''')
cursor = cnxn.cursor()
Countries= '''
SELECT DISTINCT
a.CountryRegionID AS ISO_Short,
b.Name
FROM
CustTable AS a
LEFT JOIN AddressCountryRegion AS b
ON b.CountryRegionID = a.CountryRegionID
'''
Countries= psql.read_sql(Countries, cnxn)
Countries= Countries['ISO_Short'].str.upper()
Countries= pd.DataFrame(data=Countries)
Countriesdict = Countries.to_dict()
Temp = '''
create table #tempTable
(
ISO_Short varchar(5)
)
'''
cnxn.commit()
# This is where I run into difficulty
placeholders = ', '.join(['%s'] * len(Countriesdict ))
columns = ', '.join(Countriesdict .keys())
sql = "INSERT INTO #tempTable VALUES ( %s )" % (placeholders)
cursor.execute(sql, Countriesdict.values())
This might sound little dumb but look at the error:
ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]CREATE TABLE permission denied in database 'databasename'. (262) (SQLExecDirectW)") [SQL: u'\nCREATE TABLE [Countries] (\n\t[index] BIGINT NULL, \n\t[ISO_Short] VARCHAR(max) NULL\n)\n\n'
Do you have any database called databasename? Since It can't find database, it can't create table. I ran the same code and it worked just fine. I believe that's the reason
Not strictly a SQLAlchemy concern. You need to obtain "CREATE TABLE" permissions on the server from your DBA for some username and password, and use those credentials to access your DB. Try including "UID=uname;PWD=pword;" in your params for some set of permissioned credentials.
I might have a solution that worked for me:
from sqlalchemy import create_engine
import urllib
params = urllib.parse.quote_plus("DRIVER={SQL Server};SERVER=10.233.6.52;DATABASE=databaseName;UID=xxx;PWD=Welcome1!")
engine = create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
connection = engine.connect()
df.to_sql('tempTable',engine)
I have an SQL database and am wondering what command you use to just get a list of the table names within that database.
To be a bit more complete:
import MySQLdb
connection = MySQLdb.connect(
host = 'localhost',
user = 'myself',
passwd = 'mysecret') # create the connection
cursor = connection.cursor() # get the cursor
cursor.execute("USE mydatabase") # select the database
cursor.execute("SHOW TABLES") # execute 'SHOW TABLES' (but data is not returned)
now there are two options:
tables = cursor.fetchall() # return data from last query
or iterate over the cursor:
for (table_name,) in cursor:
print(table_name)
SHOW tables
15 chars
show tables will help. Here is the documentation.
It is also possible to obtain tables from a specific scheme with execute the single query with the driver below.
python3 -m pip install PyMySQL
import pymysql
# Connect to the database
conn = pymysql.connect(host='127.0.0.1',user='root',passwd='root',db='my_database')
# Create a Cursor object
cur = conn.cursor()
# Execute the query: To get the name of the tables from a specific database
# replace only the my_database with the name of your database
cur.execute("SELECT table_name FROM information_schema.tables WHERE table_schema = 'my_database'")
# Read and print tables
for table in [tables[0] for tables in cur.fetchall()]:
print(table)
output:
my_table_name_1
my_table_name_2
my_table_name_3
...
my_table_name_x