I have a python script, which read a sql file and execute the sql command stored in it. But when executing it I got below error:
psycopg2.ProgrammingError: syntax error at or near "select"
LINE 1: select * from image
the sql file content is:
select * from image
which is simple and should be correct.
the code throwing the error(the last line, more specificately):
cur=conn.cursor()
string=open(script,'r',encoding='utf-8').read()#script is the sql file
cur.execute(string)
is there anyone can advise?
----update-----
below is the function in the python script. I don't post the whole script since it is too long.
def list_(csv, sql=None , script=None , host = None, dbname=None , user=None , pwd=None):
print(sql)
print(script)
if (sql):
print("sql")
with conn2db(host,dbname,user,pwd) as conn:
cur = conn.cursor()
cur.execute(sql)
if (script):
print("script")
with conn2db(host,dbname, user, pwd) as conn:
cur = conn.cursor()
string = open(script, 'r', encoding='utf-8').read()
print(string)
cur.execute(string)
#cur.execute(open(script, 'r', encoding='utf-8').read())
with open(csv,'w') as file:
for record in cur:
mystr=str(record)[1:-2] if str(record)[-1]==',' else str(record)[1:-1]
file.write(mystr+'\n')
#file.write('\n')
Seeing the connection string as well as your schemas and tables in your database would help. Please confirm that these are all correct. Additionally, running .strip() on the SQL string after reading it from the file or adding a semi-colon to the end of the SQL string is worth a try.
Today I hit this issue again, and I deleted the original file, created a new one, and typed the sql command. Everything works like a charm now.
My guess is that the original file contains some invisible characters which caused this issue. But why they exist there still puzzled me.
Related
import sql_connect
def main():
# check if db exists on target, if not create
qry_create_db = "if not exists(select * from sys.databases where name = '{}') create database {};".format('mydb','mydb')
with sql_connect.conn:
cur1 = sql_connect.cursor.execute(qry_create_db)
cur1.commit()
main()
def creation_table(filename):
# Open and read the file as a single buffer
fd = open(filename, 'r')
sqlFile = fd.read()
fd.close()
# all SQL commands (split on ';')
sqlCommands = sqlFile.split(';')
# Execute every command from the input file
for command in sqlCommands:
with sql_connect.conn:
cur2 = sql_connect.cursor.execute(command)
cur2.commit()
creation_table('Mypath\\schema\\TABLES\\TOSHBA.sql')
creation_table('Mypath\\schema\\TABLES\\TALM_TYPE.sql')
This my python code that creates my database and my tables. The problem is when I execute my function
"creation_table('Mypath\\schema\\TABLES\\TALM_TYPE.sql')"
I receive this error :
pyodbc.Error: ('HY090', '[HY090] [Microsoft][ODBC Driver Manager] Invalid string or buffer length (0) (SQLExecDirectW)')
This my first sql file script TOSHBA that I execute first with my function creation_table and I have no error with it :
USE mydb;
DROP TABLE IF EXISTS TOSHBA;
CREATE TABLE TOSHBA
(
TOSHBA_WORK_ID INT NOT NULL IDENTITY,
WORK_NAME NVARCHAR(200) NOT NULL,
CONSTRAINT PK_TOSHBA PRIMARY KEY (TOSHBA_WORK_ID)
);
And here the second sql script file with which I have the error:
USE mydb;
DROP TABLE IF EXISTS TALM_TYPE;
CREATE TABLE TALM_TYPE
(
TALM_ID INT NOT NULL IDENTITY,
TOSHBA_id INT NOT NULL,
TALM_NAME NVARCHAR(20) NOT NULL,
CONSTRAINT PK_TALM PRIMARY KEY (TALM_ID),
CONSTRAINT FKÖ¹_TOSHBA_id FOREIGN KEY (TOSHBA_id) REFERENCES TOSHBA (TOSHBA_WORK_ID)
);
Please help me to understand the error and to find the solution.
Python version : 3.7.3
Pyodbc version : 4.0.27
It appears removing the last ';' from the sql file solves this issue.
I'm not 100% on this but when I had this same issue and came here, I tried removing the last ';' from the sql script and it worked. So it seems that the .split() on the last ';' causes a blank line to be run through as a statement through the loop, whether there is another line there or not. As a blank line is not a valid sql statement.
In theory to work around this issue pragmatically you could either count the number of ; and only run the split that many times, or remove the last ; from the script before loading the statements. But both of these would be overload when you can just remove the last ';' in your sql file.
I have the following python code, it reads through a text file line by line and takes characters x to y of each line as the variable "Contract".
import os
import pyodbc
cnxn = pyodbc.connect(r'DRIVER={SQL Server};CENSORED;Trusted_Connection=yes;')
cursor = cnxn.cursor()
claimsfile = open('claims.txt','r')
for line in claimsfile:
#ldata = claimsfile.readline()
contract = line[18:26]
print(contract)
cursor.execute("USE calms SELECT XREF_PLAN_CODE FROM calms_schema.APP_QUOTE WHERE APPLICATION_ID = "+str(contract))
print(cursor.fetchall())
When including the line cursor.fetchall(), the following error is returned:
Programming Error: Previous SQL was not a query.
The query runs in SSMS and replace str(contract) with the actual value of the variable results will be returned as expected.
Based on the data, the query will return one value as a result formatted as NVARCHAR(4).
Most other examples have variables declared prior to the loop and the proposed solution is to set NO COUNT on, this does not apply to my problem so I am slightly lost.
P.S. I have also put the query in its own standalone file without the loop to iterate through the file in case this was causing the problem without success.
In your SQL query, you are actually making two commands: USE and SELECT and the cursor is not set up with multiple statements. Plus, with database connections, you should be selecting the database schema in the connection string (i.e., DATABASE argument), so TSQL's USE is not needed.
Consider the following adjustment with parameterization where APPLICATION_ID is assumed to be integer type. Add credentials as needed:
constr = 'DRIVER={SQL Server};SERVER=CENSORED;Trusted_Connection=yes;' \
'DATABASE=calms;UID=username;PWD=password'
cnxn = pyodbc.connect(constr)
cur = cnxn.cursor()
with open('claims.txt','r') as f:
for line in f:
contract = line[18:26]
print(contract)
# EXECUTE QUERY
cur.execute("SELECT XREF_PLAN_CODE FROM APP_QUOTE WHERE APPLICATION_ID = ?",
[int(contract)])
# FETCH ROWS ITERATIVELY
for row in cur.fetchall():
print(row)
cur.close()
cnxn.close()
I have a txt file with many mysql inserts (1.5 million).
I need to read this file with python and divide this file at each ';' for each query and run the query with python. How can I divide this file at each ';'? And run the query with python?
Until now my code is:
import MySQLdb
db = MySQLdb.connect(host = "localhost",
user="root",
passwd="da66ro",
db="test")
f = open('E:/estudos/projetos/tricae/tests_python.txt')
First open the file:
with open('youfilename.sql', 'r') as f:
fileAsString = f.read().replace("\n", "")
sqlStatements = fileAsString.split(";")
Then to run the query:
cursor = db.cursor()
for statement in sqlStatements:
try:
cursor.execute(statement)
db.commit()
except:
db.rollback()
But of course you must realize this is a terrible idea. What happens when you have a quoted ";" character in a string you are inserting? You'll have to be a bit more clever than what you posed as a question - in general it's a terrible idea to assume anything about any data you are inserting into a database.
Or even worse than a broken query: what about malicious code? SQL injection? Never trust input you haven't sanitized.
OK, so, first you need read the file and split it by ";", which is done with split() function. Then you can loop or select which queries to execute (or just execute the whole file without splitting). You can find numerous examples on each of these and I'm sure it will be easy enough to combine them in what you need.
I need to read this file with python and divide this file at
each ';' for each query and run the query with python.
I am new to python.. Here is the work around
import io
myfile = open('69_ptc_group_mappingfile_mysql.sql')
data = (myfile.read().decode("utf-8-sig").encode("utf-8")).lower()
query_list=[]
if 'delimiter' not in data:
query_list = (data.strip()).split(";")
else:
tempv = (data.rstrip()).split('delimiter')
for i in tempV:
if (i.strip()).startswith("//"):
i = i.rstrip().split("//")
for a in i:
if len(a)!=0:
query_list.append(a.strip())
else:
corr = ((i.rstrip()).split(";"))
for i in corr:
if len(i.rstrip())!=0:
query_list.append(i.rstrip())
print query_list
for j in query_list: cursor.execute(j)
I have this code in Python:
conn = sqlite3.connect("people.db")
cursor = conn.cursor()
sql = 'create table if not exists people (id integer, name VARCHAR(255))'
cursor.execute(sql)
conn.commit()
sql = 'insert into people VALUES (3, "test")'
cursor.execute(sql)
conn.commit()
sql = 'insert into people VALUES (5, "test")'
cursor.execute(sql)
conn.commit()
print 'Printing all inserted'
cursor.execute("select * from people")
for row in cursor.fetchall():
print row
cursor.close()
conn.close()
But seems is never saving to the database, there is always the same elements on the db as if it was not saving anything.
On the other side If I try to access the db file via sqlite it I got this error:
Unable to open database "people.db": file is encrypted or is not a database
I found on some other answers to use conn.commit instead of conn.commit() but is not changing the results.
Any idea?
BINGO ! people! I Had the same problem. One of thr reasons where very simple. I`am using debian linux, error was
Unable to open database "people.db": file is encrypted or is not a database
database file was in the same dir than my python script
connect line was
conn = sqlite3.connect('./testcases.db')
I changed this
conn = sqlite3.connect('testcases.db')
! No dot and slash.
Error Fixed. All works
If someone think it is usefull, you`re welcome
This seems to work alright for me ("In database" increases on each run):
import random, sqlite3
conn = sqlite3.connect("people.db")
cursor = conn.cursor()
sql = 'create table if not exists people (id integer, name VARCHAR(255))'
cursor.execute(sql)
for x in xrange(5):
cursor.execute('insert into people VALUES (?, "test")', (random.randint(1, 10000),))
conn.commit()
cursor.execute("select count(*) from people")
print "In database:", cursor.fetchone()[0]
You should commit after making changes i.e:
myDatabase.commit()
can you open the db with a tool like sqlite administrator ? this would proove thedb-format is ok.
if i search for that error the solutions point to version issues between sqlite and the db-driver used. maybe you can chrck your versions or AKX could post the working combination.
regards,khz
I have a problem with deleting a record from my SQLite3 database:
conn = sqlite3.connect('databaza.db')
c = conn.cursor()
data3 = str(input('Please enter name: '))
mydata = c.execute('DELETE FROM Zoznam WHERE Name=?', (data3,))
conn.commit()
c.close
Everything is OK, no errors, but the delete function doesn't work!
Does anyone have an idea?
The correct syntax for a parameterized query is:
mydata = c.execute("DELETE FROM Zoznam WHERE Name=?", (data3,))
Make sure the parameter uses the comma, to make it a python tuple.
This will help prevent SQL Injection which is possible when passing in a formatted string. More on SQL Injection here
Related post here.
I'm a little late to the party but if you Google search "python sqlite delete row" This is the first thing that comes up and I was having the same issue where things were not getting DELETE'd from my sqlite DB.
I'm using Python 2.7 on Debian Jessie.
Previously, when I wrote Python code for adding and retrieving information in the sqlite database, I had written the commands with correct capitalization where needed and it worked.
curs.execute("SELECT example_column1 FROM example_table WHERE example_column2=(?)", (Variable,))
However...
curs.execute("DELETE FROM example_table WHERE example_column1=(?)", (Variable,)):
This for some reason does not work with the DELETE command. I had to send that command in all lower-case before sqlite would respect the command being sent.
conn=sqlite3.connect('example.db')
curs=conn.cursor()
curs.execute("delete from example_table where example_column=(?)", (Variable,))
conn.commit()
conn.close()
I have no idea why.
I tried all the previously mentioned methods but having the command sent in lowercase was the only way I could get it to work.
Hope this helps any struggling neophytes in their journey into Python and sqlite.
Try with:
mydata = c.execute('DELETE FROM Zoznam WHERE Name = (?)', (data3))
Without the ',' and the '?' between '()'
I'm using Python 3.9 on windows.
c.execute("DELETE FROM Zoznam WHERE Name={0}".format(data3))
conn.commit()
I advise you to first make a string for the query , and then execute it. ex:
query = "delete from zoznam where name = '%s' " % data3
c.execute(query)
conn.commit()
Check the file permissions.
An aside, I prefer the tokenized method:
mydata = c.execute("DELETE FROM Zoznam WHERE Name='%s'" % data3)
Thank you to everybody who tried to help. The right code is:
conn = sqlite3.connect('databaza.db')
c = conn.cursor()
conn.text_factory = str
data3 = str(input('Please enter name: '))
query = "DELETE FROM Zoznam WHERE Name = '%s';" % data3.strip()
print(query)
mydata = c.execute(query)