This might be an obvious error but I'm trying to create a database within python from a script I've already created.
conn = sqlite3.connect('testDB')
c = conn.cursor()
c.execute('.read create.sql')
This gives an error "sqlite3.OperationalError: near ".": syntax error"
If I do the same thing at the sqlite3 cmd line it works fine
[me#myPC ~]$ sqlite3 testDB
SQLite version 3.3.6
Enter ".help" for instructions
sqlite> .read create.sql
sqlite>
It seems that any commands that start with a . give me problems.
just pass the content of the file to the .execute method:
conn = sqlite3.connect('testDB')
c = conn.cursor()
SQL = open('create.sql').read()
c.executescript(SQL)
I would suppose that commands starting with . are for the CLI client itself, not for the backend.
So you have no chance to do so and would have to do file reading and executing the queries by yourself, i.e. in Python.
Related
I am trying to run a python script from VB.Net using IronPython. So far, I have installed Python and IronPython. I have the ExecPython method shown below. It works fine when I call a simple print/hello world type of script. This DBTest.py script is just using pyodbc and connecting to the database and executing a basic select query.
The error I get at the source.Execute(scope) line is "IronPython.Runtime.Exceptions.ImportException: 'No module named 'pyodbc''"
I've installed pyodbc using pip install pyodbc. The DBTest.py script runs fine when I run it with IDLE.
I'm not sure if this is a limitation or if there's something I'm missing in the setup.
Thanks in advance for your help!!
Sub ExecPython(ByVal argv As List(Of String))
Dim engine As ScriptEngine = Python.CreateEngine
Dim scriptPath As String = "C:\scripts\DBTest.py"
Dim source As ScriptSource = engine.CreateScriptSourceFromFile(scriptPath)
argv.Add("")
engine.GetSysModule.SetVariable("argv", argv)
engine.SetSearchPaths({"C:\Users\MYUSERNAME\AppData\Local\Programs\Python\Python39\Scripts",
"C:\Users\MYUSERNAME\AppData\Local\Programs\Python\Python39\include",
"C:\Users\MYUSERNAME\AppData\Local\Programs\Python\Python39\Lib",
"C:\Program Files\IronPython 3.4\Lib"})
Dim eIO As ScriptIO = engine.Runtime.IO
Dim errors As New MemoryStream
eIO.SetErrorOutput(errors, Text.Encoding.Default)
Dim results As New MemoryStream
eIO.SetOutput(results, Text.Encoding.Default)
Dim scope As ScriptScope = engine.CreateScope
source.Execute(scope)
Console.WriteLine("ERRORS:")
Console.WriteLine(FormatResult(errors.ToArray))
Console.WriteLine("")
Console.WriteLine("RESULTS:")
Console.WriteLine(FormatResult(results.ToArray))
End Sub
Here is the python script that I am calling. It runs when I run the module from IDLE.
import pyodbc
conn = pyodbc.connect('Driver={SQL Server};'
'Server=MYSERVERNAME;'
'Database=MYDBNAME;'
'Trusted_Connection=yes;')
cursor = conn.cursor() cursor.execute('SELECT * FROM dbo.TABLENAME')
for row in cursor:
print(row)
It's been a while, probably you already found a solution.
But in general, for being able to use the 'module' you need to import clr and use the addReference function. Then you must do an import of the namespace with the python style.
Example:
import clr
clr.AddReference('System')
from System import *
Using Python 3.7, I execute a query against a MySQL database, with multiple statements, with get_warnings enabled:
import mysql.connector
cnx = mysql.connector.connect(host='xxx',
user='xxx',
password='xxx',
database='xxx',
use_pure=False,
get_warnings=True)
# Test 1, works:
cur = cnx.cursor()
cur.execute('SELECT "a"+1')
for row in cur:
print(row)
print(cur.fetchwarnings())
cur.close()
# Test 2, InterfaceError:
cur = cnx.cursor()
for rs in cur.execute('SELECT "a"+1; SELECT 2', multi=True):
for row in rs:
print(row)
print(rs.fetchwarnings())
The first test executes a single statement, iterates over the cursor, fetches data, and finally prints warnings. Output as expected:
(1.0,)
[('Warning', 1292, "Truncated incorrect DOUBLE value: 'a'")]
The second test, (you can remove the first test altogether), will execute print(row) once, then an Exception happens. Output:
Traceback (most recent call last):
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\connection_cext.py", line 472, in cmd_query
raw_as_string=raw_as_string)
_mysql_connector.MySQLInterfaceError: Commands out of sync; you can't run this command now
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\cursor_cext.py", line 138, in _fetch_warnings
_ = self._cnx.cmd_query("SHOW WARNINGS")
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\connection_cext.py", line 475, in cmd_query
sqlstate=exc.sqlstate)
mysql.connector.errors.DatabaseError: 2014 (HY000): Commands out of sync; you can't run this command now
During handling of the above exception, another exception occurred:
....etc....
Did anyone encounter the same problem? How did you solve it? What am I doing wrong? Could this be a bug in the connector?
Other things I've tried:
If you set get_warnings to False, no error happens and
fetchwarnings() returns None
If you remove the problem from the SQL code, no error happens and fetchwarnings() returns None
use_pure can be True or False, the only difference is a slightly different traceback
Using fetchall() instead of for row in rs gives the same result
Many other variations give the same error.
System:
Connector version is mysql-connector-python-8.0.17 but 8.0.16 has the same issue.
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] on win32
MySQL 5.7
The "Commands out of sync" is because MySQL client interface calls are performed in a wrong order. This is not a bug in the connector. This is expected behavior.
Executing that first SELECT returns a MySQL resultset.
Before the client issues another statement that returns a MySQL resultset, we have to do something with the resultset that is already returned. That is, there needs to be calls to either mysql_use_result and mysql_free_result, or a call to mysql_store_result. Once the client does that, then the client can execute another SQL statement that returns a result.
(Note that the execution of the MySQL SHOW WARNINGS statement returns a MySQL resultset.)
Again, this is expected behavior, as documented here:
https://dev.mysql.com/doc/refman/8.0/en/commands-out-of-sync.html
The references to mysql_free_result, mysql_store_result and mysql_use_result aren't specific to a Python interface; these reference the underlying library routines in the MySQL client code. e.g. https://dev.mysql.com/doc/refman/8.0/en/mysql-use-result.html
FOLLOWUP
I suspect the author of the MySQL Python connector didn't anticipate this use case, or if it was anticipated, the observed behavior was judged to be correct.
As far as avoiding the problem, I would avoid the use of the multii=True and do a separate execute for each SQL statement. Following the same pattern as in Test 1, we could add an outer loop to loop through the SQL statements
# Test 1.2
sqls = ['SELECT "a"+1', 'SELECT 2', ]
for sql in sqls:
cur = cnx.cursor()
cur.execute(sql)
for row in cur:
print(row)
print(cur.fetchwarnings())
cur.close()
Another option would be to avoid the call to the fetchwarnings. That is what is causing the SHOW WARNINGS statement to be executed (only after it first verifies that the count of warnings is greater than zero.) We can issue a SHOW WARNINGS statement separately, and loop through the results from that like it were the return from a SELECT.
# Test 1.3
cur = cnx.cursor()
for rs in cur.execute('SELECT "a"+1; SHOW WARNINGS; SELECT 2; SHOW WARNINGS', multi=True):
for row in rs:
print(row)
cur.close()
I want to use prepared statements to insert data into a MySQL DB (version 5.7) using python, but I keep getting a NotImplementedError.
I'm following the documentation here: https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursorprepared.html
Using Python 2.7 and version 8.0.11 of mysql-connector-python library:
pip show mysql-connector-python
---
Metadata-Version: 2.1
Name: mysql-connector-python
Version: 8.0.11
Summary: MySQL driver written in Python
Home-page: http://dev.mysql.com/doc/connector-python/en/index.html
This is a cleaned version (no specific hostname, username, password, columns, or tables) of the python script I'm running:
import mysql.connector
from mysql.connector.cursor import MySQLCursorPrepared
connection = mysql.connector.connect(user=username, password=password,
host='sql_server_host',
database='dbname')
print('Connected! getting cursor')
cursor = connection.cursor(cursor_class=MySQLCursorPrepared)
select = "SELECT * FROM table_name WHERE column1 = ?"
param = 'param1'
print('Executing statement')
cursor.execute(select, (param,))
rows = cursor.fetchall()
for row in rows:
value = row.column1
print('value: '+ value)
I get this error when I run this:
Traceback (most recent call last):
File "test.py", line 18, in <module>
cursor.execute(select, (param,))
File "/home/user/.local/lib/python2.7/site-packages/mysql/connector/cursor.py", line 1186, in execute
self._prepared = self._connection.cmd_stmt_prepare(operation)
File "/home/user/.local/lib/python2.7/site-packages/mysql/connector/abstracts.py", line 969, in cmd_stmt_prepare
raise NotImplementedError
NotImplementedError
CEXT will be enabled by default if you have it, and prepared statements are not supported in CEXT at the time of writing.
You can disable the use of CEXT when you connect by adding the keyword argument use_pure=True as follows:
connection = mysql.connector.connect(user=username, password=password,
host='sql_server_host',
database='dbname',
use_pure=True)
Support for prepared statements in CEXT will be included in the upcoming mysql-connector-python 8.0.17 release (according to the MySQL bug report). So once that is available, upgrade to at least 8.0.17 to solve this without needing use_pure=True.
I am very new to python and I just can't seem to find an answer to this error. When I run the code below I get the error
AttributeError: module 'odbc' has no attribute 'connect'
However, the error only shows in eclipse. There's no problem if I run it via command line. I am running python 3.5. What am I doing wrong?
try:
import pyodbc
except ImportError:
import odbc as pyodbc
# Specifying the ODBC driver, server name, database, etc. directly
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=PXLstr,17;DATABASE=Dept_MR;UID=guest;PWD=password')
The suggestion to remove the try...except block did not work for me. Now the actual import is throwing the error as below:
Traceback (most recent call last):
File "C:\Users\a\workspace\TestPyProject\src\helloworld.py", line 2, in <module>
import pyodbc
File "C:\Users\a\AppData\Local\Continuum\Anaconda3\Lib\site-packages\sqlalchemy\dialects\mssql\pyodbc.py", line 105, in <module>
from .base import MSExecutionContext, MSDialect, VARBINARY
I do have pyodbc installed and the import and connect works fine with the command line on windows.
thank you
The problem here is that the pyodbc module is not importing in your try / except block. I would highly recommend not putting import statements in try blocks. First, you would want to make sure you have pyodbc installed (pip install pyodbc), preferably in a virtualenv, then you can do something like this:
import pyodbc
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=PXLstr,17;DATABASE=Dept_MR;UID=guest;PWD=password')
cursor = cnxn.cursor()
cursor.execute('SELECT 1')
for row in cursor.fetchall():
print(row)
If you're running on Windows (it appears so, given the DRIVER= parameter), take a look at virtualenvwrapper-win for managing Windows Python virtual environments: https://pypi.python.org/pypi/virtualenvwrapper-win
Good luck!
Flipper's answer helped to establish that the problem was with referencing an incorrect library in External Libraries list in eclipse. After fixing it, the issue was resolved.
What is the name of your python file? If you inadvertently name it as 'pyodbc.py', you got that error. Because it tries to import itself instead of the intended pyodbc module.
here is the solution!
simply install and use 'pypyodbc' instead of 'pyodbc'!
I have my tested example as below. change your data for SERVER_NAME and DATA_NAME and DRIVER. also put your own records.good luck!
import sys
import pypyodbc as odbc
records = [
['x', 'Movie', '2020-01-09', 2020],
['y', 'TV Show', None, 2019]
]
DRIVER = 'ODBC Driver 11 for SQL Server'
SERVER_NAME = '(LocalDB)\MSSQLLocalDB'
DATABASE_NAME = 'D:\ASPNET\SHOJA.IR\SHOJA.IR\APP_DATA\DATABASE3.MDF'
conn_string = f"""
Driver={{{DRIVER}}};
Server={SERVER_NAME};
Database={DATABASE_NAME};
Trust_Connection=yes;
"""
try:
conn = odbc.connect(conn_string)
except Exception as e:
print(e)
print('task is terminated')
sys.exit()
else:
cursor = conn.cursor()
insert_statement = """
INSERT INTO NetflixMovies
VALUES (?, ?, ?, ?)
"""
try:
for record in records:
print(record)
cursor.execute(insert_statement, record)
except Exception as e:
cursor.rollback()
print(e.value)
print('transaction rolled back')
else:
print('records inserted successfully')
cursor.commit()
cursor.close()
finally:
if conn.connected == 1:
print('connection closed')
conn.close()
I want to use a variable in my SQL script. Variable's value is 100 (just a number). I have stored it as a csv. file in this directory: C:\Users\Dino\Desktop\my_file.csv.
I want in the sql script to run this:
import os
from ask_db import ask_db_params #this script creates the connection to the database
import sys, os
def my_function(cur, conn):
sql="""
\set outputdir'C:\\Users\\Dino\\Desktop'
\set new_var :outputdir '\\my_file.csv'
copy my_file to :'new_var';"""
cur.execute(sql)
conn.commit()
if __name__ == '__main__':
try:
conn = ask_db_params()
cur = conn.cursor()
analysis_data(cur,conn)
logging.info('Data analysed.')
except Exception as e:
logging.error('Failure', exc_info=True)
exit(1)
I have the error:
syntax error at or near "\" ....
It refers to the first line.
Any help regarding the syntax?
P.S. I m running python to call the sql script. Windows OS
That won't work. \set is a psql command, not an SQL command.
You will have to use string manipulation in Python to construct an SQL string that looks like this:
COPY my_file TO 'C:\Users\Dino\Desktop\my_file.csv'
Then use execute() with that SQL string.