Develop Reference

cx_Oracle Get Boolean Return Value

I have been working hard all day attempting to get a boolean value from a PL/SQL function using cx_Oracle. I've seen posts talking about using some other data type like char or integer to store the return value, but when I attempt to use such solutions, I get an incorrect data type error. First, let me show the code.
def lives_on_campus(self):
cursor = conn.cursor()
ret = cursor.callfunc('students_api.lives_on_campus', bool, [self.pidm])
return ret
If I use the 11.2.0.4 database client, I get the following error.
File "student-extracts.py", line 134, in <module>
if student.lives_on_campus():
File "student-extracts.py", line 58, in lives_on_campus
ret = cursor.callfunc('students_api.lives_on_campus', bool, [self.pidm])
cx_Oracle.DatabaseError: DPI-1050: Oracle Client library is at version 11.2 but version 12.1 or higher is needed
If I use the 12.1.0.2 database client or later, I get this error.
Traceback (most recent call last):
File "student-extracts.py", line 134, in <module>
if student.lives_on_campus():
File "student-extracts.py", line 58, in lives_on_campus
ret = cursor.callfunc('students_api.lives_on_campus', bool, [self.pidm])
cx_Oracle.DatabaseError: ORA-03115: unsupported network datatype or representation
Basically, it errors out no matter which version of the SQL Client I use. Now, I know the above code will work if the database version is 12c R2. Unfortunately, we only have that version in our TEST environment and PROD uses only the 11g database. Is there any I can make that function work with an 11g database? There must be a workaround.
~ Bob

Try a wrapper anonymous block like:
with connection.cursor() as cursor:
outVal = cursor.var(int)
sql="""
begin
:outVal := sys.diutil.bool_to_int(students_api.lives_on_campus(:pidm));
end;
"""
cursor.execute(sql, outVal=outVal, pidm='123456')
print(outVal.getvalue())

mysql-connector-python InterfaceError: Failed getting warnings when executing a query with multiple statements with get_warnings=True

Using Python 3.7, I execute a query against a MySQL database, with multiple statements, with get_warnings enabled:
import mysql.connector
cnx = mysql.connector.connect(host='xxx',
user='xxx',
password='xxx',
database='xxx',
use_pure=False,
get_warnings=True)
# Test 1, works:
cur = cnx.cursor()
cur.execute('SELECT "a"+1')
for row in cur:
print(row)
print(cur.fetchwarnings())
cur.close()
# Test 2, InterfaceError:
cur = cnx.cursor()
for rs in cur.execute('SELECT "a"+1; SELECT 2', multi=True):
for row in rs:
print(row)
print(rs.fetchwarnings())
The first test executes a single statement, iterates over the cursor, fetches data, and finally prints warnings. Output as expected:
(1.0,)
[('Warning', 1292, "Truncated incorrect DOUBLE value: 'a'")]
The second test, (you can remove the first test altogether), will execute print(row) once, then an Exception happens. Output:
Traceback (most recent call last):
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\connection_cext.py", line 472, in cmd_query
raw_as_string=raw_as_string)
_mysql_connector.MySQLInterfaceError: Commands out of sync; you can't run this command now
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\cursor_cext.py", line 138, in _fetch_warnings
_ = self._cnx.cmd_query("SHOW WARNINGS")
File "C:\Program Files\Python37\lib\site-packages\mysql\connector\connection_cext.py", line 475, in cmd_query
sqlstate=exc.sqlstate)
mysql.connector.errors.DatabaseError: 2014 (HY000): Commands out of sync; you can't run this command now
During handling of the above exception, another exception occurred:
....etc....
Did anyone encounter the same problem? How did you solve it? What am I doing wrong? Could this be a bug in the connector?
Other things I've tried:
If you set get_warnings to False, no error happens and
fetchwarnings() returns None
If you remove the problem from the SQL code, no error happens and fetchwarnings() returns None
use_pure can be True or False, the only difference is a slightly different traceback
Using fetchall() instead of for row in rs gives the same result
Many other variations give the same error.
System:
Connector version is mysql-connector-python-8.0.17 but 8.0.16 has the same issue.
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] on win32
MySQL 5.7

The "Commands out of sync" is because MySQL client interface calls are performed in a wrong order. This is not a bug in the connector. This is expected behavior.
Executing that first SELECT returns a MySQL resultset.
Before the client issues another statement that returns a MySQL resultset, we have to do something with the resultset that is already returned. That is, there needs to be calls to either mysql_use_result and mysql_free_result, or a call to mysql_store_result. Once the client does that, then the client can execute another SQL statement that returns a result.
(Note that the execution of the MySQL SHOW WARNINGS statement returns a MySQL resultset.)
Again, this is expected behavior, as documented here:
https://dev.mysql.com/doc/refman/8.0/en/commands-out-of-sync.html
The references to mysql_free_result, mysql_store_result and mysql_use_result aren't specific to a Python interface; these reference the underlying library routines in the MySQL client code. e.g. https://dev.mysql.com/doc/refman/8.0/en/mysql-use-result.html
FOLLOWUP
I suspect the author of the MySQL Python connector didn't anticipate this use case, or if it was anticipated, the observed behavior was judged to be correct.
As far as avoiding the problem, I would avoid the use of the multii=True and do a separate execute for each SQL statement. Following the same pattern as in Test 1, we could add an outer loop to loop through the SQL statements
# Test 1.2
sqls = ['SELECT "a"+1', 'SELECT 2', ]
for sql in sqls:
cur = cnx.cursor()
cur.execute(sql)
for row in cur:
print(row)
print(cur.fetchwarnings())
cur.close()
Another option would be to avoid the call to the fetchwarnings. That is what is causing the SHOW WARNINGS statement to be executed (only after it first verifies that the count of warnings is greater than zero.) We can issue a SHOW WARNINGS statement separately, and loop through the results from that like it were the return from a SELECT.
# Test 1.3
cur = cnx.cursor()
for rs in cur.execute('SELECT "a"+1; SHOW WARNINGS; SELECT 2; SHOW WARNINGS', multi=True):
for row in rs:
print(row)
cur.close()

I get NotImplementedError when trying to do a prepared statement with mysql python connector

I want to use prepared statements to insert data into a MySQL DB (version 5.7) using python, but I keep getting a NotImplementedError.
I'm following the documentation here: https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursorprepared.html
Using Python 2.7 and version 8.0.11 of mysql-connector-python library:
pip show mysql-connector-python
---
Metadata-Version: 2.1
Name: mysql-connector-python
Version: 8.0.11
Summary: MySQL driver written in Python
Home-page: http://dev.mysql.com/doc/connector-python/en/index.html
This is a cleaned version (no specific hostname, username, password, columns, or tables) of the python script I'm running:
import mysql.connector
from mysql.connector.cursor import MySQLCursorPrepared
connection = mysql.connector.connect(user=username, password=password,
host='sql_server_host',
database='dbname')
print('Connected! getting cursor')
cursor = connection.cursor(cursor_class=MySQLCursorPrepared)
select = "SELECT * FROM table_name WHERE column1 = ?"
param = 'param1'
print('Executing statement')
cursor.execute(select, (param,))
rows = cursor.fetchall()
for row in rows:
value = row.column1
print('value: '+ value)
I get this error when I run this:
Traceback (most recent call last):
File "test.py", line 18, in <module>
cursor.execute(select, (param,))
File "/home/user/.local/lib/python2.7/site-packages/mysql/connector/cursor.py", line 1186, in execute
self._prepared = self._connection.cmd_stmt_prepare(operation)
File "/home/user/.local/lib/python2.7/site-packages/mysql/connector/abstracts.py", line 969, in cmd_stmt_prepare
raise NotImplementedError
NotImplementedError

CEXT will be enabled by default if you have it, and prepared statements are not supported in CEXT at the time of writing.
You can disable the use of CEXT when you connect by adding the keyword argument use_pure=True as follows:
connection = mysql.connector.connect(user=username, password=password,
host='sql_server_host',
database='dbname',
use_pure=True)
Support for prepared statements in CEXT will be included in the upcoming mysql-connector-python 8.0.17 release (according to the MySQL bug report). So once that is available, upgrade to at least 8.0.17 to solve this without needing use_pure=True.

Error when importing CSV to postgres with python and psycopg2

I try to COPY a CSV file from a folder to a postgres table using python and psycopg2 and I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
psycopg2.ProgrammingError: must be superuser to COPY to or from a file
HINT: Anyone can COPY to stdout or from stdin. psql's \copy command also works for anyone.
I also tried to run it through the python environment as:
constr = "dbname='db_name' user='user' host='localhost' password='pass'"
conn = psycopg2.connect(constr)
cur = conn.cursor()
sqlstr = "COPY test_2 FROM '/tmp/tmpJopiUG/downloaded_xls.csv' DELIMITER ',' CSV;"
cur.execute(sqlstr)
I still get the above error. I tried \copy command but this works only in psql. What is the alternative in order to be able to execute this through my python script?
EDITED
After having a look in the link provided by #Ilja Everilä I tried this:
cur.copy_from('/tmp/tmpJopiUG/downloaded_xls.csv', 'test_copy')
I get an error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: argument 1 must have both .read() and .readline() methods
How do I give these methods?

Try using cursor.copy_expert():
constr = "dbname='db_name' user='user' host='localhost' password='pass'"
conn = psycopg2.connect(constr)
cur = conn.cursor()
sqlstr = "COPY test_2 FROM STDIN DELIMITER ',' CSV"
with open('/tmp/tmpJopiUG/downloaded_xls.csv') as f:
cur.copy_expert(sqlstr, f)
conn.commit()
You have to open the file in python and pass it to psycopg, which then forwards it to postgres' stdin. Since you're using the CSV argument to COPY, you have to use the expert version in which you pass the COPY statement yourself.

You can also use copy_from. See the code below
with open('/tmp/tmpJopiUG/downloaded_xls.csv') as f:
cur.copy_from(f, table_name,sep=',')
conn.commit()

Python Mysql CREATE and INSERT in one query

I am using Python to connect to MySQL in XAMPP.I am creating a string and passing it to a new process through queue.The string contains the MySQL query which on execution will create a table and insert into the table.following is my code:-
from MySQLdb import connect
from os import _exit
from multiprocessing import Process,Queue
q=Queue()
def pkt():
conn=connect(user='root',passwd='257911.',host='localhost',unix_socket="/opt/lampp/var/mysql/mysql.sock")
cursor=conn.cursor()
conn.select_db("try")
while True:
y=q.get()
if y=="exit":
break
else:
cursor.execute(y)
conn.commit()
cursor.close()
conn.close()
_exit(0)
if __name__=="__main__":
a=Process(target=pkt)
a.start()
query="CREATE TABLE hello(id varchar(10) NOT NULL,name varchar(20)); INSERT INTO hello(id,name) VALUES('1234','sujata'); "
q.put(query)
q.put("exit")
Upon executing the code, I am getting the following error:-
Traceback (most recent call last):
File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "try.py", line 16, in pkt
conn.commit()
ProgrammingError: (2014, "Commands out of sync; you can't run this command now")
I am getting the same error on inserting into multiple tables in one query.Is it not possible to combine create and insert in one statement??
Thanks.

No, you cannot give multiple semicolon separated queries in the same line. This is because MySQLdb is a wrapper around _mysql which makes it compatible with the Python DB API interface (Read more about the API here). The API works via PREPARE/EXECUTE statements and according to this,
The text must represent a single statement, not multiple statements.

I need to close the cursor after execution. This works for me.
from MySQLdb import connect
from os import _exit
from multiprocessing import Process,Queue
q=Queue()
def pkt():
conn=connect(user='root',host='localhost',unix_socket="/home/mysql/mysql/mysql.sock")
cursor=conn.cursor()
conn.select_db("try")
while True:
y=q.get()
if y=="exit":
break
else:
cursor.execute(y)
cursor.close()
conn.commit()
conn.close()
_exit(0)
if __name__=="__main__":
a=Process(target=pkt)
a.start()
query="CREATE TABLE hello(id varchar(10) NOT NULL,name varchar(20)); INSERT INTO hello(id,name) VALUES('1234','sujata'); "
q.put(query)
q.put("exit")