Using Python and SQlite3 where c is a cursor this code...
print("vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv")
print("SQL and parameters:",sql,parm)
c.execute(sql,parm)
# Get the row
print("Executed OK")
response = c.fetchone()
# If not successful return null
if not response:
return None
#
print("and produced ", response)
print("^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^")
give this output:
vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
SQL and parameters: select * from Links where LinkNum = ? (301,)
Executed OK
and produced (301, 'Index', 'The Independent', 'https://www.independent.co.uk/', 6, 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
SQL and parameters: select * from Links where LinkNum = ? (301,)
Encountered exception of type ProgrammingError with arguments ('Incorrect number of bindings supplied. The current statement uses 1, and there are 6 supplied.',)
The application will close
Two identical statements. 1 works and the next throws the exception. As can be seen the row I'm trying to retrieve has 6 columns but that's the only hint/clue I can see. Can anyone help with tracking down the problem? Thanks.
Whatever was causing SQLite to have a fit I remedied the problem by retrieving the second row/object out side the Links object and passing it as an argument to the method I was calling, rather than have that method attempt to retrieve the object.
The problem would still be there but must be deep within the internals of Python instantiation and/or SQLite. Whatever, this problem is solved with some less fancy code.
Related
I'm learning BigQuery API using Python Client Libraries v0.28
https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/usage.html#run-a-simple-query
Wrote this simple code to fetch data from the table
1) Create client object
client_ = bigquery.Client.from_service_account_json('/Users/xyz/key.json')
2) Begin new Async query job
QUERY = 'SELECT visitid FROM `1234567.ga_sessions_20180101`'
query_job = client_.query(QUERY
, job_id=str(uuid.uuid4()))
3) poll until the query is DONE
while (query_job.state == 'RUNNING'):
time.sleep(5)
query_job.reload()
4) Fetch the results in iteration
query_job.reload()
iter = query_job.result()
At this stage I'd like to fetch how many rows are in the table. As per the doc GitHub code iter is of type bigquery.table.RowIterator with a property [tier.total_rows][1]
5) However, at this stage when I print:
print(iter.total_rows)
It keeps returning None
I'm pretty sure this table is NOT empty an dry query is correctly formatted!
Any help to any pointers what am I missing here will be really helpful... Thanks a lot!
Cheers!
You need to also check query_job.error_result to make sure query succeeded.
You can also see your job in the UI, which can be useful for debugging, using project id and job id:
https://bigquery.cloud.google.com/results/projectid:jobid
Also, query_job.result() already waits for the job completion so you don't need to poll.
The current behavior of how RowIterator returns None is indeed perplexing. Luckily, according to this issue, tswast's comment from 10 days ago indicates that the developers are working on a better solution.
Current awkward behavior of .total_rows
Currently, .total_rows is initialized only once iteration begins. (In what follows, for clarity I renamed your iter variable to row_iter.)
row_iter = query_job.result()
itr = iter(row_iter)
first_row = next(itr)
print(row_iter.total_rows) # Now you get a number instead of None.
This is ugly because to continue the iteration, we must either handle the first row differently or call row_iter = query_job.result() again.
Temporary workaround
A currently-working alternative is to use the value of query_job._query_results.total_rows. Unfortunately this is cheating because _query_results is private, so there is no reason to expect that this will work in the future.
Future behavior
If tswast's proposal is implemented, then row_iter.total_rows will be initialized at the beginning, just as you expect.
Suggestion
In my code, I'm going to use something like
try:
num_rows = row_iter.total_rows or query_job._query_results.total_rows
except NameError:
num_rows = None
to be compatible with future behavior while falling-back to the temporary workaround if necessary.
So I have an issue very similar to this question, but a bit different.
I am calling cursor.execute(sqlString) on a piece of sql that works fine when I run it directly on the mysql workbench. When I run the code however I get no result set.
I have exactly the same issue symptons as stated in the link and I have tried the linked solutions but it turns out that I do not have the same issue.
my _stored_results[] is empty when returning.
I am using the code in a try/except block, I have another python program that uses the same code to load a csv into a my mySQL db and it works dandy.
The code where I am having the issue is within an #app.route if that makes any differnce.
My code looks like this:
def functionName() :
try:
import mysql.connector
from mysql.connector import errorcode
cnx = mysql.connector.connect(user=init["dbDetails"][0], password=init["dbDetails"][1], host=init["dbDetails"][2], database=init["dbDetails"][3])
cur = cnx.cursor()
cur.close() #I deffo don't need the two lines below but they were added for a sanity check, just to make sure the cur was not being read from any other code.
cur = cnx.cursor() # and this one obviously
sqlString = 'CALL `schemaName`.`getProcedureName_sp`(1, 1, 0)'
cur.execute(sqlString, multi=True) # tried it here without the multi=True and got the msg telling me to use it.
getSomeDetails = cur.fetchall()
cnx.commit() # probably don't need to commit here I am just reading from the dB but I am trying anything as I have no idea what my issue might be.
return render_template('success.html')
except Exception as e:
return render_template('error.html', error = str(e))
finally:
cur.close()
cnx.close()
I am so baffled as I have this same code working in several places.
So I was beating my head against the wall with this, and when I couldn't get anywhere, I just decided to leave it and move on, then come back with a fresh mind. Well... It worked, kinda.
So I haven't found the solution but I have found a work around that does the job and might even shed some light as to what is actually happening in my code.
I decided that as the fetchall() method was what was causing me the trouble I should try to circumvent it.
I probed the cursor(cur) just before the fetchall() method was called and saw that cur._rows contains the results from the SQL call.
So I changed the line
getSomeDetails = cur.fetchall()
to
if len(cur._rows) > 0 :
getSomeDetails = list(cur._rows[0]) #I only ever expect one result in this query
#getSomeDetails should now have the row I am looking for
getSomeDetails[0] #gets me the field I am looking for
and now my variable getSomeDetails has the return values from the procedure call
They are however not in the nice format that I should have gotten them from the fetchall() function, so I had to do some processing, I had to ensure that I was getting some values back and I noted that these values were returned in a tuple
I have come across this issue on two different machines running two different OS's and two different versions of python (Windows 7 with Python 2.7 and Windows 10 with Python 3) both pieces of code were different so obviously infact I was using two different MySQL libraries so the actual code for the fix was slightly different in both cases but I am now in both cases getting data from my DB into variables in Python, so that's cool.
However, this is a hack and I am aware of that, I would rather be using the proper function cur.fetchall() so I am still open to suggestions of what could be going wrong here.
I am trying to call dbms_random.seed(42) on my DB, see ref. I need to use python and the JayDeBeApi connector. So far I've only been able to execute select statement without issue. I fail to understand what I am doing wrong.
It seems that JayDeBeApi does not provide the callproc method, so I cannot use it:
AttributeError: 'Cursor' object has no attribute 'callproc'
I've naively tried:
conn = jaydebeapi.connect('oracle.jdbc.driver.OracleDriver',
['jdbc:oracle:thin:#server:1521/dbname', 'user', 'password'])
curs = conn.cursor()
sql="exec dbms_random.seed(42)"
curs.execute(sql)
but this leads to: Error: ORA-00900: invalid SQL statement
I've tried two solutions which seems to have correct syntax, but since the generator is not deterministic, I believe they actually failed:
Using begin/end:
sql="begin dbms_random.seed(42); end;"
curs.execute(sql)
Using call:
sql="{ call dbms_random.seed(42) }"
curs.execute(sql)
So my question is: how do I call dbms_random.seed(42) on Oracle using JayDeBeApi ? As a side question how do I check that a statement has actually failed to execute (no exception was thrown and return value for execute is undefined.)
Update:
In fact the seed initialization is working as expected, since the results are what I expect for:
sql="SELECT DBMS_RANDOM.value FROM dual"
curs.execute(sql)
data = curs.fetchall()
print(data)
However I still see some odd behavior for my random query selection:
SELECT *
FROM (
SELECT *
FROM table
ORDER BY DBMS_RANDOM.RANDOM)
WHERE rownum < 21;
For some reason in the latter case, the DBMS_RANDOM.RANDOM is actually random...
After multiple trial and error, I believe this is just a side effect. Since I was not doing the cleanup pass:
curs.close()
conn.close()
jpype.shutdownJVM()
The query would lead to somewhat undefined behavior. Now that I have a proper cleanup code, I am getting deterministic results when calling multiples times my python script.
I'm relatively new to Python.
I'm currently working on SQL statement execution in Oracle DB.
When I execute query:
query = 'select * from table'
cursor.execute(query)
result = cursor.fetchall()
everything is going fine, but when I try to execute script:
script in plain text:
begin
SIEBEL_DBA.X_DR_DEPLOY(id => '1-4NANEI', env_code => 'SVE_SIT');
end;
/
code from script
script = "begin\nSIEBEL_DBA.X_DR_DEPLOY(id => '1-4NANEI', env_code => 'SVE_SIT');\nend;"
cursor.execute(script)
result = cursor.fetchall()
I get an exception, that this is not a query, but still this script has worked.
So from what I've googled, looks like I should use callproc function:
cursor.callproc['SIEBEL_DBA.X_DR_DEPLOY',{'id' : '1-4NANEI', 'env_code' : 'SVE_SIT'}]
connection.commit()
result = cursor.fetchall()
When I'm executing this statement, I'm also getting exception, but this time nothing has been changed in DB:
'builtin_function_or_method' object has no attribute 'getitem'
Could someone please point where I'm not correct and how should I modify statement so it would be working.
Huge thanks in advance!
RESOLUTION:
I was frustrated by the syntax and the complexity of callproc and callfunc functions.
I've found good resource: http://dbaportal.eu/sidekicks/sidekick-cx_oracle-code-paterns/#part1
in this link I found all needed info and examples on how to work with cx_Oracle library.
at the end I just needed to modify a bit my code:
cursor.callproc('SIEBEL_DBA.X_DR_DEPLOY', ['1-4NANEI', 'SVE_SIT'])
and the needed part was done, I didn't need to specify any return type, as script that I'm executing doesn't return any value, it just sets it.
The exception is because you are using [ ] where you should be using ():
cursor.callproc('SIEBEL_DBA.X_DR_DEPLOY',{'id' : '1-4NANEI', 'env_code' : 'SVE_SIT'})
Keep in mind the return type is required:
Cursor.callfunc(name, returnType, parameters=[], keywordParameters =
{})
Call a function with the given name. The return type is specified in
the same notation as is required by setinputsizes(). The sequence of
parameters must contain one entry for each argument that the function
expects. Any keyword parameters will be included after the positional
parameters. The result of the call is the return value of the
function.
I have a database table with UNIQUE key. If I want to insert some record there are two possible ways. First, the unique item doesn't exist yet, that's OK, just return new id. Second, the item already exists and I need to get the id of this unique record.
The problem is, that anything I try, I get always some exception.
Here's example of the code:
def __init__(self, host, user, password, database):
# set basic attributes
super().__init__(host, user, password, database)
#open connection
try:
self.__cnx = mysql.connector.connect(
database=database, user=user, password=password, host = host)
#self.__cursor = self.__cnx.cursor()
except ...
def insert_domain(self, domain):
insertq = "INSERT INTO `sp_domains` (`domain`) VALUES ('{0}')".format(domain)
cursor = self.__cnx.cursor()
try:
cursor.execute(insertq)
print("unique")
except (mysql.connector.errors.IntegrityError) as err:
self.__cnx.commit()
print("duplicate")
s = "SELECT `domain_id` FROM `sp_domains` WHERE `domain` = '{0}';".format(domain)
try:
id = cursor.execute(s).fetchone()[0]
except AttributeError as err:
print("Unable to execute the query:", err, file=sys.stderr)
except mysql.connector.errors.ProgrammingError as err:
print("Query syntax error:", err, file=sys.stderr)
else:
self.__cnx.commit()
cursor.close()
but anyting I try, on the first duplicate record I get either 'MySQL Connection not available', 'Unread result'. The code is just example to demonstrate it.
This is my first program using Connector/python, so I don't know all the rules, about fetch the results, commiting queries and so on.
Could anyone help me with this issue, please? Or is there any efficient way to such task ('cause this one seems to be not the best solution to me). Thank you for any advice.
I can't fix your code, because you've given us two different versions of the code and two partially-described errors without full information, but I can tell you how to get started.
From a comment:
In previous version it was type error I guess, something like "NoneType has no attribute 'fetchone'.
Looking at your code, the only place you call fetchone is here:
id = cursor.execute(s).fetchone()[0]
So obviously, cursor.execute(s) returned None. Why did it return None? Well, that's what it's supposed to return, according to the documentation.*
What you want to do is:
cursor.execute(s)
id = cursor.fetchone()[0]
… as all of the sample code does.
And for future reference, it's a lot easier to debug an error like this if you first note which line it happens on instead of throwing away the traceback, and then breaking that line into pieces and logging the intermediate values. Usually, you'll find one that isn't what you expected, and the problem will be much more obvious at that point, then three steps later on when you get a bizarre exceptions.
* Technically, the documentation just says that "Return values are not defined" for cursor.execute, so it would be perfectly legal for a DB-API module to return self here. Then again, it would also be legal to return some object that erases your hard drive when you call a method on it.