This SQL Alchemy 0.9.7 code executes without error -- but does not update the underlying database as expected.
Here is the python:
print t #prints TITLE ABSTRACTOR 1
print newtitle #prints TITLE ABSTRACTOR I
print session.query(Basic).filter(Basic.title==t).count() #prints 1
ret = update(Basic).where(Basic.title==t).values(title=newtitle)
session.commit()
Here is what the database looks like after the update:
select count(*) from basics where title='TITLE ABSTRACTOR 1';
count
-------
1
(1 row)
select count(*) from basics where title='TITLE ABSTRACTOR I';
count
-------
0
(1 row)
Have I hit a SQL alchemy bug or am I missing something?
You're just constructing an update statement:
ret = update(Basic).where(Basic.title==t).values(title=newtitle)
That doesn't do anything unless you execute the statement:
stmt = update(Basic).where(Basic.title==t).values(title=newtitle)
ret = conn.execute(stmt)
But I think you were trying to use the ORM interface, not the core interface. In which case, although I don't remember the details, I'm pretty sure you do that by modifying a query object, not by calling anything named update. Hopefully if this is what you're looking for, hopefully someone who's fresher on this will provide a better answer, but something like this:
ret = session.query(Basic).filter(Basic.title==t)
ret.title = newtitle
If this doesn't make sense to you, see Executing in the tutorial. But I'm guessing you know this and it was just one of those stupid bugs we all make and all have a hard enough time seeing in other people's code, and it's 100x worse in our own. :)
Related
I am teaching myself python and have a function called mysqltimedcheck()
This executes a very basic mySQL SELECT statement and returns the rowcount.
If i cange the value of 'shown' from 0 to 1 in the mysql table (while the script is running) for example the rowcount does not change.
I can see when debugging that it reruns the cursor.execute line again but still prints an old value
here is my function:
def mysqltimedcheck():
cursor = mydb.cursor(buffered=True)
window.after(1500, mysqltimedcheck)
cursor.execute("SELECT * FROM alerts WHERE shown = 0")
rc = cursor.rowcount
print(rc)
If i restart my test application then it pulls the correct rowcount value but changing this value while running does not show as correct.
I feel im missing something very basic.
Thanks for your help
All credit to #LabibaKanij
adding mydb.commit() worked correctly and allowed me to change the value in the table and it changed for my script without restarting.
Using Python and SQlite3 where c is a cursor this code...
print("vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv")
print("SQL and parameters:",sql,parm)
c.execute(sql,parm)
# Get the row
print("Executed OK")
response = c.fetchone()
# If not successful return null
if not response:
return None
#
print("and produced ", response)
print("^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^")
give this output:
vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
SQL and parameters: select * from Links where LinkNum = ? (301,)
Executed OK
and produced (301, 'Index', 'The Independent', 'https://www.independent.co.uk/', 6, 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
SQL and parameters: select * from Links where LinkNum = ? (301,)
Encountered exception of type ProgrammingError with arguments ('Incorrect number of bindings supplied. The current statement uses 1, and there are 6 supplied.',)
The application will close
Two identical statements. 1 works and the next throws the exception. As can be seen the row I'm trying to retrieve has 6 columns but that's the only hint/clue I can see. Can anyone help with tracking down the problem? Thanks.
Whatever was causing SQLite to have a fit I remedied the problem by retrieving the second row/object out side the Links object and passing it as an argument to the method I was calling, rather than have that method attempt to retrieve the object.
The problem would still be there but must be deep within the internals of Python instantiation and/or SQLite. Whatever, this problem is solved with some less fancy code.
I'm learning BigQuery API using Python Client Libraries v0.28
https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/usage.html#run-a-simple-query
Wrote this simple code to fetch data from the table
1) Create client object
client_ = bigquery.Client.from_service_account_json('/Users/xyz/key.json')
2) Begin new Async query job
QUERY = 'SELECT visitid FROM `1234567.ga_sessions_20180101`'
query_job = client_.query(QUERY
, job_id=str(uuid.uuid4()))
3) poll until the query is DONE
while (query_job.state == 'RUNNING'):
time.sleep(5)
query_job.reload()
4) Fetch the results in iteration
query_job.reload()
iter = query_job.result()
At this stage I'd like to fetch how many rows are in the table. As per the doc GitHub code iter is of type bigquery.table.RowIterator with a property [tier.total_rows][1]
5) However, at this stage when I print:
print(iter.total_rows)
It keeps returning None
I'm pretty sure this table is NOT empty an dry query is correctly formatted!
Any help to any pointers what am I missing here will be really helpful... Thanks a lot!
Cheers!
You need to also check query_job.error_result to make sure query succeeded.
You can also see your job in the UI, which can be useful for debugging, using project id and job id:
https://bigquery.cloud.google.com/results/projectid:jobid
Also, query_job.result() already waits for the job completion so you don't need to poll.
The current behavior of how RowIterator returns None is indeed perplexing. Luckily, according to this issue, tswast's comment from 10 days ago indicates that the developers are working on a better solution.
Current awkward behavior of .total_rows
Currently, .total_rows is initialized only once iteration begins. (In what follows, for clarity I renamed your iter variable to row_iter.)
row_iter = query_job.result()
itr = iter(row_iter)
first_row = next(itr)
print(row_iter.total_rows) # Now you get a number instead of None.
This is ugly because to continue the iteration, we must either handle the first row differently or call row_iter = query_job.result() again.
Temporary workaround
A currently-working alternative is to use the value of query_job._query_results.total_rows. Unfortunately this is cheating because _query_results is private, so there is no reason to expect that this will work in the future.
Future behavior
If tswast's proposal is implemented, then row_iter.total_rows will be initialized at the beginning, just as you expect.
Suggestion
In my code, I'm going to use something like
try:
num_rows = row_iter.total_rows or query_job._query_results.total_rows
except NameError:
num_rows = None
to be compatible with future behavior while falling-back to the temporary workaround if necessary.
So I have an issue very similar to this question, but a bit different.
I am calling cursor.execute(sqlString) on a piece of sql that works fine when I run it directly on the mysql workbench. When I run the code however I get no result set.
I have exactly the same issue symptons as stated in the link and I have tried the linked solutions but it turns out that I do not have the same issue.
my _stored_results[] is empty when returning.
I am using the code in a try/except block, I have another python program that uses the same code to load a csv into a my mySQL db and it works dandy.
The code where I am having the issue is within an #app.route if that makes any differnce.
My code looks like this:
def functionName() :
try:
import mysql.connector
from mysql.connector import errorcode
cnx = mysql.connector.connect(user=init["dbDetails"][0], password=init["dbDetails"][1], host=init["dbDetails"][2], database=init["dbDetails"][3])
cur = cnx.cursor()
cur.close() #I deffo don't need the two lines below but they were added for a sanity check, just to make sure the cur was not being read from any other code.
cur = cnx.cursor() # and this one obviously
sqlString = 'CALL `schemaName`.`getProcedureName_sp`(1, 1, 0)'
cur.execute(sqlString, multi=True) # tried it here without the multi=True and got the msg telling me to use it.
getSomeDetails = cur.fetchall()
cnx.commit() # probably don't need to commit here I am just reading from the dB but I am trying anything as I have no idea what my issue might be.
return render_template('success.html')
except Exception as e:
return render_template('error.html', error = str(e))
finally:
cur.close()
cnx.close()
I am so baffled as I have this same code working in several places.
So I was beating my head against the wall with this, and when I couldn't get anywhere, I just decided to leave it and move on, then come back with a fresh mind. Well... It worked, kinda.
So I haven't found the solution but I have found a work around that does the job and might even shed some light as to what is actually happening in my code.
I decided that as the fetchall() method was what was causing me the trouble I should try to circumvent it.
I probed the cursor(cur) just before the fetchall() method was called and saw that cur._rows contains the results from the SQL call.
So I changed the line
getSomeDetails = cur.fetchall()
to
if len(cur._rows) > 0 :
getSomeDetails = list(cur._rows[0]) #I only ever expect one result in this query
#getSomeDetails should now have the row I am looking for
getSomeDetails[0] #gets me the field I am looking for
and now my variable getSomeDetails has the return values from the procedure call
They are however not in the nice format that I should have gotten them from the fetchall() function, so I had to do some processing, I had to ensure that I was getting some values back and I noted that these values were returned in a tuple
I have come across this issue on two different machines running two different OS's and two different versions of python (Windows 7 with Python 2.7 and Windows 10 with Python 3) both pieces of code were different so obviously infact I was using two different MySQL libraries so the actual code for the fix was slightly different in both cases but I am now in both cases getting data from my DB into variables in Python, so that's cool.
However, this is a hack and I am aware of that, I would rather be using the proper function cur.fetchall() so I am still open to suggestions of what could be going wrong here.
I'm looking for a complete example of using select for update in SQLAlchemy, but haven't found one googling. I need to lock a single row and update a column, the following code doesn't work (blocks forever):
s = table.select(table.c.user=="test",for_update=True)
# Do update or not depending on the row
u = table.update().where(table.c.user=="test")
u.execute(email="foo")
Do I need a commit? How do I do that? As far as I know you need to:
begin transaction
select ... for update
update
commit
If you are using the ORM, try the with_for_update function:
foo = session.query(Foo).filter(Foo.id==1234).with_for_update().one()
# this row is now locked
foo.name = 'bar'
session.add(foo)
session.commit()
# this row is now unlocked
Late answer, but maybe someone will find it useful.
First, you don't need to commit (at least not in-between queries, which I'm assuming you are asking about). Your second query hangs indefinitely, because you are effectively creating two concurrent connections to the database. First one is obtaining lock on selected records, then second one tries to modify locked records. So it can't work properly. (By the way in the example given you are not calling first query at all, so I'm assuming in your real tests you did something like s.execute() somewhere). So to the point—working implementation should look more like:
s = conn.execute(table.select(table.c.user=="test", for_update=True))
u = conn.execute(table.update().where(table.c.user=="test"), {"email": "foo"})
conn.commit()
Of course in such simple case there's no reason to do any locking but I guess it is example only and you were planning to add some additional logic between those two calls.
Yes, you do need to commit, which you can execute on the Engine or create a Transaction explicitely. Also the modifiers are specified in the values(...) method, and not execute:
>>> conn.execute(users.update().
... where(table.c.user=="test").
... values(email="foo")
... )
>>> my_engine.commit()