I'm trying to get a list of warnings after a mySQL query, using Django admin. I can see from the documentation here that it's possible to record warnings by setting connection.get_warnings to true. But I can't find anything explaining how to read those warnings.
I do not want to throw an exception - I am deleting items from the database using a DELETE IGNORE statement and want to get all instances of deletions that failed (due to external keys, etc.)
I've tried returning the result of the execute function itself (just gave me a number) and calling fetchwarnings() on the cursor (threw a "Cursor object has no attribute fetchwarnings" error).
I'm still new to both Python and Django. I'm looking through all the documentation I can find but can't find anything that works.
from django.db import connection
query = "{query here}"
connection.get_warnings = True
with connection.cursor() as cursor:
cursor.execute(query) <-- Returns a number
return cursor.fetchwarnings() <-- Throws an error
Related
I'm using sqlalchemy in combination with sqlite and the databases library and I'm trying to wrap my head around what that combination returns when doing update queries. I'm running a testcase and I have sqlalchemy set up to roll back upon execution of each testcase via force_rollback=True.
db = databases.Database(DB_URL, force_rollback=True)
query = update(my_table).where(my_table.columns.id == some_id_to_update).values(**values)
res = await db.execute(query)
When working with psql, I'd expect res to be the number of rows that were affected by the UPDATE query, but from reading the documentation, sqlite seems to behave differently in that it doesn't return anything. I tested this manually by connecting to the database via sqlite3 and as expected, there is no return when doing UPDATE queries. sqlalchemy however does return something, which I assume is the number of total rows in the table, but I'm not sure. Can anybody shed some light into what is actually returned?
What's more, when I tried to get the number of rows affected by the UPDATE query via SELECT changes(), I'm also getting the number of total rows in the table and not the rows affected by the most recent query. Do I have a misunderstanding of what changes() does?
"The changes() function returns the number of database rows that were changed or inserted or deleted by the most recently completed INSERT, DELETE, or UPDATE statement, exclusive of statements in lower-level triggers."
When you use the Python sqlite3 module, you use .executeXXX interfaces to evaluate/prepare your query. If the query is supposed to modify the database, it does it at this stage. You have to use the same interface to prepare a SELECT statement. In either case, the .executeXXX interfaces never return anything. To get the result of a SELECT query, you have to use a .fetchXXX interface after running .executeXXX.
To get the number of changed rows after INSERT, DELETE, or UPDATE statement via sqlite3, you can also take the difference in con.total_changes before/after running .executeXXX.
I have a Flask application that uses SQLAlchemy (with some Marshmallow for serialization and deserialization).
I'm currently encountering some intermittent issues when trying to dump an object post-commit.
To give an example, let's say I have implemented a (multi-tenant) system for tracking system faults of some sort. This information is contained in a fault table:
class Fault(Base):
__tablename__ = "fault"
fault_id = Column(BIGINT, primary_key=True)
workspace_id = Column(Integer, ForeignKey('workspace.workspace_id'))
local_fault_id = Column(Integer)
name = Column(String)
description = Column(String)
I've removed a number of columns in the interest of simplicity, but this is the core of the model. The columns should be largely self explanatory, with workspace_id effectively representing tenant, and local_fault_id representing a tenant-specific fault sequence number, which is handled via a separate fault_sequence table.
This fault_sequence table holds a counter against workspace, and is updated by means of a simple on_fault_created() function that is executed by a trigger:
CREATE TRIGGER fault_created
AFTER INSERT
ON "fault"
FOR EACH ROW
EXECUTE PROCEDURE on_fault_created();
So - the problem:
I have a Flask endpoint for fault creation, where we create an instance of a Fault entity, add this via a scoped session (session.add(fault)), then call session.commit().
It seems that this is always successful in creating the desired entities in the database, executing the sequence update trigger etc. However, when I then try to interrogate the fault object for updated fields (after commit()), around 10% of the time I find that each key/field just points to an Exception:
psycopg2.errors.InFailedSqlTransaction: current transaction is aborted, commands ignored until end of transaction block
Which seems to boil down to the following:
(psycopg2.errors.InvalidTextRepresentation) invalid input syntax for integer: ""
[SQL: SELECT fault.fault_id AS fault_fault_id, fault.workspace_id AS fault_workspace_id, fault.local_fault_id AS fault_local_fault_id, fault.name as fault_name, fault.description as fault_description
FROM fault
WHERE fault.fault_id = %(param_1)s]
[parameters: {'param_1': 166}]
(Background on this error at: http://sqlalche.me/e/13/2j8
My question, then, is what do we think could be causing this?
I think it smells like a race condition, with the update trigger not being complete before SQLAlchemy has tried to get the updated data; perhaps local_fault_id is null, and this is resulting in the invalid input syntax error.
That said, I have very low confidence on this. Any guidance here would be amazing, as I could really do with retrieving that sequence number that's incremented/handled by the update trigger.
Thanks
Edit 1:
Some more info:
I have tried removing the update trigger, in the hope of eliminating that as a suspect. This behaviour is still intermittently evident, so I don't think it's related to that.
I have tried adopting usage of flush and refresh before the commit, and this allows me to get the values that I need - though commit still appears to 'break' the fault object.
Edit 2:
So it really seems to be more postgres than anything else. When I interrogate my database logs, this is the weirdest thing. I can copy and paste the command it says is failing, and I struggle to see how this integer value in the WHERE clause is possibly evaluating to an empty string.
This same error is reproducible with SELECT ... FROM fault WHERE fault.fault_id = '', which in no way seems to be the query making to the DB.
I am stumped.
Your sentence "This same error is reproducible with SELECT ... FROM fault WHERE fault.fault_id = '', which in no way seems to be the query making to the DB." seems to indicate that you are trying to access an object that does not have the database primary key "fault_id".
I guess, given that you did not provide the code, that you are adding the object to your session (session.add), committing (session.commit) and then using the object. As fault_id is autogenerated by the database, the fault object in the session (in memory) does not have fault_id.
I believe you can correct this with:
session.add(fault)
session.commit()
session.refresh(fault)
The refresh needs to be AFTER commit to refresh the fault object and retrieve fault_id.
If you are using async, you need
session.add(fault)
await session.commit()
await session.refresh(fault)
Setup:
Oracle DB running on a windows machine
Mac connected with the database, both in the same network
Problem:
When I created a sequence in SQL Developer, I can see and use the sequence in this session. If I logoff and login again the sequence is still there. But if I try to use the sequence via Python and cx_Oracle, it doesn't work. It also doesn't work the other way around.
[In SQL Developer: user: uc]
create SEQUENCE seq1;
select seq1.nextval from dual; ---> 1
commit; --> although the create statement is a DDL method, just in case
[login via Python, user: uc]
select seq1.currval from dual;--> ORA-08002 Sequence seq1.currval isn't defined in this session
The python code:
import cx_Oracle
cx_Oracle.init_oracle_client(lib_dir="/Users/benreisinger/Documents/testclients/instantclient_19_8", config_dir=None, error_url=None, driver_name=None)
# Connect as user "hr" with password "hr" to the "orclpdb" service running on a remote computer.
connection = cx_Oracle.connect("uc", "uc", "10.0.0.22/orcl")
cursor = connection.cursor()
cursor.execute("""
select seq1.currval from dual
""")
print(cursor)
for seq1 in cursor:
print(seq1)
The error says, that [seq1] wasn't defined in this session, but why does the following work:
select seq1.nextval from dual
--> returns 2
Even after issuing this, I can't use seq1.currval
Btw., select sequence_name from user_sequences returns seq1in Python
[as SYS user]
select * from v$session
where username = 'uc';
--> returns zero rows
Why is seq1 not in reach for the python program ?
Note: With tables, everything just works fine
EDIT:
also with 'UC' being upper case, no rows returned
first issuing
still doesn't work
Not sure how to explain this. The previous 2 answers are correct, but somehow you seem to miss the point.
First, take everything that is irrelevant out of the equation. Mac client on Windows db: doesn't matter. SQLDeveloper vs python: doesn't matter. The only thing that matters is that you connect twice to the database as the same schema. You connect twice, that means that you have 2 separate sessions and those sessions don't know about each other. Both sessions have access to the same database objects, so you if you execute ddl (eg create sequence), that object will be visible in the other session.
Now to the core of your question. The oracle documentation states
"To use or refer to the current sequence value of your session, reference seq_name.CURRVAL. CURRVAL can only be used if seq_name.NEXTVAL has been referenced in the current user session (in the current or a previous transaction)."
You have 2 different sessions, so according to the documentation, you should not be able to call seq_name.CURRVAL in the other session. That is exactly the behaviour you are seeing.
You ask "Why is seq1 not in reach for the python program ?". The answer is: you're not correct, it is in reach for the python program. You can call seq1.NEXTVAL from any session. But you cannot invoke seq1.NEXTVAL from one session (SQLDeveloper) and then invoke seq1.CURRVAL from another session (python) because that is just how sequences works as stated in documentation.
Just to confirm you're not in the same session, execute the following statement for both clients (SQLDeveloper and python):
select sys_context('USERENV','SID') from dual;
You'll notice that the session id is different.
CURRVAL returns the last allocated sequence number in the current session. So it only works when we have previously executed a NEXTVAL. So these two statements will return the same value when run in the same session:
select seq1.nextval from dual
/
select seq1.currval from dual
/
It's not entirely clear what you're trying to achieve, but it looks like your python code is executing a single statement for the connection, so it's not tapping into an existing session.
This statement returns zero rows ...
select * from v$session
where username = 'uc';
... because database objects in Oracle are stored in UPPER case (at least by default, but it's wise to stick with that default. So use where username = 'UC' instead.
Python established a new session. In it, sequence hasn't been invoked yet, so its currval doesn't exist. First you have to select nextval (which, as you said, returned 2) - only then currval will make sense.
Saying that
Even after issuing this, I can't use seq1.currval
is hard to believe.
This: select * From v$session where username = 'uc' returned nothing because - by default - all objects are stored in uppercase, so you should have ran
.... where username = 'UC'
Finally:
commit; --> although the create statement is a DDL method, just in case
Which case? There's no case. DDL commits. Moreover, commits twice (before and after the actual DDL statement). And there's nothing to commit either. Therefore, what you did is unnecessary and pretty much useless.
I've read some q&a about "syntax error at or near" but none could solve my issue.
A sample of the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
psycopg2.ProgrammingError: syntax error at or near "UPDATE"
LINE 1: DECLARE "teste" CURSOR WITHOUT HOLD FOR UPDATE applicant SET...
^
Please, note the ^ right on the UPDATE. I've tested the update script on pgadmin 4 and everything worked fine.
The script is really simple:
UPDATE applicant
SET cv_entities = %s
WHERE id = %s
My code is basically:
def _connect_database():
return psy.connect(
dbname=settings.DATABASE['DBNAME'],
host=settings.DATABASE['HOST'],
port=settings.DATABASE['PORT'],
user=settings.DATABASE['USER'],
password=settings.DATABASE['PASSWORD'],
application_name=settings.env
)
# Connects to database
conn = _connect_database()
# Creats a named cursor
cur = conn.cursor('test')
# Execute
cur.execute(update_script, ('{"json": "test"}', '11125ba0-748f-11e8-b1d0-d8108ee3at21'))
I've explicitly written the script on the execute method as well as the parameters, but I still get the same error:
cur.execute("UPDATE applicant SET cv_entities = 'json here' WHERE id = '11125ba0-748f-11e8-b1d0-d8108ee3at21'")
Note that I even removed the double quotes from the first parameter ('json here')
Am I missing something?!
With
cur = conn.cursor('test')
you are trying to open a server-side cursor. Per the documentation:
cursor(name=None, cursor_factory=None, scrollable=None, withhold=False)
Return a new cursor object using the connection.
If name is specified, the returned cursor will be a server side cursor (also known as named cursor). Otherwise it will be a regular client side cursor.
Use just
cur = conn.cursor()
Server side (named) cursors can be used only for SELECT or VALUES queries. They implement Postgres cursors:
query
A SELECT or VALUES command which will provide the rows to be returned by the cursor.
With named cursors the resulting data is gathered on the server and sent (maybe partialy) to the client when needed (fetched). The data is stored only during the transaction in which the cursor was opened, so commit or rollback releases it.
Client side (unnamed) cursors give the possibility to execute any valid SQL query. A possible resultset is sent to the client immediately after executing the query.
It's possible to use named and unnamed cursors utilizing a single connection but if you want to do this concurrently, you should use two separate connections.
I am trying to call dbms_random.seed(42) on my DB, see ref. I need to use python and the JayDeBeApi connector. So far I've only been able to execute select statement without issue. I fail to understand what I am doing wrong.
It seems that JayDeBeApi does not provide the callproc method, so I cannot use it:
AttributeError: 'Cursor' object has no attribute 'callproc'
I've naively tried:
conn = jaydebeapi.connect('oracle.jdbc.driver.OracleDriver',
['jdbc:oracle:thin:#server:1521/dbname', 'user', 'password'])
curs = conn.cursor()
sql="exec dbms_random.seed(42)"
curs.execute(sql)
but this leads to: Error: ORA-00900: invalid SQL statement
I've tried two solutions which seems to have correct syntax, but since the generator is not deterministic, I believe they actually failed:
Using begin/end:
sql="begin dbms_random.seed(42); end;"
curs.execute(sql)
Using call:
sql="{ call dbms_random.seed(42) }"
curs.execute(sql)
So my question is: how do I call dbms_random.seed(42) on Oracle using JayDeBeApi ? As a side question how do I check that a statement has actually failed to execute (no exception was thrown and return value for execute is undefined.)
Update:
In fact the seed initialization is working as expected, since the results are what I expect for:
sql="SELECT DBMS_RANDOM.value FROM dual"
curs.execute(sql)
data = curs.fetchall()
print(data)
However I still see some odd behavior for my random query selection:
SELECT *
FROM (
SELECT *
FROM table
ORDER BY DBMS_RANDOM.RANDOM)
WHERE rownum < 21;
For some reason in the latter case, the DBMS_RANDOM.RANDOM is actually random...
After multiple trial and error, I believe this is just a side effect. Since I was not doing the cleanup pass:
curs.close()
conn.close()
jpype.shutdownJVM()
The query would lead to somewhat undefined behavior. Now that I have a proper cleanup code, I am getting deterministic results when calling multiples times my python script.