Develop Reference

Why is a Pandas Oracle DB query faster with literals?

When I use the bind variable approach found here: https://cx-oracle.readthedocs.io/en/latest/user_guide/bind.html#bind
and here: Python cx_Oracle bind variables
my query takes around 8 minutes, but when I use hardcoded values (literals), it takes around 20 seconds.
I'm struggling to comprehend what's happening "behind-the-scenes" (variables/memory access/data transfer/query parsing) to see if there's any way for me to adhere to the recommended approach of using bind variables and get the same ~20s performance.
This python script will be automated and the values will be dynamic, so I definitely can't use hardcoded values.
Technical background: Python 3.6; Oracle 11g; cx_Oracle 8
---- python portion of code -----
first version
param_dict = {“startDate:”01-Jul-21”, “endDate:”31-Jul-2021”}
conn = (typical database connection code….)
cur = conn.cursor()
###### this query has the bind variables and param_dict keys match bind variable aliases; runtime ~480s (8mins)
cur_df = pandas.DataFrame(cur.execute("inserted_query_here", param_dict))
second version
conn = (typical database connection code….)
cur = conn.cursor()
###### this query has the hardcoded values (literals); runtime ~20s
cur_df = pandas.DataFrame(cur.execute("inserted_query_here"))

#ChristopherJones and Alex thanks for the referenced articles.
I've been able to solve the issue by thoroughly examining the EXPLAIN PLAN. The query that performed faster wasn't using index (faster to do full table scan); the other was (bind variable version of query).
I applied NO_INDEX hint accordingly and now have ~20s result for bind variable version of query.

Sequence nextval/currval in two sessions

Setup:
Oracle DB running on a windows machine
Mac connected with the database, both in the same network
Problem:
When I created a sequence in SQL Developer, I can see and use the sequence in this session. If I logoff and login again the sequence is still there. But if I try to use the sequence via Python and cx_Oracle, it doesn't work. It also doesn't work the other way around.
[In SQL Developer: user: uc]
create SEQUENCE seq1;
select seq1.nextval from dual; ---> 1
commit; --> although the create statement is a DDL method, just in case
[login via Python, user: uc]
select seq1.currval from dual;--> ORA-08002 Sequence seq1.currval isn't defined in this session
The python code:
import cx_Oracle
cx_Oracle.init_oracle_client(lib_dir="/Users/benreisinger/Documents/testclients/instantclient_19_8", config_dir=None, error_url=None, driver_name=None)
# Connect as user "hr" with password "hr" to the "orclpdb" service running on a remote computer.
connection = cx_Oracle.connect("uc", "uc", "10.0.0.22/orcl")
cursor = connection.cursor()
cursor.execute("""
select seq1.currval from dual
""")
print(cursor)
for seq1 in cursor:
print(seq1)
The error says, that [seq1] wasn't defined in this session, but why does the following work:
select seq1.nextval from dual
--> returns 2
Even after issuing this, I can't use seq1.currval
Btw., select sequence_name from user_sequences returns seq1in Python
[as SYS user]
select * from v$session
where username = 'uc';
--> returns zero rows
Why is seq1 not in reach for the python program ?
Note: With tables, everything just works fine
EDIT:
also with 'UC' being upper case, no rows returned
first issuing
still doesn't work

Not sure how to explain this. The previous 2 answers are correct, but somehow you seem to miss the point.
First, take everything that is irrelevant out of the equation. Mac client on Windows db: doesn't matter. SQLDeveloper vs python: doesn't matter. The only thing that matters is that you connect twice to the database as the same schema. You connect twice, that means that you have 2 separate sessions and those sessions don't know about each other. Both sessions have access to the same database objects, so you if you execute ddl (eg create sequence), that object will be visible in the other session.
Now to the core of your question. The oracle documentation states
"To use or refer to the current sequence value of your session, reference seq_name.CURRVAL. CURRVAL can only be used if seq_name.NEXTVAL has been referenced in the current user session (in the current or a previous transaction)."
You have 2 different sessions, so according to the documentation, you should not be able to call seq_name.CURRVAL in the other session. That is exactly the behaviour you are seeing.
You ask "Why is seq1 not in reach for the python program ?". The answer is: you're not correct, it is in reach for the python program. You can call seq1.NEXTVAL from any session. But you cannot invoke seq1.NEXTVAL from one session (SQLDeveloper) and then invoke seq1.CURRVAL from another session (python) because that is just how sequences works as stated in documentation.
Just to confirm you're not in the same session, execute the following statement for both clients (SQLDeveloper and python):
select sys_context('USERENV','SID') from dual;
You'll notice that the session id is different.

CURRVAL returns the last allocated sequence number in the current session. So it only works when we have previously executed a NEXTVAL. So these two statements will return the same value when run in the same session:
select seq1.nextval from dual
/
select seq1.currval from dual
/
It's not entirely clear what you're trying to achieve, but it looks like your python code is executing a single statement for the connection, so it's not tapping into an existing session.
This statement returns zero rows ...
select * from v$session
where username = 'uc';
... because database objects in Oracle are stored in UPPER case (at least by default, but it's wise to stick with that default. So use where username = 'UC' instead.

Python established a new session. In it, sequence hasn't been invoked yet, so its currval doesn't exist. First you have to select nextval (which, as you said, returned 2) - only then currval will make sense.
Saying that
Even after issuing this, I can't use seq1.currval
is hard to believe.
This: select * From v$session where username = 'uc' returned nothing because - by default - all objects are stored in uppercase, so you should have ran
.... where username = 'UC'
Finally:
commit; --> although the create statement is a DDL method, just in case
Which case? There's no case. DDL commits. Moreover, commits twice (before and after the actual DDL statement). And there's nothing to commit either. Therefore, what you did is unnecessary and pretty much useless.

why the stored procedure called from sqlalchemy is not working but calling from workbench is working?

I have a stored procedure.
calling it via MySQL workbench as follows working;
CALL `lobdcapi`.`escalatelobalarm`('A0001');
But not from the python program. (means it is not throwing any exception, process finish execution silently) if I make any error in column names, then at python I get an error. So it calls my stored procedure but not working as expected. (it is an update query .it needs SAFE update )
Why through the python sqlalchemy this update didn't update any records?
CREATE DEFINER=`lob`#`%` PROCEDURE `escalatelobalarm`(IN client_id varchar(50))
BEGIN
SET SQL_SAFE_UPDATES = 0;
update lobdcapi.alarms
set lobalarmescalated=1
where id in (
SELECT al.id
from (select id,alarmoccurredhistoryid from lobdcapi.alarms where lobalarmpriorityid=1 and lobalarmescalated=0 and clientid=client_id
and alarmstatenumber='02' ) as al
inner join lobdcapi.`alarmhistory` as hi on hi.id=al.alarmoccurredhistoryid
and hi.datetimestamp<= current_timestamp() )
);
SET SQL_SAFE_UPDATES = 1;
END
I call it like;
from sqlalchemy import and_, func,text
db.session.execute(text("CALL escalatelobalarm(:param)"), {'param': clientid})
I suspect the param I pass via code didn't get bind properly?

I haven't called stored procs from SQLAlchemy, but it seems possible that this could be within a transaction because you're using the session. Perhaps calling db.session.commit() at the end would help?
If that fails, SQLAlchemy calls out calling stored procs here. Perhaps try their method of using callproc. Adapting to your use-case, something like:
connection = db.session.connection()
try:
cursor = connection.cursor()
cursor.callproc("escalatelobalarm", [clientid])
results = list(cursor.fetchall())
cursor.close()
connection.commit()
finally:
connection.close()

Stored procedure stops executing after few steps when executed from python

I am calling a stored procedure from python using pyodbc from python. It stops executing after a few statements.
Previously it wasn't reflecting changes in database. Fixed that issue using autocommit=True
Divided steps into different two separate procedures and it works. Nothing wrong with the code
I am using SQL Server 2017 RDS, couldn't find any time specific setting at the server that needs to be specified
It would be great to know reason of stopping stored procedure execution after some steps.
In the following script it stops executing from table3_trn processing:
SELECT #l_cnt = count(1)
FROM plutotrn.table3_trn
CREATE OR ALTER PROCEDURE sp_load_data
AS
DECLARE #l_cnt INT
DECLARE #def_schema NVARCHAR(20) ='dbo'
DECLARE #transit_schema NVARCHAR(20)='trn_schema'
BEGIN
SET NOCOUNT ON;
BEGIN TRY
SELECT #l_cnt = count(1)
FROM table1_trn
IF #l_cnt > 0
BEGIN
EXEC sp_modify_indexes #table_name='table1',#schema_name=#def_schema,#enable_or_rebuild='DISABLE' ;
EXEC sp_switch_partitions #source_table = 'table1_trn' , #target_table = 'table1' ,#src_schema_name=#transit_schema ,#tgt_schema_name =#def_schema
END
SELECT #l_cnt = count(1)
FROM plutotrn.table2_trn
IF #l_cnt > 0
BEGIN
EXEC sp_modify_indexes #table_name='table2',#schema_name=#def_schema,#enable_or_rebuild='DISABLE' ;
EXEC sp_switch_partitions #source_table = 'table2_trn' , #target_table = 'table2' ,#src_schema_name=#transit_schema ,#tgt_schema_name =#def_schema
END
SELECT #l_cnt = count(1)
FROM plutotrn.table3_trn
IF #l_cnt > 0
BEGIN
EXEC sp_modify_indexes #table_name='table3',#schema_name=#def_schema,#enable_or_rebuild='DISABLE' ;
ALTER TABLE table3 NOCHECK CONSTRAINT ALL ;
ALTER TABLE table3_trn NOCHECK CONSTRAINT ALL ;
EXEC sp_switch_partitions #source_table = 'table3_trn' , #target_table = 'table3' ,#src_schema_name=#transit_schema ,#tgt_schema_name =#def_schema
END TRY
BEGIN CATCH
INSERT INTO debug VALUES (concat(ERROR_NUMBER(),ERROR_LINE (),ERROR_MESSAGE()))
END CATCH
END;

I have solved it by adding sleep time for 2 mins after procedure call from python and all steps from the procedure are executed successfully.May not be the best solution but worked for me.

pypyodbc - Invalid cursor state when executing stored procedure in a loop

I have a python program which uses pypyodbc to interact with MSSQL database. A stored procedure is created in MSSQL and is run via python. If I execute the stored procedure only once (via python), there are no problems. However, when it is executed multiple times within a for loop, I get the following error:
pypyodbc.ProgrammingError: ('24000', '[24000] [Microsoft][SQL Server Native Client 11.0]Invalid cursor state')
My python code details are below:
connection_string_prod = 'Driver={SQL Server Native Client 11.0};Server=PSCSQCD1234.TEST.AD.TEST.NET\SQLINS99,2222;Database=Test;Uid=python;Pwd=python;'
connection_prod = pypyodbc.connect(connection_string_prod)
cursor_prod = connection_prod.cursor()
get_files_to_load_query = "Get_Files_To_Load"
files_to_load = cursor_prod.execute(get_files_to_load_query)
for file_name in files_to_load:
load_query = "Load_Query_Stored_Proc #File_Name = '" + file_name + "'"
cursor_prod.execute(load_query)
cursor_prod.commit()
cursor_prod.close()
connection_prod.close()
In some posts it was suggested to use "SET NOCOUNT ON" at the top of the SQL stored procedure. I've done that already and it did not help with this issue.
Below is a code of a simple stored procedure that is causing the issue:
CREATE PROCEDURE [dbo].[Test]
AS
SET NOCOUNT ON
INSERT INTO Test(a)
SELECT 1
Why executing the stored procedure within a for loop only causes an issue?
Please advise.
Thank you!

You are using cursor_prod values to control the for loop and then using that same cursor object inside the loop to run the stored procedure, thus invalidating the previous state of the cursor for controlling the loop. The first iteration of the loop is where the cursor gets overwritten, which is why you don't encounter the error until you try to go through the loop a second time.
You don't need to create a second connection, but you do need to use a second cursor to execute the stored procedure. Or, you could use .fetchall to stuff all of the file names into a files_to_load list and then iterate over that list, freeing up the cursor to run the stored procedure.