Develop Reference

%s variable in Query Execution Python 3.8 (pymssql)

I have a python script with a basic GUI that logs into a DB and executes a query.
The Python script also asks for 1 parameter called "collection Name" which is taken from the tkinter .get function and is added as a %s inside the Query text. The result is that each time I can execute a query with a different "Collection name". This works and it is fine
Now, I want to add a larger string of Collection Names into my .get function so I can do cursor.execute a query with multiple collection names to get more complex data. But I am having issues with inputing multiple "collection names" into my app.
Below is a piece of my Query1, which has the %s variable that it then gets from the input to tkinter.
From #Session1
Join vGSMRxLevRxQual On(#Session1.SessionId = vGSMRxLevRxQual.SessionId)
Where vGSMRxLevRxQual.RxLevSub<0 and vGSMRxLevRxQual.RxLevSub>-190
and #Session1.CollectionName in (%s)
Group by
#Session1.Operator
Order by #Session1.Operator ASC
IF OBJECT_ID('tempdb..#SelectedSession1') IS NOT NULL DROP TABLE #SelectedSession1
IF OBJECT_ID('tempdb..#Session1') IS NOT NULL DROP TABLE #Session1
Here, is where I try to execute the query
if Query == "GSMUERxLevelSub" :
result = cursor.execute(GSMUERxLevelSub, (CollectionName,))
output = cursor.fetchmany
df = DataFrame(cursor.fetchall())
filename = "2021_H1 WEEK CDF GRAPHS().xlsx"
df1 = DataFrame.transpose(df, copy=False)
Lastly, here is where I get the value for the Collection name:
CollectionName = f_CollectionName.get()
enter image description here
enter code here

Your issues are due to a list/collection being a invalid parameter.
You'll need to transform collectionName
collection_name: list[str] = ['collection1', 'collection2']
new_collection_name = ','.join(f'"{c}"' for c in collection_name)
cursor.execute(sql, (new_collection_name,))
Not sure if this approach will be susceptible to SQL injection if that's a concern.
Edit:
Forgot the DBAPI would put another set of quotes around the parameters. If you can do something like:
CollectionName = ["foo", "bar"]
sql = f"""
From #Session1
Join vGSMRxLevRxQual On(#Session1.SessionId = vGSMRxLevRxQual.SessionId)
Where vGSMRxLevRxQual.RxLevSub<0 and vGSMRxLevRxQual.RxLevSub>-190
and #Session1.CollectionName in ({",".join(["%s"] * len(CollectionName))})
"""
sql += """
Group by
#Session1.Operator
Order by #Session1.Operator ASC
"""
cursor.execute(sql, (CollectionName,))
EDIT: Update to F-string

How to select all values in case of default in drop-down menu in sqlite3 query where clause

I want to display a data in QTableWidget according to QComboBoxes. In case of select all gender or select all ages, I want apply select all in the column in sqlite3 query
I want gender to be all
gender = "select all both male and female"
connection.execute("SELECT * FROM child where region=? and hospital=? and ageInMonths=? and gender=?", (region,hospital,ageInMonths,gender))

Welcome to Stackoverflow.
While it's a little tedious, the most sensible way to attack this problem is to build a list of the conditions you want to apply, and another of the data values that need to be inserted. Something like the following (untested) code, in which I assume that the variables are set to None if they aren't required in the search.
conditions = []
values = []
if region is not None:
conditions.append('region=?')
values.append(region)
# And similar logic for each other value ...
if gender is not None:
conditions.append('gender=?')
values.append(gender)
query = 'SELECT * FROM child'
if conditions:
query = query + ' WHERE ' + ' AND '.join(conditions)
connection.execute(query, values)
This way, if you want to include all values of a column you simply exclude if from the conditions by setting it to None.

You can build your where clause and your parameter list conditionally.
Below I am assuming that the ageInMonths and gender variables actually contain the value 'all' when this is selected on your form. You can change this to whichever value is actually passed to your code, if it is something different.
When it comes to your actual query, the best way to get all values for a field is to simply exclude it from the where clause of your query entirely.
So something like:
query_parameters = []
query_string = "SELECT * FROM child where region=? and hospital=?"
query_parameters.append(region)
query_parameters.append(hospital)
if ageInMonths != 'all':
query_string += " and ageInMonths=?"
query_parameters.append(ageInMonths)
if gender != 'all':
query_string += " and gender=?"
query_parameters.append(gender)
connection.execute(query_string, query_parameters)
Basically, at the same time we are testing and building the dynamic parts of the SQL statement (in query_string), we are also dynamically defining the list of variables to pass to the query in query_parameters, which is a list object.

Read optimisation cassandra using python

I have a table with the following model:
CREATE TABLE IF NOT EXISTS {} (
user_id bigint ,
pseudo text,
importance float,
is_friend_following bigint,
is_friend boolean,
is_following boolean,
PRIMARY KEY ((user_id), is_friend_following)
);
I also have a table containing my seeds. Those (20) users are the starting point of my graph. So I select their ID and search in the table above to get their Followers and friends, and from there I build my graph (networkX).
def build_seed_graph(cls, name):
obj = cls()
obj.name = name
query = "SELECT twitter_id FROM {0};"
seeds = obj.session.execute(query.format(obj.seed_data_table))
obj.graph.add_nodes_from(obj.seeds)
for seed in seeds:
query = "SELECT friend_follower_id, is_friend, is_follower FROM {0} WHERE user_id={1}"
statement = SimpleStatement(query.format(obj.network_table, seed), fetch_size=1000)
friend_ids = []
follower_ids = []
for row in obj.session.execute(statement):
if row.friend_follower_id in obj.seeds:
if row.is_friend:
friend_ids.append(row.friend_follower_id)
if row.is_follower:
follower_ids.append(row.friend_follower_id)
if friend_ids:
for friend_id in friend_ids:
obj.graph.add_edge(seed, friend_id)
if follower_ids:
for follower_id in follower_ids:
obj.graph.add_edge(follower_id, seed)
return obj
The problem is that the time it takes to build the graph is too long and I would like to optimize it.
I've got approximately 5 millions rows in my table 'network_table'.
I'm wondering if it would be faster for me if instead of doing a query with a where clauses to just do a single query on whole table? Will it fit in memory? Is that a good Idea? Are there better way?

I suspect the real issue may not be the queries but rather the processing time.
I'm wondering if it would be faster for me if instead of doing a query with a where clauses to just do a single query on whole table? Will it fit in memory? Is that a good Idea? Are there better way?
There should not be any problem with doing a single query on the whole table if you enable paging (https://datastax.github.io/python-driver/query_paging.html - using fetch_size). Cassandra will return up to the fetch_size and will fetch additional results as you read them from the result_set.
Please note that if you have many rows in the table that are non seed related then a full scan may be slower as you will receive rows that will not include a "seed"
Disclaimer - I am part of the team building ScyllaDB - a Cassandra compatible database.
ScyllaDB have published lately a blog on how to efficiently do a full scan in parallel http://www.scylladb.com/2017/02/13/efficient-full-table-scans-with-scylla-1-6/ which applies to Cassandra as well - if a full scan is relevant and you can build the graph in parallel than this may help you.

It seems like you can get rid of the last 2 if statements, since you're going through data that you already have looped through once:
def build_seed_graph(cls, name):
obj = cls()
obj.name = name
query = "SELECT twitter_id FROM {0};"
seeds = obj.session.execute(query.format(obj.seed_data_table))
obj.graph.add_nodes_from(obj.seeds)
for seed in seeds:
query = "SELECT friend_follower_id, is_friend, is_follower FROM {0} WHERE user_id={1}"
statement = SimpleStatement(query.format(obj.network_table, seed), fetch_size=1000)
for row in obj.session.execute(statement):
if row.friend_follower_id in obj.seeds:
if row.is_friend:
obj.graph.add_edge(seed, row.friend_follower_id)
elif row.is_follower:
obj.graph.add_edge(row.friend_follower_id, seed)
return obj
This also gets rid of many append operations on lists that you're not using, and should speed up this function.

Adding parameters to SQLITE3 SELECT column queries python

I am trying to streamLine queries to SQLITE3. I use it for financial price modelling and so am re-using the same basic query alot, but have to keep changing the hard coding to get out different column queries each time. So I want a generic query where I just write in what I want once, then it spits out the columns as lists. This is a basic version of what I want but basically still hard coded so you can see what I am trying to create.
dbName = 'NASDAQ_Equities'
ticker = 'AAPL'
def pullDataTest(dbPathName, ticker, *args):
datep = []
openp = []
highp = []
db = sqlite3.connect(dbPathName + '.mydb', detect_types=sqlite3.PARSE_DECLTYPES | sqlite3.PARSE_COLNAMES, timeout=3)
cursor = db.cursor()
cursor.execute('''SELECT ''' + str(args) + ''' FROM ''' + ticker)
for row in cursor:
datep.append(row[0])
openp.append(row[1])
highp.append(row[2])
pullData(dbName, ticker, 'datep', 'openp', 'highp')
At the moment I am lost on how to enter an *args into the select statement as it rejects it because of the () brackets. Also what will be an issue is creating empty lists and appending to those lists from from an *args. Would it be better to create a order dict to append to, then brake that into lists at the end somehow? On returning values for use later down the track I was thinking of making them globals? Any suggestions? Thanks

What is the correct way to form MySQL queries in python?

I am new to python, I come here from the land of PHP. I constructed a SQL query like this in python based on my PHP knowledge and I get warnings and errors
cursor_.execute("update posts set comment_count = comment_count + "+str(cursor_.rowcount)+" where ID = " + str(postid))
# rowcount here is int
What is the right way to form queries?
Also, how do I escape strings to form SQL safe ones? like if I want to escape -, ', " etc, I used to use addslashes. How do we do it in python?
Thanks

First of all, it's high time to learn to pass variables to the queries safely, using the method Matus expressed. Clearer,
tuple = (foovar, barvar)
cursor.execute("QUERY WHERE foo = ? AND bar = ?", tuple)
If you only need to pass one variable, you must still make it a tuple: insert comma at the end to tell Python to treat it as a one-tuple: tuple = (onevar,)
Your example would be of form:
cursor_.execute("update posts set comment_count = comment_count + ? where id = ?",
(cursor_.rowcount, postid))
You can also use named parameters like this:
cursor_.execute("update posts set comment_count = comment_count + :count where id = :id",
{"count": cursor_.rowcount, "id": postid})
This time the parameters aren't a tuple, but a dictionary that is formed in pairs of "key": value.

from python manual:
t = (symbol,)
c.execute( 'select * from stocks where symbol=?', t )
this way you prevent SQL injection ( suppose this is the SQL safe you refer to ) and also have formatting solved