So I am trying to do an IF statement after the select is done in python. Hard to explain, better to provide an example of it:
customer = cursor.execute("SELECT * FROM customer")
try:
customer['name']="john"
(do something here)
except:
customer['name']!="john"
(do something else here)
I hope this makes sense and sorry if this is not enough information. Trying to think of how to explain this. I don't want it to do a WHERE statement in the select because I don't want it to NOT SELECT certain information just because their 'name' is not "john"
First of all, you are not telling us which library you are using for working with MySQL. I assume its MySQLdb.
After executing SQL query with cursor.execute you have to call fetchall or fetchone. In later case you will be able to do customer['name'].
Example:
results = cursor.execute("SELECT * FROM customers")
customer = cursor.fetchone()
You shouldn't be using try/except since there is not an obvious case when any exception is thrown or is expected.
Related
I'm completely stumped.
The query looks something like this:
WITH e AS(
INSERT INTO TEAMS(TEAM_NAME, SPORT_ID, TEAM_GENDER)
VALUES ('Cameroon U23','1','M')
ON CONFLICT (TEAM_NAME, SPORT_ID, TEAM_GENDER)
DO NOTHING
RETURNING TEAM_ID
)
SELECT * FROM e
UNION
SELECT TEAM_ID FROM TEAMS WHERE LOWER(TEAM_NAME)=LOWER('Cameroon U23') AND SPORT_ID='1' AND LOWER(TEAM_GENDER)=LOWER('M');
And the python code like this:
sqlString = """WITH e AS(
INSERT INTO TEAMS(TEAM_NAME, SPORT_ID, TEAM_GENDER)
VALUES (%s,%s,%s)
ON CONFLICT (TEAM_NAME, SPORT_ID, TEAM_GENDER)
DO NOTHING
RETURNING TEAM_ID
)
SELECT * FROM e
UNION
SELECT TEAM_ID FROM TEAMS WHERE LOWER(TEAM_NAME)=LOWER(%s) AND SPORT_ID=%s AND LOWER(TEAM_GENDER)=LOWER(%s);"""
cur.execute(sqlString, (TEAM_NAME, SPORT_ID, TEAM_GENDER, TEAM_NAME, SPORT_ID, TEAM_GENDER,))
fetch = cur.fetchone()[0]
The error that I get is on "cur.fetchone()[0]" because "cur.fetchone()" doesn't return any values for some reason. I have also tried "cur.fetchall()" but it's the same issue.
This query works every time without fail in the normal postgres shell. However, in my python code using psycopg2, it will sometimes error out and not return anything. When I check the DB from the shell, the data I am looking for is there so it is the select query that should be returning something but isn't.
I am not sure if this is relevant, but I am creating concurrent connections (not connection pools) and doing multiple of these queries at once. Each query has a different team, however, to prevent deadlock.
I have found the issue. It was to do with me using concurrency. I was wrong in saying that each query has a different team. The teams might sometimes be the same.
But the main issue was occurring because my INSERT would try and put some data in and find a duplicate because a concurrent query was also trying to put the same data in. But then for some reason, the SELECT wouldn't find that data. I don't exactly what the issue is but that's my understanding.
I had to change to doing a SELECT, checking if there was a result, then doing an INSERT if there wasn't and then doing a final SELECT if the INSERT didn't return anything. The INSERT does not return anything sometimes because it encounters a conflict with an entry that appeared after the first SELECT was executed.
EDIT:
Nevermind. The problem was, in fact, that my deadlock_timeout was too low. My program wasn't actually reaching deadlock (where two processes are waiting on each other and cannot because they are also dependent on the other finishing). So increasing the deadlock_timeout to be larger than the average time for one of my processes to complete was the solution.
THIS WILL NOT WORK if your program is actually reaching deadlock. In that case, fix it, because it should not be reaching deadlock ever.
Hope this helps someone.
I am writing a simple python program to retrieve information from a localhost database i set up.
I am encountering the following error:
('24000', '[24000] [Microsoft][ODBC Driver 17 for SQL Server]Invalid cursor state')
My code:
try:
cursor = conn.cursor()
cursor.execute(getSQLStatement('SelectRecipe'),[item])
recipe = cursor.fetchone()[0]
cursor.close()
except Exception as e:
print(e)
return
The query i am running works fine in the database client, the query isnt the issue, it is a simple SELECT statement:
SELECT
Recipe
FROM recipes
WHERE Name = ? --(item)
My desired result is just the most basic statement just so i can get it running, and then be able to expand the complexity later, but something is wrong at this level. I am just trying to get the value of one record and one column i believe with this code. When i searched other similar issues people had with this the answer was always related to multiple result sets being attempted to be returned? My query is just one result set and i am only running it once? I am kind of scratching my head here to find what i am missing?
The error seems to be related to the:
recipe = cursor.fetchone()[0]
...line as it runs fine without throwing the error without this line, however then i am not sure how to get the data as a variable if i dont do it that way.
Wondering if anyone could help me out here, if there is a better way to get my data or if there is something wrong with my implementation of this way. Thanks.
I am trying to use the sqlite3 module in python to do a database lookup in a table that only has one column. The column contains phone numbers in the format of:
9545551212
???5551212
Here's what I am running in python:
import sqlite3
cti = '/home/user/test/cti.db'
conn = sqlite3.connect(cti)
c = conn.cursor()
c.execute('select * from ani_table_1 where number = 9545551212')
<sqlite3.Cursor object at 0x7f6b435316c0>
When I run that exact same select statement in sqlite3 I get the expected result:
sqlite> select * from ani_table_1 where number = 9545551212;
9545551212
I'm using python 3.6.5 and sqlite 3.7.17
What have I got wrong in my code? Any help is much appreciated.
You didn't iterate over the result. The sqlite3 command line tool is not the same thing as Python code; the latter always prints the results, because it is a command-line tool and will make sure you don't get flooded with large result sets.
When accessing a database in code, however, the library can't assume you want to print out all the rows to the end user. You maybe wanted to do something different with the data instead.
So you need to loop over the cursor and print each row:
c.execute('select * from ani_table_1 where number = 9545551212')
for row in c:
print(*row, sep='\t')
You may want to familiarise yourself with how the Python database API standard works; search around for a good tutorial. At a glance, this specific tutorial looks like it covers the most important basics.
I have the following code:
def executeQuery(conn, query):
cur = conn.cursor()
cur.execute(query)
return cur
def trackTagsGenerator(chunkSize, baseCondition):
""" Returns a dict of trackId:tag limited to chunkSize. """
sql = """
SELECT track_id, tag
FROM tags
WHERE {baseCondition}
""".format(baseCondition=baseCondition)
limit = chunkSize
offset = 0
while True:
trackTags = {}
# fetch the track ids with the coresponding tag
limitPhrase = " LIMIT %d OFFSET %d" % (limit, offset)
query = sql + limitPhrase
offset += limit
cur = executeQuery(smacConn, query)
rows = cur.fetchall()
if not rows:
break
for row in rows:
trackTags[row['track_id']] = row['tag']
yield trackTags
I want to use it like this:
for trackTags in list(trackTagsGenerator(DATA_CHUNK_SIZE, baseCondition)):
print trackTags
break
This code produces the following error without even fetching one chunk of track tags:
Exception _mysql_exceptions.ProgrammingError: (2014, "Commands out of sync; you can't run this command now") in <bound method SSDictCursor.__del__ of <MySQLdb.cursors.SSDictCursor object at 0x10b067b90>> ignored
I suspect it's because I have the query execute logic in the body of loop in the generator function.
Is someone able to tell me how to fetch chunks of data using mysqldb in such way?
I'm pretty sure this is because it can run into situations where you've got two queries
running simultaniously because of the yield. Depending on how you call the function (threads, async, etc..) I'm pretty sure your cursor might get clobbered too?
As well, you're opening yourself up to (sorry, but I can't sugar coat this part) horrific SQL injection holes by inserting baseConditional using essentially a printf. Take a look at the DB-API’s parameter substitution docs for help.
Yield isn't going to save you time or energy here at all, the full sql command will always need to run before you'll get a single result. (Hence you're using LIMIT and OFFSET to make it more friendly, kudos)
i.e. someone updates the table while you're yielding out some data, in this particular case - not the end of the world. In many others, it gets ugly.
If you're just goofing around and you want this to work 'right-now-dammit', it'd probably work to modify executeQuery as such:
def executeQuery(conn, query):
cur = conn.cursor()
cur.execute(query)
cur = executeQuery(smacConn, query)
rows = cur.fetchall()
cur.close()
return rows
One thing that also kinda jumps out at me - you define trackTags = {}, but then you update tagTrackIds, and yield trackTags.. Which will always be empty dict.
My suggestion would be to not bother yourself with the headache of hand writing SQL if you're just trying to get a hobby project working. Take a look at Elixir which is built on top of SQLAlchemy.
Using an ORM (object-relational-mapper) can be a much more friendly introduction to databases. Defining what your objects look like in Python, and having it automatically generate your schema for you - and being able to add/modify/delete things in a Pythonic manner is really nifty.
If you really need to be async, check out ultramysql python module.
You use a SSDictCursor, something that maps to mysql_use_result() on MySQL-API-side. This requires that you read out the complete result before you can issue a new command.
As this happens before you receive the first chunk of data after all: are yu sure that this doesn't happen in the context of the query before this part of code is executed? The results of that last query might be still in the line, and executing the next one (i. e., the fist one in this context) might break things...
It should be simple, bit I've spent the last hour searching for the answer. This is using psycopg2 on python 2.6.
I need something like this:
special_id = 5
sql = """
select count(*) as ct,
from some_table tbl
where tbl.id = %(the_id)
"""
cursor = connection.cursor()
cursor.execute(sql, {"the_id" : special_id})
I cannot get this to work. Were special_id a string, I could replace %(the_id) with %(the_id)s and things work well. However, I want it to use the integer so that it hits my indexes correctly.
There is a surprising lack of specific information on psycopg2 on the internet. I hope someone has an answer to this seemingly simple question.
Per PEP 249, since in psycopg2 paramstyle is pyformat, you need to use %(the_id)s even for non-strings -- trust it to do the right thing.
BTW, internet searches will work better if you use the correct spelling (no h there), but even if you mis-spelled, I'm surprised you didn't get a "did you mean" hint (I did when I deliberately tried!).