Python mysql fetchall() returns additional characters - python

I am trying to retrieve values from a table to use in a calculation. The code below
mycursor = mydb.cursor()
mycursor.execute("SELECT number FROM info")
rows = mycursor.fetchall()
print (rows)
This returns this list
[(Decimal('30.00'),), (Decimal('66.00'),), (Decimal('72.00'),)]
How can I retrieve the numerical value only either in a list or tuple like
[30.00, 66.00, 72.00]

The original data type in mydb might be a Decimal object.
So you can cast the datatype in the MySQL query or python code.
1) Case in MySQL:
SELECT CAST(number AS DOUBLE) FROM info
but this code returns the fetched rows as
[(30.00,), (66.00,), (72.00,)] because the tuples represent the columns of the query result.
2) Case in python code:
converted_rows = list(map(lambda row:float(row[0]), rows))
It will return [30.00, 66.00, 72.00] list.

Related

How to pass tuple with one element as param in sql query Python?

How can I pass tuple with only one element as param in sql query Python?
I have tried this solution suggested here:
imploding a list for use in a python MySQLDB IN clause
ids = [(UUID('1232a4df-5849-46c2-8c76-74fe74168d82'),)] # list of ids
cursor.execute('''
SELECT
...
WHERE
id IN %(ids)s
AND created_at > %(start_dt)s
''', {
'ids': tuple(ids), 'start_dt': '2019-10-31 00:00:00'
})
but this solution does not work in python 3 and mysql-connector-python 8.0.21.
I get this error:
An error "Python tuple cannot be converted to MySQL type" is returned.
I don't think what you are trying to do is possible in the Python3 version of the MySQL Connector. The code converts Python dict values into SQL syntax, depending on the Python type. But in the connector code, only scalar types are supported, no list or tuple. See https://github.com/mysql/mysql-connector-python/blob/master/lib/mysql/connector/conversion.py#L221-L372
I'd use the solution in the accepted answer to imploding a list for use in a python MySQLDB IN clause, which is to generate N placeholders for the number of items in your list, plus one more placeholder for the value to compare to created_at. Then merge the list with the date value and pass that.
ids = ['1232a4df-5849-46c2-8c76-74fe74168d82'] # list of ids
id_placeholders = ','.join(['%s'] * len(ids))
query = f'''
SELECT *
FROM mytable
WHERE
id IN ({id_placeholders})
AND created_at > %s
'''
params = ids
params.append('2019-10-31 00:00:00')
cursor.execute(query, params)

iterating over tuples inside a tuple and using string inside tuples as a variable

I have many number of tables inside a database, I am using pymysql to connect to my db and got all table names. When I print all table names they are stored something like follows:
tables = (('table1',), ('table2',), ('table3',), etc...)
What I want to do is for each table, turn them into a dataframe with their own table name. This is what I've tried:
for table in tables:
table[0] = pd.read_sql(f'select * from {table[0]}', con = conn)
and also by converting tuples into a list, however I think the string is the problem, how do I get rid of apostrophe so I can use them as a variable or is there no such way?
I couldn't find any relevant question on SO, only some stuff that gave me some ideas.
TypeError: 'tuple' object does not support item assignment when swapping values
How can I iterate over only the first variable of a tuple
tuple are immutable object you cannot do this, the result returned by first query is a tuple of tuple:
some_tuple[dome_index] = value
this will generate a tuple of tuple with one element is the dataframe:
tables = tuple((pd.read_sql(f'select * from {table_name}', con = conn),) for (table_name,) in tables)
EDITS:
You could create a tuple of pairs table_name, dataframe:
tables = tuple((table_name, pd.read_sql(f'select * from {table_name}', con = conn)) for (table_name,) in tables)
You need dictionary:
tables = {table_name: pd.read_sql(f'select * from {table_name}', con = conn) for (table_name,) in tables}

Cannot Use the Value of an Item in SQL Database

I Have this example of sql database. I want to use a certain item from the database in math calculation but I can't because the value looks like this: (25.0,) instead of just 25.0. Please see the attached picture
https://imgur.com/a/j7JOZ5H
import sqlite3
#Create the database:
connection = sqlite3.connect('DataBase.db')
c = connection.cursor()
c.execute('CREATE TABLE IF NOT EXISTS table1 (name TEXT,age NUMBER)')
c.execute("INSERT INTO table1 VALUES('Jhon',25)")
#Pull out the value:
c.execute('SELECT age FROM table1')
data =c.fetchall()
print(data[0])
#simple math calculation:
r=data[0]+1
print(r)
According to Python's PEP 249, the specification for most DB-APIs including sqlite3, fetchall returns a sequence of sequences, usually list of tuples. Therefore, to retrieve the single value in first column to do arithmetic, index the return twice: for specific row and then specific position in row.
data = c.fetchall()
data[0][0]
Alternatively, fetchone returns a single row, either first or next row, in resultset, so simply index once: the position in single row.
data = c.fetchone()
data[0]
The returned data from fetchall always comes back as a list of tuples, even if the tuple only contains 1 value. Your data variable should be:
[(25,)]
You need to use:
print(data[0][0])
r = data[0][0] + 1
print(r)

Python CX_Oracle: How to use returned tuples in another query

I am using cx_Oracle to query my databse with Python. I have a query that returns multiple results. In cx_Oracle using the .fetchall() command on a query that returns multiple results puts each row into a tuple, and then makes a list of tuples (1 for every returned row).
After fetching the results of my query it is formatted like this:
[('R100',), ('R200',)]
I now want to use those results in another query. The next query is as follows:
base_query = "select MODEL from USER.TABLE where SERIES in :series"
where :series is a special parameter marker that can be substituted when you execute the query like this:
cursor.execute(base_query, series=[('R100',), ('R200',)])
When I attempt to set series to my list of tuples I get this error:
cx_Oracle.NotSupportedError: element 0 value is unsupported
I understand that this is probably a syntax issue since in raw SQL what I am trying to to is probably creating a query that reads like this:
base_query = "select MODEL from USER.TABLE where SERIES in [('R100',), ('R200',)]"
when what I actually want is this:
base_query = "select MODEL from USER.TABLE where SERIES in ('R100','R200')
I am having trouble making the parsed raw query look like the second example though because I am unsure as to how python data types are being interpreted (I am guessing my 1st example isn't even a correct interpretation of what the raw SQL looks like)
UPDATE:
So I think you are supposed to be able to do this using by doing:
cursor.executemany(base_query, [('R100',), ('R200',)])
But I am getting an error: cx_Oracle.DatabaseError: DPI-1013: not supported
I am on cx_oracle Ver 7.0.0 trying to figure out what version my DB is now
You can use this function to convert the format and then use it in your query.
def to_tuple(first_output):
series = []
for py_tuple in first_output:
series.append(py_tuple[0])
return tuple(series)
series = to_tuple(first_output) # Output : ('R100', 'R200')
query = "select MODEL from USER.TABLE where SERIES in {}".format(series)
#Output:"select MODEL from USER.TABLE where SERIES in ('R100', 'R200')"
Got it. Posting here for anyone else doing this. Basically you have to dynamically generate n number of query params and convert your list of tuples to a list of strings.
# var series is what you will get from a query that returns multiple results
print(series) # >> [('R100',), ('R200',)]
tuple_list_to_str_list = [str(i[0]) for i in results['series']]
print(tuple_list_to_str_list) # >> ['R100', 'R200']
vars_list = ','.join(':%d' % i for i in range(len(results['series'])))
print(vars_list) # Makes n number of query params >> :0,:1
base_query = "select MODEL from USER.TABLE where SERIES in (%s)" % vars_list # Base query
print(base_query) # select MODEL from USER.TABLE where SERIES in (:0,:1)
cursor.execute(base_query, tuple_list_to_str_list)

Why is `for...in` returning a tuple when trying to iterate through rows returned by query?

I select 1 column from a table in a database. I want to iterate through each of the results. Why is it when I do this it’s a tuple instead of a single value?
con = psycopg2.connect(…)
cur = con.cursor()
stmt = "SELECT DISTINCT inventory_pkg FROM {}.{} WHERE inventory_pkg IS NOT NULL;".format(schema, tableName)
cur.execute(stmt)
con.commit()
referenced = cur.fetchall()
for destTbl in referenced:#why is destTbl a single element tuple?
print('destTbl: '+str(referenced))
stmt = "SELECT attr_name, attr_rule FROM {}.{} WHERE ppm_table_name = {};".format(schema, tableName, destTbl)#this fails because the where clause gets messed up because ‘destTbl’ has a comma after it
cur.execute(stmt)
Because that's what the db api does: always returns a tuple for each row in the result.
It's pretty simple to refer to destTbl[0] wherever you need to.
Because you are getting rows from your database, and the API is being consistent.
If your query asked for * columns, or a specific number of columns that is greater than 1, you'd also need a tuple or list to hold those columns for each row.
In other words, just because you only have one column in this query doesn't mean the API suddenly will change what kind of object it returns to model a row.
Simply always treat a row as a sequence and use indexing or tuple assignment to get a specific value out. Use:
inventory_pkg = destTbl[0]
or
inventory_pkg, = destTbl
for example.

Categories

Resources