This subject can be already covered. If so, apologizes for that. I have a problem when fetching rows from database (with "for" and "while" loops) and executing the script from console.
I need to fetch a huge amount of rows from database and I'm building a script so that i can insert the user ID's and i'll get the account ID's of the clients and the statuses.
I have realized that when I run the script from Eclipse, full output is fetched from DB. When I run the script from console, there is a limit of rows. So, i'd like to know if i have a "while row is not None" loop... why my row becomes None if there are more rows in database??
Also: I need to solve this issue. No matter how. I'd prefer to not load the full list to a local file (if possible). But, if there is no other way... okay, please help!! This is my example.
#!/usr/bin/env python3
# encoding: utf-8
import configparser
import pymysql
from prettytable import PrettyTable
conn = pymysql.connect(host=host, user=user, passwd=password, db=database)
print()
userid = input('Insert User ID(s) > ') # userid is a list of 2000 users in comma-separated format
userids = userid.replace(" ", "").replace("'", "").replace(",", "','")
cur = conn.cursor()
cur.execute(""" SELECT user_ID, account_ID, status FROM Account WHERE user_ID IN ('%s'); """ % userids)
rows = cur.fetchall()
table = PrettyTable(["user_ID", "account_ID", "status"])
table.padding_width = 1
for line in rows:
user_ID = str(line[0])
account_ID = str(line[1])
status = str(line[2])
table.add_row([user_ID, account_ID, status])
print()
print(table)
conn.close()
print()
print()
exit()
Without modifying too much your code you could try using from_db_cursor() provided by the prettytable module like this:
#!/usr/bin/env python3
import pymysql
from prettytable import from_db_cursor
conn = pymysql.connect(host=host, user=user, passwd=password, db=database)
userid = input('Insert User ID(s) > ') # userid is a list of 2000 users in comma-separated format
userids = userid.replace(" ", "").replace("'", "").replace(",", "','")
cur = conn.cursor()
cur.execute(""" SELECT user_ID, account_ID, status FROM Account WHERE user_ID IN ('%s'); """ % userids)
table = from_db_cursor(cur, padding_width=1)
print(table)
conn.close()
But looking at the source code of prettytable my guess is that it won't change the situation much since it does more or less what you were doing explicitly in your code.
What would probably work better is to add rows one at a time to table instead of fetching all rows and looping through them to add them. Something like:
#!/usr/bin/env python3
import pymysql
from prettytable import PrettyTable
conn = pymysql.connect(host=host, user=user, passwd=password, db=database)
userid = input('Insert User ID(s) > ') # userid is a list of 2000 users in comma-separated format
userids = userid.replace(" ", "").replace("'", "").replace(",", "','")
cur = conn.cursor()
cur.execute(""" SELECT user_ID, account_ID, status FROM Account WHERE user_ID IN ('%s'); """ % userids)
row = cur.fetchone()
table = PrettyTable(["user_ID", "account_ID", "status"], padding_width=1)
while row is not None:
table.add_row(row)
row = cur.fetchone()
print(table)
conn.close()
You don't need to transform the row elements to string values since Prettytable does that internally.
But it is possible to take advantage of different features to simplify your code and to make it more pythonic:
#!/usr/bin/env python3
import re
import pymysql
from prettytable import PrettyTable
userid = input('Insert User ID(s) > ') # userid is a list of 2000 users in comma-separated format
userids = re.sub("[ ']", "", userid).replace(",", "','")
table = PrettyTable(["user_ID", "account_ID", "status"], padding_width=1)
with pymysql.connect(host=host, user=user, passwd=password, db=database) as cur:
cur.execute("""SELECT user_ID, account_ID, status FROM Account WHERE user_ID IN ('%s'); """ % userids)
map(table.add_row, cur)
print(table)
In this version:
I have used the re.sub() to do some of the substitutions (some may say that's a tad overkill but it could be useful to you in the future)
A Pymysql connection implements the context manager that provides you directly with a cursor.
Since a Pymysql cursor can provide an iterator we can use it with map() to go through all the rows even if we don't care for the result.
Related
firstly apologies for the basic question, just starting off with Python.
I have the following code:
import sqlite3
conn = sqlite3.connect("test.sqb")
cursor = conn.cursor()
sql = "SELECT * FROM report WHERE type LIKE 'C%'"
cursor.execute(sql)
data = cursor.fetchall()
for row in data:
print (row[0])
cursor.execute("UPDATE report SET route='ABCDE'")
conn.commit()
conn.close()
Why is it updating all records and not just the filtered records from sql query, even though the print (row[0]) just shows the filtered records.
Many thanks.
What's actually happening is you are running this query for each record returned from the SELECT query.
UPDATE report SET route='ABCDE'
If you only want to update route where type starts with C add the criteria to the UPDATE query and execute it once.
import sqlite3
conn = sqlite3.connect("test.sqb")
cursor = conn.cursor()
sql = "SELECT * FROM report WHERE type LIKE 'C%'"
cursor.execute(sql)
data = cursor.fetchall()
cursor.execute("UPDATE report SET route='ABCDE' WHERE type LIKE 'C%'")
conn.commit()
conn.close()
I have an issue to run my SQL queries on a Postgres ElephantSql hosted:
This is my code to connect (except dynamo, user, password which are replaced by XXX
DATABASE_URL = 'postgres://YYYY:ZZZZ#drona.db.elephantsql.com:5432/YYYY'
# ---------------------------- CONNECT ELEPHANT DB
def ElephantConnect():
up.uses_netloc.append("postgres")
url = up.urlparse(DATABASE_URL)
conn = psycopg2.connect(dbname='YYYY',
user='YYYY',
password='ZZZZ',
host='drona.db.elephantsql.com',
port='5432'
)
cursor = conn.cursor()
# cursor.execute("CREATE TABLE notes(id integer primary key, body text, title text);")
#conn.commit()
# conn.close()
return conn
this code seems to connect well to db
My issue is when I want to delete a table:
def update(df, table_name, deleteYes= 'Yes'):
conn = ElephantConnect()
db = create_engine(DATABASE_URL)
cursor =conn.cursor()
if deleteYes == 'Yes': # delete
queryCount = "SELECT count(*) FROM {};".format(table_name)
queryDelete = "DELETE FROM {};".format(table_name)
count = db.execute(queryCount)
rows_before = count.fetchone()[0]
try:
db.execute(queryDelete)
logging.info('Deleted {} rows into table {}'.format(rows_before, table_name))
except:
logging.info('Deleted error into table {}'.format(table_name))
else:
pass
It seems when I run db.execute(queryDelete), it goes to the exception.
I have no message of error. But the query with count data is working...
thanks
I think that the reason for the error is because there are foreign keys against the table. In order to be sure, assign the exception into a variable and print it:
except Exception as ex:
print(ex)
By the way, if you want to quickly delete all of the rows from a table then
It will be much more efficient to truncate the table instead of deleting all the rows:
truncate table table_name
Delete is more useful when you want to delete rows under some conditions:
delete from table_name where ...
I have the following function:
def credential_check(username, password):
conn = sqlite3.connect('pythontkinter.db')
c = conn.cursor()
idvalue = c.execute('''SELECT ID FROM userdetails WHERE username = "{0}"'''.format(username)).fetchall()
print(idvalue)
I wish to assign the value of ID in my userdetails table to the variable idvalue in the row where the inputted username = userdetails username, however when I use this fetchall() I get [('0',)] printed out rather than just 0.
How do I go about doing this?
Thanks
You can use fetchone() if you only want one value. However, the result will still be returned as a tuple, just without the list.
import sqlite3
conn = sqlite3.connect('test.db')
c = conn.cursor()
c.execute('''CREATE TABLE IF NOT EXISTS testing(id TEXT)''')
conn.commit()
c.execute("""INSERT INTO testing (id) VALUES ('0')""")
conn.commit()
c.execute("""SELECT id FROM testing""")
data = c.fetchone()
print data
# --> (u'0',)
You can also use LIMIT if you want to restrict the number of returned values with fetchall().
More importantly, don't format your queries like that. Get used to using the ? placeholder as a habit so that you are not vulnerable to SQL injection.
idvalue = c.execute("""SELECT ID FROM userdetails WHERE username = ?""", (username,)).fetchone()
I am trying to query the records for a specific ID in an Oracle table based on what the user inputs.
Here is my code:
import cx_Oracle
con = cx_Oracle.connect('dbuser/dbpassword#oracle_host/service_ID')
cur = con.cursor()
id_number = raw_input('What is the ID Number?')
cur.execute('select id, info from oracle_table_name where id=:id_number')
for result in cur:
print "test", result
cur.close()
con.close()
The following error pops up: cx_Oracle.DatabaseError: ORA-01008: not all variables bound
When I remove the user input and the variable substitution and run the query, everything works fine.
:id_number in your SQL is a parameter (variable). You need to provide its value.
execute method accepts parameters as the second argument.
Example:
query = "select * from some_table where col=:my_param"
cursor.execute(query, {'my_param': 5})
Check the documentation at http://cx-oracle.readthedocs.org/en/latest/cursor.html#Cursor.execute
I assigned a name to the user_value:
user_value = raw_input('What is the ID Number?')
And then referenced it in the execute statement:
cur.execute(query, {'id': (user_value)})
Thanks to Radoslaw-Roszkowiak for the assist!!
I'm running pyodbc connected to my db and when i run a simply query I get a load of results back such as
(7L, )(12L,) etc.
How do I replace the the 'L, ' with '' so I can pass the ids into another query
Thanks
Here's my code
import pyodbc
cnxn = pyodbc.connect('DSN=...;UID=...;PWD=...', ansi=True)
cursor = cnxn.cursor()
rows = cursor.execute("select id from orders")
for row in rows:
test = cursor.execute("select name from customer where order_id = %(id)s" %{'id':row})
print test
Use parameters:
...
test = cursor.execute("select name from customer where order_id = ?", row.id)
...
The L after the number indicates that the value is a long type.