I created a table with 100 records and now i want to generate a python function which takes a number and the table name and pulls records from the referenced table. ie. function should selects 'n' records from specified table.
I have already stated querying the database using python scripts. I can do a regular select but every time i want to select i have to edit the query. Is there a way for me to write a function in python that would take two parameters; eg.(n, table) that will allow me to select n records from any table in my database?
Is this possible and if it is where should i start?
You can use below function
def query_db( no_of_rows, table_name ):
cur = db.cursor()
query = """
select top %d rows from %s
""" %( int(no_of_rows), table_name )
cur.execute(query)
for row in cur.fetchall() :
print row[0]
Is this what you want, or am i missing something?
Related
I have a list comprised of several queries which are executed by a for loop. I would like to prompt the user to enter the origin (ilink) that will be utilized by the fourth query in the list.
The script runs fine when the origin is manually defined within the query. I have tried the following syntax which have all failed:
cursor.execute(lines, ilink)
cursor.execute(lines, [ilink])
cursor.execute(lines, (ilink))
I have also run the script with each query defined in its own cursor.execute(query) which accepts the argument, but does not pass any results due to multiple cursors.
import MySQLdb
ilink = raw_input("Choose and ilink to query (include 199N):" )
db = MySQLdb.connect(host="host",user="user",passwd="pass")
queries = [
"""USE monthly_audit;""",
"""DROP TEMPORARY TABLE IF EXISTS monthly_audit.tmp_order_ids;""",
"""DROP TEMPORARY TABLE IF EXISTS monthly_audit.tmp_internalselect;""",
"""CREATE TEMPORARY TABLE monthly_audit.tmp_order_ids AS
(SELECT DISTINCT order_id AS orders
FROM ng_tradeserver_db_history.fix_execution_reports_201906
WHERE FROM_UNIXTIME(TIMESTAMP/1000000) >= '2019-06-19 16:59:59'
AND FROM_UNIXTIME(TIMESTAMP/1000000) <= '2019-06-20 23:59:59'
AND TargetCompID = %s);""",]
cursor = db.cursor()
for lines in queries:
lines.split(",")
cursor.execute(lines, [ilink])
results = cursor.fetchall()
**This is only the relevant snippet of sql, total query is over 500 lines*
I expect the script to run the set of queries and return the results of said query to be stored in a csv. I am currently getting the following error when executing:
_mysql_exceptions.ProgrammingError: not all arguments converted during string formatting
I'm not sure if I understand your questions correct, but you can try using fstrings. I believe the quotes cause the problems during the string formatting.
Example:
query = f'''select ID, lat, lon from tbl order by st_distance(tbl.geom,st_setsrid(st_makepoint({lon},{lat}), 4326)) asc limit 1;'''
cursor.execute(query)
In this query the {lon}, {lat} are variables. Have a look at the docs for f strings https://docs.python.org/3/whatsnew/3.6.html
I would like to get names from one db and initiate a new table with it. I want to add more analysis, but this is my starting point where I'm already struggling and I have no idea where I made the mistake.
mydb = db_login()
# get team
team = pd.read_sql('SELECT * FROM team', con=mydb)
names = team.name.to_list()
this will output something like ['name1', 'name2' ...]
mycursor = mydb.cursor()
mycursor.execute("DROP TABLE IF EXISTS team_analyse")
mycursor.execute("CREATE TABLE team_analyse (name VARCHAR(50))") #todo add all needed columns
sqlFormula = "INSERT INTO team_analyse (name) VALUES (%s)" #todo initial team commit
mycursor.executemany(sqlFormula, names)
mydb.commit()
In the end I get the following error:
mysql.connector.errors.ProgrammingError: Not all parameters were used in the SQL statement
You should be using a single INSERT INTO ... SELECT here:
INSERT INTO team_analyse (name)
SELECT name
FROM team
Your updated Python script:
mycursor = mydb.cursor()
sql = """INSERT INTO team_analyse (name)
SELECT name
FROM team"""
mycursor.execute(sql)
The error message you are seeing is telling you that you passed a list as the parameters to bind to the statement, but not all parameters were used. Actually, the single insert statement you were trying to run only has a single parameter. But in any case, it is unnecessary to bring the result set from the team table into Python's memory. Instead, use my suggestion and let MySQL do the heavy lifting.
I have a MySQL database of some measurements taken by a device and I'm looking for a way to retrieve specific columns from it, where the user chooses what columns he needs from a python interface/front end. All the solutions I've seen till now either retrieves all columns or had the columns specified in the code itself.
Is there a possible way I could do this?
Thanks!
Your query can look something like this :
select
table_name, table_schema, column_name
from information_schema.columns
where table_schema in ('schema1', 'schema2')
and column_name like '%column_name%'
order by table_name;
you can definitely pass the column_name as a parameter(fetch it from python code) run it dynamically.
import MySQLdb
#### #GET COLUMN NAME FROM USER PRESENT WITH IN TABLE
column = input()
#### #Open database connection
db = MySQLdb.connect("host","username","password","DB_name" )
#### #prepare a cursor object using cursor() method
cursor = db.cursor()
#### #execute SQL query using execute() method.
cursor.execute("SELECT * FROM TABLE")
# Fetch a all rows using fetchall() method.
result_set = cursor.fetchall()
for row in result_set:
print(row[column])
# disconnect from server
db.close()
OR you can use .execute() to run a specific query with column name.
Say I need a table that has to have two columns (A TEXT, B TEXT).
Every time before I run a program, I want to check if the table exists, and create it if it doesn't. Now say that the table with that name exists already, but has only one column (A TEXT), or maybe (A INT, B INT)
So in general, different columns.
How do I check that on CREATE query? And if there's a conflict back it up somewhere and drop, then create a new correct table. If there's no conflict - don't do anything.
I am working in Python, using sqlite3 by the way. Database is stored locally for now and program is distributed to multiple people, that's why I need to check the database.
Currently I have
con = sqlite3.connect(path)
with con:
cur = con.cursor()
cur.execute('CREATE TABLE IF NOT EXISTS table (A TEXT, B TEXT);')
You can use the pragma table_info in order to get information about the table, and use the result to check your columns:
def validate(connection):
cursor = connection.cursor()
cursor.execute('PRAGMA table_info(table)')
columns = cursor.fetchall()
cursor.close()
return (len(columns) == 2
and columns[0][1:3] == ('A', 'TEXT')
and columns[1][1:3] == ('B', 'TEXT'))
So if validate returns False you can rename the table and create the new one.
I want to use sqlite3 in Python. I have a table with four columns (id INT, other_no INT, position TEXT, classification TEXT, PRIMARY KEY is id). In this table, the column for classification is left empty and will be updated by the information from table 2. See my code below. I then have a second table which has three columns. (id INT, class TEXT, type TEXT, PRIMARY KEY (id)). Basically, the two tables have two common columns. In both tables, the primary key is the id column, the classification and class column would eventually have to be merged. So the code needs to be able to go through table 2 and whenever it finds a matching id in table 1 to updating the class column (of table 1) from the classification column of table 2. The information to build the two tables comes from two separate files.
# function to create Table1...
# function to create Table2...
(the tables are created as expected). The problem occurs when I try to update table1 with information from table2.
def update_table1():
con = sqlite3.connect('table1.db', 'table2.db') #I know this is wrong but if how do I connect table2 so that I don't get error that the Table2 global names is not defined?
cur = con.cursor()
if id in Table2 == id in Table1:
new_classification = Table2.class # so now instead of Null it should have the class information from table2
cur.execute("UPDATE Table1 SET class = ? WHERE id =? ", (new_classification, id))
con.commit()
But, I get an error for line2: TypeError: a float is required. I know that it's because I put two parameters in the connect method. But then if I only connect with Table1 I get the error Table2 is not defined.
I read this post Updating a column in one table through a column in another table I understand the logic around it but I can't translate the SQL code into Python. I have been working on this for some time and can't seem to just get it. Would you please help? Thanks
After the comments of a user I got this code but it still doesn't work:
#connect to the database containing the two tables
cur.execute("SELECT id FROM Table1")
for row in cur.fetchall():
row_table1 = row[0]
cur.execute("SELECT (id, class) FROM Table2")
for row1 in cur.fetchall():
row_table2 = row[0] #catches the id
row_table2_class = row[1] #catches the name
if row_table1 == row_table2:
print "yes" #as a test for me to see the loop worked
new_class = row_table_class
cur.execute("UPDATE Table1 SET classification=? WHERE id=?", (new_class, row_table1))
con.commit()
From this however I get an operational error. I know it's my syntax, but like I said I am new to this so any guidance is greatly appreciated.
You need a lot more code than what you have there. Your code logic should go something like this:
connect to sqlite db
execute a SELECT query on TABLE2 and fetch rows. Call this rows2.
execute a SELECT query on TABLE1 and fetch rows. Call this rows1.
For every id in rows1, if this id exists in rows2, execute an UPDATE on that particular id in TABLE1.
You are missing SELECT queries in your code:
cur = con.cursor()
if id in Table2 == id in Table1:
new_classification = Table2.class
You can't just directly test like this. You need to first fetch the rows in both tables using SELECT queries before you can test them out the way you want.
Find below modified code from what you posted above. I have just typed that code in here directly, so I have not had the chance to test it, but you can look at it to get an idea. This could probably even run.
Also, this is by no means the most efficient way to do this. This is actually very clunky. Especially because for every id in Table1, you are fetching all the rows for Table2 everytime to match. Instead, you would want to fetch all the rows for Table1 once, then all the rows for Table2 once and then match them up. I will leave the optimization to make this faster upto you.
import sqlite3
#connect to the database containing the two tables
conn = sqlite3.connect("<PUT DB FILENAME HERE>")
cur = conn.execute("SELECT id FROM Table1")
for row in cur.fetchall():
row_table1_id = row[0]
cur2 = conn.execute("SELECT id, class FROM Table2")
for row1 in cur2.fetchall():
row_table2_id = row1[0] # catches the id
row_table2_class = row1[1] # catches the name
if row_table1_id == row_table2_id:
print "yes" # as a test for me to see the loop worked
new_class = row_table2_class
conn.execute("UPDATE Table1 SET classification=? WHERE id=?", (new_class, row_table1_id))
conn.commit()