I am new to working with SQL and Postgres specifically and am trying to write a simple program that stores a course id and some URLs in an SQL table with two columns. I am using the psycopg2 python library.
I am able to read from the table using:
def get_course_urls(course):
con = open_db_connection()
cur = con.cursor()
query = f"SELECT urls FROM courses WHERE course = '{course}'"
cur.execute(query)
rows = cur.fetchall()
cur.close()
close_db_connection(con)
urls = []
for url in rows:
urls.extend(url[0])
return urls
However, I am unable to insert into the table using:
def format_urls_string(urls):
return '{"' + '","'.join(urls) + '"}'
def add_course_urls(course, urls):
con = open_db_connection()
cur = con.cursor()
query = f"INSERT INTO courses (course, urls) VALUES ('{course}', '{format_urls_string(urls)}');"
print(query)
cur.execute(query)
cur.close()
close_db_connection(con)
add_course_urls("CS136", ["http://google.com", "http://wikipedia.com"])
I do not think anything is wrong with my query because when I run the same query in the SQL Shell it works as I want it to.
The locks on the columns say that the columns are READ-ONLY, however, I am able to insert through the shell. I feel like this is a very minor fix but since I am new to PostgreSQL, I am having some trouble.
Your help is appreciated!
This is the danger of doing the substitution yourself, instead of letting the db connector do it. You looked at your string, yes? You're writing
... VALUES ('CS136', '['http://google.com','http://wikipedia.com']')
which is obviously the wrong syntax. It needs to be
... VALUES ('CS136', '{"http://google.com","http://wikipedia.com"}')
which Python's formatter won't generate. So, you can either format the insertion string by hand, or put placeholders and pass the parameters to the cursor.execute call:
query = "INSERT INTO courses (course, urls) VALUES (%s,%s);"
cur.execute( query, (course, urls) )
Related
fairly new to SQL in general. I'm currently trying to bolster my general understanding of how to pass commands via cursor.execute(). I'm currently trying to grab a column from a table and rename it to something different.
import mysql.connector
user = 'root'
pw = 'test!*'
host = 'localhost'
db = 'test1'
conn = mysql.connector.connect(user=user, password=pw, host=host, database=db)
cursor = conn.cursor(prepared=True)
new_name = 'Company Name'
query = f'SELECT company_name AS {new_name} from company_directory'
cursor.execute(query)
fetch = cursor.fetchall()
I've also tried it like this:
query = 'SELECT company_name AS %s from company_directory'
cursor.execute(query, ('Company Name'),)
fetch = cursor.fetchall()
but that returns the following error:
stmt = self._cmysql.stmt_prepare(statement)
_mysql_connector.MySQLInterfaceError: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '? from company_directory' at line 1
I'm using python and mySQL. I keep reading about database injection and not using string concatenation but every time I try to use %s I get an error similar to the one below where. I've tried switching to ? syntax but i get the same error.
If someone could ELI5 what the difference is and what exactly database injection is and if what I'm doing in the first attempt qualifies as string concatenation that I should be trying to avoid.
Thank you so much!
If a column name or alias contains spaces, you need to put it in backticks.
query = f'SELECT company_name AS `{new_name}` from company_directory'
You can't use a placeholder for identifiers like table and column names or aliases, only where expressions are allowed.
You can't make a query parameter in place of a column alias. The rules for column aliases are the same as column identifiers, and they must be fixed in the query before you pass the query string.
So you could do this:
query = f"SELECT company_name AS `{'Company Name'}` from company_directory'
cursor.execute(query)
I am new in this and this is my first question. I hope you guys will help.
If my question format is wrong, feel free to comment on that also.
The code is pretty simple. I have DB connection, 2 functions - one for printing and another for choosing how many SQL queries I want to execute and input for those queries.
Idea is to enter a number(INT) of SQL queries - for example, 2 and then in another line user must enter 2 SQL queries.
After that, call_table function will print out current table status/situation/data.
For example - user wants to print out into console table data (table have 2 columns, [name][college], varchar type)
Insert a number of SQL queries you want to execute: 1
Insert SQL statement:
select * from student
('ivan', 'ino')
('nena', 'fer')
('tomislav', 'ino')
('marko', 'fer')
('tomislav', 'ino')
('marko', 'fer')
When I try to insert some values into the same table nothing happens with the table, data is not entered.
The query is 100% correct since I tested it in workbench, also I've tried to create another table from this program and the query was executed normally and the table was created.
I receive no errors.
Code is below:
import pymysql
db = pymysql.connect(host='localhost', user='root', passwd='123456', database='test')
mycursor = db.cursor()
def call_table(data_print):
for i in data_print:
print(i)
def sql_inputs(cursor):
container = []
no = int(input("Insert a number of SQL queries you want to execute: "))
for i in range(no):
container = [input("Insert SQL statement: \n").upper()]
for y in container:
cursor.execute(y)
sql_inputs(mycursor)
call_table(mycursor)
What am I doing wrong?
I tried even more complicated SQL queries but insert into the table is not working.
Thank you
Everything is good with the code, you're just missing cursor.commit()
By default cursor commit is false in python for insert queries.
cursor.execute(y)
cursor.commit()
and if you're done with queries
db.close()
You should append the queries to the container variable
import pymysql
db = pymysql.connect(host='localhost', user='root', passwd='123456', database='test')
mycursor = db.cursor()
def call_table(data_print):
for i in data_print:
print(i)
def sql_inputs(cursor):
container = []
no = int(input("Insert a number of SQL queries you want to execute: "))
for i in range(no):
container.append(input("Insert SQL statement: \n").upper())
for y in container:
cursor.execute(y)
sql_inputs(mycursor)
call_table(mycursor)
At the end of the program, I've added db.commit(), and everything works fine now.
import pymysql
db = pymysql.connect(host='localhost', user='root', passwd='45fa6cb2',
database='ivan')
mycursor = db.cursor()
def call_table(data_print):
for i in data_print:
print(i)
def sql_inputs(cursor):
container = []
no = int(input("Insert a number of SQL queries you want to execute: "))
for i in range(no):
container.append(input("Insert SQL statement: \n").upper())
for y in container:
cursor.execute(y)
sql_inputs(mycursor)
db.commit()
call_table(mycursor)
I have a Python application, in which I'm calling a MySQL stored procedure from my view, like so:
import mysql.connector
proc = 'audit_report'
parms = [data['schoolid'], dateToISO(data['startdatedefault'],'from'), dateToISO(data['enddatedefault'],'to'), joinIntList(data['studypgms'], joinWith), joinIntList(data['fedpgms'], joinWith), joinIntList(data['statuses'], joinWith), data['fullssndefault']]
conn = mysql.connector.connect(user='usr', database='db', password='pwd')
cursor = conn.cursor(dictionary=True)
cursor.callproc(proc, parms)
for result in cursor.stored_results():
print(result.fetchall())
I am getting the data returned as a list of tuples, the standard output. Since I'm using connector version 2.1.7, the docs say adding
dictionary=True
to the cursor declaration should cause the rowset to be returned as a list of dictionaries, with column name as the key of each dictionary. Main difference between my application and the example in the docs is that I'm using cursor.callproc(), whereas the examples use cursor.execute() with actual sql code.
I tried
print(cursor.column_names)
to see if I could get the column names that way, but all I get is
('#_audit_report_arg1', '#_audit_report_arg2', '#_audit_report_arg3', '#_audit_report_arg4', '#_audit_report_arg5', '#_audit_report_arg6', '#_audit_report_arg7')
which looks more like the input parameters to the stored procedure.
Is there any way to actually get the column names of the returned data? The procedure is somewhat complex and contains crosstab-type manipulation, but calling the same stored procedure from MySQL Workbench happily supplies the column names.
Normally, knowing what the output is supposed to be, I could hard-code column names, except this procedure crosstabs the data for the last few columns, and it is unpredictable what they will be until after the query runs.
Thanks...
You can use pymysql in python3 and it should work fine !!
import pymysql.cursors
connection = pymysql.connect(host='',
user='',
password='',
db='test',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
try:
with connection.cursor() as cursor:
# Read a single record
sql = "query"
cursor.execute(sql)
result = cursor.fetchone()
num_fields = len(cursor.description)
field_names = [i[0] for i in cursor.description]
print (field_names)
finally:
connection.close()
I've written my first 'update' query in python, while it seems correct, I'm not sure how to receive back the output to confirm it worked..
This is supposed to load a CSV file and replace the values in the first column with those in the second:
def main():
try:
conn=psycopg2.connect("dbname='subs' user='subs' host='localhost' password=''")
except:
print "I am unable to connect to the database."
sys.exit()
with open("dne.txt", "r+") as f:
for line in f:
old = line.split(',')[0].strip()
new = line.split(',')[1].strip()
cur = conn.cursor()
cur.execute("UPDATE master_list SET subs = '{0}' WHERE subs = '{1}';".format(new, old))
conn.commit()
results = cur.fetchall()
for each in results:
print str(each)
if __name__=="__main__":
main()
I thought the results (UPDATE 1 for each change?) would come back as a tuple, but I got an error instead:
psycopg2.ProgrammingError: no results to fetch
I'm not sure if this means my query just didn't work and there were no updates, or if I can't use fetchall() like I'm trying to.
Any feedback or suggestions welcome!
The UPDATE statement won't return any values as you are asking the database to update its data not to retrieve any data.
By far the best way to get the number of rows updated is to use cur.rowcount. This works with other drivers too, like with Psycopg2 for Postgresql it's the same syntax.
cur.execute("UPDATE master SET sub = ('xyz') WHERE sub = 'abc'")
print(cur.rowcount)
A more roundabout way of checking the update is by running a SELECT against the table after updating it; you should get the data returned. In my example below the first SELECT will return the row(s) where the update will happen. The second SELECT after the update should then return no rows as you have already updated all fields. The third SELECT should return the rows you have updated, plus any that already existed with the 'xyz' value.
import sqlite3
def main():
try:
conn=sqlite3.connect(":memory:")
cur = conn.cursor()
cur.execute("create table master(id text, sub text)")
cur.execute("insert into master(id, sub) values ('1', 'abc')")
cur.execute("insert into master(id, sub) values ('2', 'def')")
cur.execute("insert into master(id, sub) values ('3', 'ghi')")
conn.commit()
except:
print("I am unable to connect to the database.")
sys.exit()
cur.execute("select id, sub from master where sub='abc'")
print(cur.fetchall())
cur.execute("UPDATE master SET sub = ('xyz') WHERE sub = 'abc'")
conn.commit()
cur.execute("select id, sub from master where sub='abc'")
print(cur.fetchall())
cur.execute("select id, sub from master where sub='xyz'")
print(cur.fetchall())
if __name__=="__main__":
main()
In PostgreSQL 9.5 or later you can add RETURNING * to end your query that then returns the modified rows.
PostgreSQL docs: https://www.postgresql.org/docs/9.5/dml-returning.html
Sometimes it is useful to obtain data from modified rows while they
are being manipulated. The INSERT, UPDATE, and DELETE commands all
have an optional RETURNING clause that supports this. Use of RETURNING
avoids performing an extra database query to collect the data, and is
especially valuable when it would otherwise be difficult to identify
the modified rows reliably.
I have a python SQL query which is intended to call a procedure and then return that query in results however with the below code I only get this output:
adodbapi.adodbapi._SQLrows object at 0x0000000004CDE278
instead of it selecting the rows and data that it should be grabbing, I don't know anything about SQL and limited python so be gentle :)
# Connect to the SQL DB
conn = adodbapi.connect("Provider=SQLOLEDB; SERVER=xx.x.x.x; Initial Catalog=master_db;User Id=User; Password=Pass; ")
curs = conn.cursor()
# Execute SQL procedure "
curs.execute('util.referencing_procedure', )
results = curs.fetchall()
print results
conn.close()
You need to iterate through the results. Probably:
for row in results:
print row