having trouble with these two functions
was wondering if people could tell me where I am going wrong
this is a separate function as part of a spider that searches through a website of house prices
def save_house_to_db(id, address, postcode, bedrooms):
conn = sqlite3.connect('houses_in_london.db')
d = conn.cursor()
d.execute('INSERT INTO TABLE houses (id, address, postcode, bedrooms) VALUES (%d %s %s %d)' %(id, str(address), str(postcode), float(bedrooms)))
d.commit()
d.close()
def save_transactions_to_db(id, sale_price, date):
conn = sqlite3.connect('houses_in_london.db')
d = conn.cursor()
d.execute('INSERT INTO TABLE transactions (transaction_id NOT NULL AUTO_INCREMENT, house_id, date, sale_price) VALUES'
'(%d %s %s)' %(id, sale_price, str(date)))
d.commit()
d.close()
here is the error raised:
Traceback (most recent call last):
File "/Users/saminahbab/Documents/House_Prices/final_spider.py", line 186, in <module>
final_function(link_set=areas,id_counter=40)
File "/Users/s/Documents/House_Prices/final_spider.py", line 158, in final_function
page_stripper(link=(root+page), id_counter=id_counter)
File "/Users/s/Documents/House_Prices/final_spider.py", line 79, in page_stripper
save_house_to_db(id=float(id_counter), address=address, postcode=postcode, bedrooms=bedrooms)
File "/Users/s/Documents/House_Prices/final_spider.py", line 25, in save_house_to_db
d.execute('INSERT INTO TABLE houses VALUES (%d %s %s %d)' %(id, str(address), str(postcode), float(bedrooms)))
sqlite3.OperationalError: near "TABLE": syntax error
and for reference here is the execute for the databse
# conn = sqlite3.connect('houses_in_london.db')
# database = conn.cursor()
# database.execute('CREATE TABLE houses (id INTEGER PRIMARY KEY, address TEXT,'
# 'postcode TEXT, bedrooms TEXT)')
#
# database.execute('CREATE TABLE transactions (transaction_id NOT NULL AUTO_INCREMENT, house_id INTEGER '
# ' REFERENCES houses(id), date TEXT, sale_price INTEGER )')
as always, thank you for the support
You have many issues:
INSERT-clause has no TABLE keyword
You're trying to pass variables to an SQL query using string formatting; don't do it, ever – use placeholders, or face the consequences
Your VALUES-clause is missing commas between the value-expressions
The sqlite3 module uses "?" as a placeholder instead of percent formatters
"transaction_id NOT NULL AUTO_INCREMENT" is not a valid column name
"AUTO_INCREMENT" is not valid SQLite syntax and you probably meant for transaction_id to be INTEGER PRIMARY KEY – also AUTOINCREMENT should usually not be used
The below functions fix some of the errors, barring the DDL-corrections to the transactions table.
def save_house_to_db(id, address, postcode, bedrooms):
conn = sqlite3.connect('houses_in_london.db')
d = conn.cursor()
# Remove the TABLE "keyword"
d.execute('INSERT INTO houses (id, address, postcode, bedrooms) '
'VALUES (?, ?, ?, ?)', (id, address, postcode, bedrooms))
d.commit()
d.close()
def save_transactions_to_db(id, sale_price, date):
conn = sqlite3.connect('houses_in_london.db')
d = conn.cursor()
# This here expects that you've fixed the table definition as well
d.execute('INSERT INTO transactions (house_id, date, sale_price) '
'VALUES (?, ?, ?)', (id, sale_price, date))
d.commit()
d.close()
Related
This question already has answers here:
How to use variables in SQL statement in Python?
(5 answers)
Closed 3 years ago.
I’m setting up a new flask app, and I'm using sqlite3 as DB. It is possible to maintain the setup even if I have to insert 5 values?
def encrypt():
now = datetime.now()
date_time = now.strftime("%d/%m/%Y - %H:%M")
filename = secure_filename(request.form['filename'].replace(" ", "_").replace("(", "").replace(")", ""))
password = request.form['password']
username = session.get('username')
id = request.form['id']
type = infile[-4:]
file = filename[:-4] + '.enc'
infile = os.path.join(app.config['DATA_FOLDER'], filename)
outfile = os.path.join(app.config['DATA_FOLDER'], filename[:-4] + '.enc')
con = sqlite3.connect(app.config['DataBase'])
cur = con.cursor()
cur.executemany('INSERT INTO keys (id, file, type, date_time, attempts) VALUES (?,?,?,?,?)', id, file, type, date_time, "0")
con.commit()
con.close()
return 'ok'
The following error is shown in logs:
File "./myapp.py", line 524, in encrypt
cur.executemany('INSERT INTO keys (id, file, type, date_time, attempts) VALUES (?,?,?,?,?)', id, file, type, date_time, "0")
TypeError: function takes exactly 2 arguments (6 given)
Firstly, you don't need to use executemany as that is used when you want to insert multiple rows into a single table. What you have there is just multiple values that will represent a single row. Use placeholders for the values in the SQL statement, and pass a tuple as the second argument to execute.
cur.execute('INSERT INTO keys (id, file, type, date_time, attempts) VALUES (?, ?, ?, ?, ?)', (id, file, type, date_time, "0"))
Bonus answer (the executemany case)
Now, when you want to insert multiple rows in the same table, you'd use the cursor.executemany method. And that takes 2 arguments, like you've found out in your error above:
a string, which represents the SQL query
a collection of parameters, where each parameter is a list of values representing a row
The sql query is executed against all parameters in the collection.
Working example with both execute and executemany that can be pasted in a Python file and run
import sqlite3
conn = sqlite3.connect(':memory:')
cursor = conn.cursor()
cursor.execute('CREATE TABLE person (first_name text, last_name text, age integer)')
cursor.execute('SELECT * FROM person')
print(cursor.fetchall()) # outputs []
first_name, last_name, age = 'Carl', 'Cox', 47
cursor.execute('INSERT INTO person (first_name, last_name, age) VALUES (?, ?, ?)', (first_name, last_name, age))
cursor.execute('SELECT * FROM person')
print(cursor.fetchall()) # outputs [('Carl', 'Cox', 47)]
many_values = [
('Boris', 'Brejcha', 37),
('Mladen', 'Solomun', 43),
]
cursor.executemany('INSERT INTO person (first_name, last_name, age) VALUES (?, ?, ?)', many_values)
cursor.execute('SELECT * FROM person')
print(cursor.fetchall()) # outputs [('Carl', 'Cox', 47), ('Boris', 'Brejcha', 37), ('Mladen', 'Solomun', 43)]
conn.close()
So you see in the executemany case how the method takes just 2 parameters, but the 2nd parameter is a sequence of sequences.
Dears,
how can I check if pos_cli from database is equal to variable pos_id? for now with code below I get the following error
cur.execute("CREATE TABLE IF NOT EXISTS Magnit_Coor (pos_cli INTEGER PRIMARY KEY, lat INTEGER, long INTEGER);")
cur.execute('SELECT * FROM Magnit_pos')
data = cur.fetchall()
while True:
for coo in data:
full_add = coo[6:11]
pos_id = coo[0]
print (pos_id)
yand_add = ", ".join(full_add)
g = cur.execute('SELECT EXISTS (SELECT * FROM Magnit_Coor WHERE pos_cli = (?))',pos_id)
g = cur.fetchone()[0]
error below
10001
Traceback (most recent call last):
File "geoco.py", line 17, in <module>
g = cur.execute('SELECT EXISTS (SELECT * FROM Magnit_pos WHERE pos_cli = (?))',pos_id)
ValueError: parameters are of unsupported type
The initial code to create Magnit_pos table and pos_cli especially below
cur.execute("DROP TABLE IF EXISTS Magnit_Pos;")
cur.execute(
"CREATE TABLE Magnit_Pos (pos_cli INTEGER PRIMARY KEY, magnit_name TEXT, codesfa TEXT, codewsot TEXT, pos_sap TEXT, source_dc TEXT, zip TEXT, region TEXT, area TEXT, city TEXT, street TEXT, house TEXT, build TEXT);")
with open('magnit.csv') as csvfile:
magnit = csv.reader(csvfile, delimiter=';')
print(magnit)
for row in magnit:
print(row[0])
# to_db = [unicode(row[0], "utf8"), unicode(row[1], "utf8")]
cur.execute("INSERT INTO Magnit_Pos (pos_cli, magnit_name, codesfa, codewsot, pos_sap, source_dc, zip, region, area, city, street, house, build) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?);", row)
From python's sqlite3 documentation (emphasis mine):
Put ? as a placeholder wherever you want to use a value, and then provide a tuple of values as the second argument to the cursor’s execute() method.
So you should be using:
g = cur.execute('SELECT EXISTS (SELECT * FROM Magnit_Coor WHERE pos_cli = (?))',(pos_id,))
I am new to python and working on using the psycopg2 to insert data in postgres database. I am trying to insert items but get the error message
"Psycopg2.ProgrammingError: syntax error at or near "cup"
LINE 1: INSERT INTO store VALUES(7,10.5,coffee cup)
with the ^ next to coffee cup. I am assuming the order is wrong but i thought you could enter it this way as long as you specified the values.
Here is the code.
import psycopg2
def create_table():
conn=psycopg2.connect("dbname='db1' user='postgres' password='postgress123' host='localhost' port='5432'")
cur=conn.cursor()
cur.execute("CREATE TABLE IF NOT EXISTS store (item TEXT, quantity INTEGER, price REAL)")
conn.commit()
conn.close()
def insert(quantity, price, item):
conn=psycopg2.connect("dbname='db1' user='postgres' password='postgress123' host='localhost' port='5432'")
cur=conn.cursor()
cur.execute("INSERT INTO store VALUES(%s,%s,%s)" % (quantity, price, item))
conn.commit()
conn.close()
create_table()
insert(7, 10.5, 'coffee cup')
Remember to always use the second argument of the execute command to pass the variables, as stated here.
Also, use the name of the fields in your syntax:
cur.execute("INSERT INTO store (item, quantity, price) VALUES (%s, %s, %s);", (item, quantity, price))
That should do the trick.
Problem in your case is coffee cup parameter value is considered as string but psycopg2 accept the value in single quote.
Basically as per my understanding when we create SQL query for psycopg2 it ask for single quote for data parameters [if you have given double quote for query start and end]
In your case you have given double quote for Query Start and end so you need to give single quote for the parameters.
My Observation is you provide single quote for each data paramater in psycopg2
import psycopg2
def create_table():
conn=psycopg2.connect("dbname='db1' user='postgres' password='postgress123' host='localhost' port='5432'")
cur=conn.cursor()
cur.execute("CREATE TABLE IF NOT EXISTS store (item TEXT, quantity INTEGER, price REAL)")
conn.commit()
conn.close()
def insert(quantity, price, item):
conn=psycopg2.connect("dbname='db1' user='postgres' password='postgress123' host='localhost' port='5432'")
cur=conn.cursor()
#cur.execute("INSERT INTO store VALUES(%s,%s,%s)" % (quantity, price, item))
cur.execute("INSERT INTO store VALUES('%s','%s','%s')" % (quantity, price, item))
conn.commit()
conn.close()
create_table()
insert(7, 10.5, 'coffee cup')
I also faced the very same problem, and after a while troubleshooting the code, I found that I forgot to add commas(,) in the Insert query.
The code that causes the error:
data['query'] = 'insert into contacts (name, contact_no, alternate_contact_no, email_id, address)' \
'values (%s %s %s %s %s)'
As you can see in above code, I forgot to add commas after every '%s'.
The correct code:
data['query'] = 'insert into contacts (name, contact_no, alternate_contact_no, email_id, address)' \
'values (%s, %s, %s, %s, %s)'
Hope, It helps!
I am trying to insert data to the table that was created earlier using python script. Here is the code I am trying to execute. I want to insert data into table with date as well.
date_today = dt.date.today()
conn = psycopg2.connect(host = serverip, port = port, database = database, user = uid, password = pwd)
cursor = conn.cursor()
cursor.execute("INSERT INTO My_TABLE (Date, Class, Total_students, failed_students, Percent_passed_students) VALUES (date_today, 'Class Name', int1, int2, int3)")
print "Data Inserted successfully"
conn.commit()
conn.close()
Here is the error I see from my job. what am i missing here?
psycopg2.ProgrammingError: column "date_today" does not exist
I created the table using different job with the following query:
cursor.execute("""CREATE TABLE MY_TABL(Date date, Lob varchar(30), Total_Students int, failed_students int, Percent_passed_students int)""")
And the table is created with above five columns.
This line:
cursor.execute("INSERT INTO My_TABLE (Date, Class, Total_students, failed_students, Percent_passed_students) VALUES (date_today, 'Class Name', int1, int2, int3)")
Is the incorrect way to dynamically insert values into a database.
Here's a functional and correct example:
cursor.execute("INSERT INTO table VALUES (%s, %s, %s)", (var1, var2, var3))
And applying it in your case...
cursor.execute("INSERT INTO My_TABLE VALUES (%s, %s, %s, %s, %s)", (date_today, 'Class Name', int1, int2, int3))
I wrote a program in order to dynamically update a database table but I am getting an error. I stuffed the program with whatever I know little about. Here's my code:
import MySQLdb
class data:
def __init__(self):
self.file123 = raw_input("Enter film: ")
self.title_ = raw_input("Enter film: ")
self.year = raw_input("Enter year: ")
self.director = raw_input("Enter director: ")
a=data()
db = MySQLdb.connect(host="localhost", # your host, usually localhost
user="root", # your username
passwd="mysql", # your password
db="sakila") # name of the data base
cursor = db.cursor()
cursor.execute("INSERT INTO films (file123, title_, year, director) VALUES (?, ?, ?, ?)", (a.file123, a.title_, a.year, a.director))
db.commit()
db.close()
This is the error:
File "C:\Python27\maybe1.py", line 20, in <module>
cursor.execute("INSERT INTO films (file123, title_, year, director) VALUES (?, ?, ?, ?)", (a.file123, a.title_, a.year, a.director))
File "C:\Python27\lib\site-packages\MySQLdb\cursors.py", line 184, in execute
query = query % db.literal(args)
TypeError: not all arguments converted during string formatting
How can I fix this issue ?
You should change ? to %s.
Here is question about why mysqldb use %s instead of ?.
I would do it this way:
query = "INSERT INTO films (file123, title_, year, director) VALUES (%s, %s, %s, %s)" % (a.file123, a.title_, a.year, a.director)
cursor.execute(query)
Replace %s with correct data type, else it will try everything as string which might break at table level.