I am trying to import rows of a csv file into a mysql table, I am using Python to do this.
Here's a snippet of my mysql statement from my python script:
sql = """INSERT INTO tbl_celebrants(id, name, DATE_FORMAT(birthday,'%m/%d/%Y'), address) \
VALUES(%s , %s, %s, %s)"""
I am getting an error where it says
ValueError: unsupported format character 'm' (0x6d) at index 60
The Date format in my csv file is mm/dd/yyyy. I have tried using %% in the DATE_FORMAT( '%%m/%%d/%%Y') in my python script as suggested by what I have read somewhere in this site but it did not work for me. I appreciate any help and many thanks in advance.
P.S
Here's how I am executing the statement
for row in reader:
cursor = conn.cursor()
sql = """INSERT INTO tbl_celebrants(id, name, DATE_FORMAT(birthday,'%%m/%%d /%%Y'),address) VALUES(%s,%s,%s,%s)"""
cursor.execute(sql, row)
cursor.execute("commit")
cursor.close()
It's right, you should double all % char inside DATE_FORMAT() to didn't get them interpreted as prepared statement placeholder. But you're using the DATE_FORMAT() function in the wrong place, you should use it inside the VALUES(...) declaration, so to fix all the issue try that:
sql = """INSERT INTO tbl_celebrants(id, name, birthday, address)
VALUES(%s , %s, DATE_FORMAT(%s,'%%m/%%d/%%Y'), %s)"""
Related
I am using python sql cursor to dynamically access my database and I am in a situation where I want to use a variable in place of a table name. So far all of my attempts have resulted in syntax errors, although I (think?) I am doing things as expected? Unless a table name as a variable is different from a value as a variable:
here is what I currently have:
cursor.execute("INSERT INTO %s (word=%s,item_id=%s,word_tag=%s,unstemmed_word=%s, word_position=%s, TF=%s, normalized_term_frequency=%s, sentence=%s,anthology_id=%s) "%(table_name, stemmedWord,fle.split()[0], str(word[1]), uniqeWord, word_pos, TF, normalized_term_frequency, sentence, fle.split()[1].split(".")[0]))
and I have also tried this:
cursor.execute("INSERT INTO %s (word,item_id,word_tag,unstemmed_word, word_position, TF, normalized_term_frequency, sentence,anthology_id) values(%s, %s,%s, %s, %s, %s, %s, %s, %s)",(table_name, stemmedWord,fle.split()[0], str(word[1]), uniqeWord, word_pos, TF, normalized_term_frequency, sentence, fle.split()[1].split(".")[0]))
You cannot dynamically bind object names, only values. You'll have to resort to string manipulation for the table's name. E.g.:
sql = "INSERT INTO {} (word=%s,item_id=%s,word_tag=%s,unstemmed_word=%s, word_position=%s, TF=%s, normalized_term_frequency=%s, sentence=%s,anthology_id=%s)".format(table_name)
cursor.execute(sql % (stemmedWord,fle.split()[0], str(word[1]), uniqeWord, word_pos, TF, normalized_term_frequency, sentence, fle.split()[1].split(".")[0]))
If you are on python >= 3.6 this is probably better:
cursor.execute(f'INSERT INTO {table_name} (word="{stemmedWord}",item_id={fle.split()[0]},word_tag={str(word[1])},unstemmed_word="{oword_posrmuniqeWord}", word_position=word_pos, TF={TF}, normalized_term_frequency={normalized_term_frequency}, sentence="{sentence}",anthology_id={fle.split()[1].split(".")[0])}'
but I think your syntax errors are coming from two things:
you have provided a string to split fle on. (Correction this defaults to space - so is OK!)
you haven't quoted what seem to be obvious strings in you sql fields.
I am using python 2.7 and postgresql 10.0.
For learning purposes I am attempting to get user raw_input and place into an insert execute, but no matter what I do, either it be %s or {} and using .format i am receiving errors.
all values are string except age (int)
specifically
with conn:
c.execute("INSERT INTO people(person_first, person_last, person_email,
person_age) VALUES ({}, {}, {}, {})".format(person_first, person_last,
person_email, person_age))
gives me non-string values (from the inputs)
and %s method gives me an error at the first '%' VALUES(%s, %s, %s, %s)
also have attempted VALUES (?, ?, ?, ?) and also unsuccessful similar to %s
The code, as pasted, looks wrong. You have with conn and c.execute. Assuming c is the cursor, and conn is the connection, the way to use them would look like this: with conn.cursor() as c:. The cursor is a context manager that will properly clean itself up when the with block exits.
Also, don't get in the habit of using .format() on your SQL. That will 1) be a vector for SQL injection vulnerabilities and 2) it will break if the input contains a single quote character.
So, combining those two points, your code should look like this:
with conn.cursor() as c:
c.execute("INSERT INTO people(person_first, person_last, person_email,
person_age) VALUES (%s, %s, %s, %s)", (person_first, person_last,
person_email, person_age,))
Note that the parameters are passed as a tuple directly to execute; the driver will parse the query, translate to appropriate SQL/parameter for the server, manage quoting, etc. If you are still seeing errors, post the traceback.
See also -
http://initd.org/psycopg/docs/usage.html#with-statement
http://initd.org/psycopg/docs/usage.html#the-problem-with-the-query-parameters
Hope this helps.
I'm using pymssql to connect to a MS SQL DB on Azure and insert records from a CSV file. I've verified that the connection is successful, and that the list used for the executemany parameter contains all of the data in the correct format and with the correct number of values. However, when I run the script 0 rows are inserted - but no error is thrown.
I looked around and it seems like most others that have experienced something similar were missing the commit(), but that isn't the issue here.
Here is the code. Any help is greatly appreciated.
with open('file.csv') as csvfile:
data = csv.reader(csvfile)
next(data)
dicts = ({'col1': line[0], 'col1': line[1], 'col3': line[2], 'col4': int(line[3]), 'col5': int(line[4]), 'col6': float(line[5])} for line in data)
to_db = ((i['col1'], i['col2'], i['col3'], i['col4'], i['col5'], i['col6']) for i in dicts)
cursor.executemany(
'INSERT INTO myTable VALUES (%s, %s, %s, %d, %d, %f)',
to_db)
print str(cursor.rowcount) + " rows inserted"
conn.commit()
Edit: If I execute the query using cursor.execute() and include the values explicitly in the query then I can successfully insert rows into the database (see below for example).
cursor.execute("INSERT INTO myTable VALUES ('4/18/2016','test','test',0,0,0.0)")
But if I user the cursor.executemany(operation,parameters) syntax and pass a list of the values as the parameter then it results in an incorrect syntax error.
cursor.executemany("INSERT INTO myTable VALUES(%s,%s,%s,%d,%d,%f)",list_of_values)
I was just reading in the module reference that only %s and %d are supported. So I'm thinking that might be the issue. But how do I pass a float?
Using the float placeholder (%f) was in fact the issue. Only %s and %d are supported, but are purely placeholders and do not have any impact on formatting the way that they typically do in python, so really only %s is needed. The working code is as follows:
cursor.executemany("INSERT INTO myTable VALUES(%s,%s,%s,%s,%s,%s)",list_of_values)
I'm trying to run this script, but I have this error:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(than appear 3 users)' at
line 1
Code:
conn = mysql.connector.connect('localhost','root,'','dbtest')
cursor = conn.cursor()
add_data = ("""INSERT INTO data VALUES %s,%s,%s""" %(user, twet, time))
cursor.execute(add_data)
conn.commit()
cursor.close()
conn.close()
I belive that the error is in the %s,%s,%s, I tried a lot of diferent formats but it's always the same result.
Any ideas?
Thanks.
The correct syntax for your connection to your database should read in the lines of:
conn = mysql.connector.connect('localhost','root','','dbtest')
And maybe change your insert statement to:
add_data = ("INSERT INTO data VALUES (\"%s\",\"%s\",\"%s\")" % (user, twet, time))
You must never ever use string substitution in SQL queries. Use the db-api's parameter substitution. Apart from anything else, it will solve one of your syntax problems, which is that your strings are not quoted; it will do that automatically.
The other syntax issue you have, as Mario pointed out, is that arguments to VALUES need to be in parentheses. So, putting those together:
add_data = """INSERT INTO data VALUES (%s,%s,%s)"""
cursor.execute(add_data, (user, twet, time))
The right syntax should be with parenthesis:
INSERT INTO Customers (CustomerName, ContactName, Address, City, PostalCode, Country) VALUES ('Cardinal','Tom B. Erichsen','Skagen 21','Stavanger','4006','Norway');
I am utilizing PyMySQL with Python 2.7 and try to execute the following statement:
'INSERT INTO %s (%s, %s) VALUES (%s, %s) ON DUPLICATE KEY UPDATE %s = %s'
With the following parameters:
('artikel', 'REC_ID', 'odoo_created', u'48094', '2014-12-23 10:00:00', 'odoo_modified', '2014-12-23 10:00:00')
Always resulting in:
{ProgrammingError}(1064, u"You have an error in your SQL syntax; check
the manual that corresponds to your MySQL server version for the right
syntax to use near ''artikel' ('REC_ID', 'odoo_created')\n
VALUES ('48094', '2014-12-' at line 1")
Which seems to me like PyMySQL is escaping all strings during formatting. How can I prevent that for database identifiers like table and column names? It corrupts the statement, since SELECTs from string literals (SELECT * FROM "table") are not possible in comparison to tables (SELECT * FROM table).
That's the point of DB API's substitution. It escapes values. You really shouldn't need to escape table/column names.
I'd use something like:
execute('''INSERT INTO {} ({}, {}) VALUES (%s, %s) ON DUPLICATE KEY UPDATE {} = %s'''
.format('artikel', 'REC_ID', 'odoo_created', 'odoo_modified'),
(u'48094', '2014-12-23 10:00:00', '2014-12-23 10:00:00')
)
Or, you know, just write the names directly.