Load csv data into MySQL Python - python

I am trying to load csv data into a mysql table in python:
import csv
import MySQLdb
mydb = MySQLdb.connect(host='localhost', user='xuanyue', passwd='txn-data', db='eversafe_ml_dev')
cursor = mydb.cursor()
csv_data = csv.reader(file('txns.csv'))
for row in csv_data:
cursor.execute("INSERT INTO txn(UserID, AccountID, TransactionDate, Amount, Description, SimpleDescription, TransactionState, TransactionType, CategoryID, AcctCategoryID, CheckNum, City)"
"VALUES(%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)", row)
mydb.commit()
cursor.close()
The output is:
File "load_data.py", line 10, in <module>
"VALUES(%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)", row)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ription, TransactionState, TransactionType, CategoryID, AcctCategoryID, CheckNum' at line 1")
I don't understand in my csv file, are the data fields all strings even if it contains transaction date, amount and etc.?
Or should I use numbers, datetime and etc in VALUES( )?

Related

1241, 'Operand should contain 1 column(s)' when saving

I am trying to save some items to mysql with python and keep getting this error
I have connected to the db and then do a query on table1 and then later am trying to save to Resturants like this
cursor.execute("SELECT * FROM `table1`")
#do some stuff
cursor.execute("INSERT INTO Resturants (School,Resturant_Name,Phone,Street,City, State,Postal_Code,Country,Lat,Lng, Rating,Review_Count,Mobile_Url,Url) VALUES (%s,%s,%s,%s,%s, %s,%s,%s,%s,%s, %s,%s,%s,%s)", (SCHOOL,bus_name,bus_phone,bus_address,bus_city, bus_state,bus_post,bus_country,bus_lat,bus_lng, bus_rating,bus_rating_count,bus_mobile_url,bus_url))
here is error
File "build/bdist.linux-x86_64/egg/MySQLdb/cursors.py", line 174, in execute
File "build/bdist.linux-x86_64/egg/MySQLdb/connections.py", line 36, in defaulterrorhandler
_mysql_exceptions.OperationalError: (1241, 'Operand should contain 1 column(s)')
And then I get the error on the INSERT line why?

python mysqldb how to correctly escape urls

I'm trying to correctly escape urls to enter into a mysql connection, but apparently I'm doing it wrong:
>>> import MySQLdb
>>> db = MySQLdb.connect(host="localhost",user="...",passwd="...",db="...")
>>> cur = db.cursor()
>>> cmd = "insert into S3_data (url) VALUES ('http://google.com')"
>>> cur.execute(cmd)
1
>>> cur.execute(MySQLdb.escape_string(cmd))
Traceback (most recent call last):
File "<pyshell#49>", line 1, in <module>
cur.execute(MySQLdb.escape_string(cmd))
File "C:\Python34\lib\site-packages\MySQLdb\cursors.py", line 220, in execute
self.errorhandler(self, exc, value)
File "C:\Python34\lib\site-packages\MySQLdb\connections.py", line 36, in defaulterrorhandler
raise errorvalue
File "C:\Python34\lib\site-packages\MySQLdb\cursors.py", line 209, in execute
r = self._query(query)
File "C:\Python34\lib\site-packages\MySQLdb\cursors.py", line 371, in _query
rowcount = self._do_query(q)
File "C:\Python34\lib\site-packages\MySQLdb\cursors.py", line 335, in _do_query
db.query(q)
File "C:\Python34\lib\site-packages\MySQLdb\connections.py", line 280, in query
_mysql.connection.query(self, query)
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '\\'http://google.com\\')' at line 1")
As you can see the command works ok, but the escaping fails. What am I missing here?
Also, does escape string handle multi-byte encodings?
Thanks
I believe if you use parameters you shouldn't have a problem.
cmd = "INSERT INTO S3_data (url) VALUES (%s)"
args = 'http://google.com'
cur.execute(cmd, args)

to strip ' or " while using string in python

I have column names in string , now to update table in mysql in the following code :
cursor.execute("""update websites SET %s = %s where weblink = %s""",(key,value,x))
gives error:
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''blog' = 1 where weblink = 'http://blogspot.com/'' at line 1")
key,value = 'blog',2
in cursor.execute key is string and sql table columns are without string , how to solve this problem
Traceback (most recent call last):
File "pgrank.py", line 28, in <module>
cursor.execute("""update websites SET %s = %s where weblink = %s""",(key,value,x))
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (1064, 'You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '\'blog\'' = 1 where weblink = 'http://blogspot.com/' at line 1')
The "inherent" replacing works fine for data, but not for table names.
In SET %s = %s, the first %s gets replaced by 'blog' while it should be blog or even `blog`.
You should do
cursor.execute("""update websites SET `%s` = %%s where weblink = %%s""" % key, (value,x))
because these are two distinct technologies.
Better readability would be provided by
cursor.execute("update websites SET `" + key +
"` = %s where weblink = %s", (value,x))
and safety is increased if you check if key contains the ` character.

Adding url to mysql row in python

I am trying to add a url to a text row in mysql using python and the MySQLdb library, but when I run my code it says there is an error in my sql syntax. Can you tell me what im doing wrong?
Here is my code:
import MySQLdb as mdb
connection = mdb.connect("Localhost", "root", "", "db")
cursor = connection.cursor()
url = mdb.escape_string("http://www.google.com")
cursor.execute("""INSERT INTO index(url) VALUES(%s)""", (url,))
Here is the error:
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 551, in __bootstrap_inner
self.run()
File "E:\prospector\webworker.py", line 77, in run
cursor.execute("INSERT INTO index(url) VALUES('%s')", (url_t,))
File "C:\Python27\lib\site-packages\MySQLdb\cursors.py", line 202, in execute
self.errorhandler(self, exc, value)
File "C:\Python27\lib\site-packages\MySQLdb\connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'index(url) VALUES('http://www.google.com/')' at line 1")
I was able to replicate your problem like this:
mysql> create table `index` (url varchar(50));
Query OK, 0 rows affected (0.05 sec)
mysql> insert into index(url) values ('http://www.google.com');
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'index(url) values ('http://www.google.com')' at line 1
mysql> insert into `index`(url) values ('http://www.google.com');
Query OK, 1 row affected (0.00 sec)
index is a keyword in MySQL. Your life will be easier if you do not use it as a table name.
However, if you really want to, you can use it, but then you have to quote it:
cursor.execute("""INSERT INTO `index`(url) VALUES(%s)""", (url,))
PS: No need to call
url = mdb.escape_string("http://www.google.com")
MySQLdb will do that automatically for you when you call
cursor.execute("""INSERT INTO index(url) VALUES(%s)""", (url,))
In fact, since cursor.execute calls mdb.escape_string for you, doing it yourself could cause undesired values to be inserted into the database depending on the value of url:
In [105]: MySQLdb.escape_string("That's all folks")
Out[105]: "That\\'s all folks"
In [106]: MySQLdb.escape_string(MySQLdb.escape_string("That's all folks"))
Out[106]: "That\\\\\\'s all folks"

Python, MySQL and a weird error

I have a bug that I don't know how to fix or even reproduce:
query = "SELECT id, name FROM names ORDER BY id"
results = database.execute(query)
where the class Database contains:
def execute(self, query):
cursor = self.db.cursor()
try:
cursor.execute(query)
return cursor.fetchall()
except:
import traceback
traceback.print_exc(file=debugFile)
return []
This is how I open the database connection:
self.db = MySQLdb.connect(
host=mysqlHost,
user=mysqlUser,
passwd=mysqlPasswd,
db=mysqlDB
)
This is the stacktrace of the error:
File "foo.py", line 169, in application results = config.db.execute(query)
File "Database.py", line 52, in execute
return cursor.fetchall()
File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py", line 340, in fetchall
self._check_executed()
File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py", line 70, in _check_executed
self.errorhandler(self, ProgrammingError, "execute() first")
File "/usr/lib/pymodules/python2.6/MySQLdb/connections.py", line 35, in defaulterrorhandler
raise errorclass, errorvalue
ProgrammingError: execute() first
Do you have any ideas of why this is happening and how can I fix it? I searched on the internet and I found out that the reason may be having 2 cursors, but I have only one.
try this in your traceback it's for debugging:
except ProgrammingError as ex:
if cursor:
print "\n".join(cursor.messages)
# You can show only the last error like this.
# print cursor.messages[-1]
else:
print "\n".join(self.db.messages)
# Same here you can also do.
# print self.db.messages[-1]

Categories

Resources