Keep getting error in trying to INSERT into MySQL - python

I keep getting the same error everytime I try to INSERT data into a MySQL table in my Python script:
tools.cerabot#tools-login:~/wikitool-tasks2$ python didyouknow.py
didyouknow.py:62: Warning: Table 'did_you_know' already exists
self.cursor.execute(self.create_query)
Traceback (most recent call last):
File "didyouknow.py", line 121, in <module>
test._parse_page()
File "didyouknow.py", line 109, in _parse_page
self.cursor.execute(record_exists.format(item["name"]))
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ':Did you know nominations/Cirrus (song)' at line 1")
The code for this is openly viewable on GitHub, though you'd be particularly interested in lines 95 through 116. I've tried escaping and unicoding the string, modifying my query, nothing. (Admittedly, I'm a basic MySQL programmer.) Could anyone experienced in the area help me figure this out please?

The problem is that the "funny characters" in your Wikipedia titles are messing with the SQL syntax. You need to handle that. This can be done by escaping, but the best practice is to use SQL parameterization, like so:
record_exists = u"SELECT COUNT(*) FROM did_you_know WHERE " \
"name = %s"
# and later on
self.cursor.execute(record_exists, item["name"])
You're actually already doing this later on (line 112).

Related

Which python library required for cleardb mysql?

I know that I need to "import MySQLdb" to connect to a MySQL database. But what is the name of the library that we need to import when we are using "cleardb mysql"?
I am hard coding as below to connect, but I guess due to wrong library, I am getting errors. Below are my points to explain my situation::
1) I have installed "MySQldb", and imported it through import keyword.
2) when I use port number in the connectivity syntax, I got "TypeError: an integer is required".
db = MySQLdb.connect("server_IP",3306,"uid","pwd","db_name")
so I removed the port number
import MySQLdb
db = MySQLdb.connect("server_IP","uid","pwd","db_name")
cur = db.cursor()
and the error vanishes. Is that the right method?
3) Everything goes fine until I execute the function "curson.execution("SELECT VERSION()")" to execute sql queries.
curson.execution("SELECT VERSION()")
I am getting error as below
Traceback (most recent call last):
File "<pyshell#10>", line 1, in <module>
cursor.execute("use d_7fc249f763d6fc2")
File "path_to\cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "path_to\connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (2006, 'MySQL server has gone away')
So, is this happening due to library that I have imported? or, if the library is correct, then what seems to be the issue?
The port number is the fifth positional argument, not the second.
db = MySQLdb.connect("server_IP", "uid", "pwd", "db_name", 3306)

Error in creating SQL table using python script

I am trying to create a sql table using python script.
Here is the code :
import MySQLdb
db1 = MySQLdb.connect(host="localhost",user="root",passwd="")
cursor = db1.cursor()
sql = 'use test'
cursor.execute(sql)
query='CREATE TABLE IF NOT EXISTS 3112017(service_area_code VARCHAR(100),phone_numbers VARCHAR(100),preferences VARCHAR(100),opstype VARCHAR(100),phone_type VARCHAR(100))'
cursor.execute(query)
db1.commit()
Following is the error I am getting:
cursor.execute(query)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 226, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorvalue
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '3112017(service_area_code VARCHAR(100),phone_numbers VARCHAR(100),preferences VA' at line 1")
As mentioned in the traceback, you have an error in your SQL syntax. Table name is wrong.
From the SQL manual -
Identifiers may begin with a digit but unless quoted may not consist
solely of digits.

"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1"

I'm trying to execute an insert query. It works when I directly copy and paste it to the mysql command prompt, but fails when I execute it from Python. I'm getting this error with MySQLdb (also tried using _mysql directly and get the same error).
The error is the same as this question, but the answer does not work for my problem (my query is on a single line): MySQL the right syntax to use near '' at line 1 error
query = """INSERT INTO %s(%s) VALUES (%f) ON DUPLICATE KEY UPDATE %s=%f"""%(table_name,measurement_type,value,measurement_type,value)
print query
cur.execute(query)
Result (it prints the query, which when copied directly into MySQL command prompt executes fine, and then crashes):
INSERT INTO D02CA10B13E5$accelerometer_X(periodic) VALUES (79.000000) ON DUPLICATE KEY UPDATE periodic=79.000000
Traceback (most recent call last):
File "collect_data.py", line 145, in <module>
process_data_array(data_bytes[start:start+sensor_packet_size])
File "collect_data.py", line 105, in process_data_array
record_data(MAC,sensor_name,"X",code_name,X,cur)
File "collect_data.py", line 58, in record_data
cur.execute(query)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1")
I tried turning on profiling because I can't seem be able to access _last_executed from my cursor (something others have suggested). It seems that my cursor does not have this property (was it removed?).
query = """INSERT INTO %s(%s) VALUES (%f) ON DUPLICATE KEY UPDATE %s=%f"""%(table_name,measurement_type,value,measurement_type,value)
print query
#cur.execute(query)
try:
cur.execute(query)
except:
cur.execute('show profiles')
for row in cur:
print row
cur.execute("set profiling = 0")
exit()
This shows me an incomplete query:
INSERT INTO D02CA10B13E5$accelerometer_X(periodic) VALUES (80.000000) ON DUPLICATE KEY UPDATE periodic=80.000000
(1L, 5.925e-05, 'INSERT INTO D02CA10B13E5')
Some suggest splitting the query into multiple lines (which contradicts the suggestion in the link I posted above). Anyway, it narrows the search to one line (apparently the Python module doesn't like either my table name or column name which are on the third line)
query = "INSERT \nINTO \n%s(%s) \nVALUES \n(%f) \nON \nDUPLICATE \nKEY \nUPDATE %s=%f"%(table_name,measurement_type,value,measurement_type,value)
print query
cur.execute(query)
Result:
INSERT
INTO
D02CA10B13E5$accelerometer_X(periodic)
VALUES
(80.000000)
ON
DUPLICATE
KEY
UPDATE periodic=80.000000
Traceback (most recent call last):
File "collect_data.py", line 145, in <module>
process_data_array(data_bytes[start:start+sensor_packet_size])
File "collect_data.py", line 105, in process_data_array
record_data(MAC,sensor_name,"X",code_name,X,cur)
File "collect_data.py", line 58, in record_data
cur.execute(query)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 3")
Again, this query executes fine on MySQL, so I'm not sure what's going on here in Python. Any pointers are appreciated. Thanks!
Ok so '' in the MySQL error apparently referred to a \0 character that got appended to my table name. This was not visible as you can see for yourself:
>>> print chr(0)
>>> print "hello" + chr(0)
hello
The \0 character is completely invisible and doesn't get copied onto the clipboard from the terminal (so it worked when pasted onto the MySQL console). Sneaky!
I found this out by comparing string lengths with expected string lengths and finding a difference of 1 character that was invisible to my eyes. The \0 character came about in my Python code when I read the string (sensor ID) from a socket (the application on the other end was a C program that was appending these \0 characters).
Hope my answer saves someone else a lot of trouble!

Python Mysql Cursor.Execute Syntax

I have been looking for the proper way to do my cursor.execute line and have not found one that works for me. Here is my code:
sa = 12345
alert_text = "Help me"
logtype = "Event"
cursor.execute("""INSERT INTO logs(customer_id, log_type, log_entry) VALUES (%s, %s, %s)""", (sa,logtype,alert_text,))
I have tried every method which I have found (except the correct one) and get various errors from "not enough arguments" to mysql syntax errors.
As you can see, sa, logtype and alert_text are variables, not the actual text which must be inserted. I have simply added definition to each variable above as each of these can be something different each time.
Exact Error:
Traceback (most recent call last):
File "listen.py", line 57, in <module>
cursor.execute("""INSERT INTO logs(customer_id, log_type, log_entry) VALUES (%s, %s, %s)""", (sa,logtype,alert_text,))
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '))' at line 1")

MySQLdb execute

I have searched high and low on this site and many others and have found similar questions, but none of the answers have worked for me (usually just accounting for the tuple). I'm writing a python script to parse html pages and populate a database. I have almost everything working except the populating part...
Here is the code segment that deals with the mySQL database (note: using MySQLdb module in python)
conn = MySQLdb.connect(user="root", passwd="xxxxx",db="nutrients")
cur = conn.cursor()
test = "Canned Corn"
cur.execute("INSERT INTO food (name) VALUES (%s)", (test,))
conn.commit()
I was first testing it with parsed string but that wasn't working. This gives me 2 errors:
Traceback (most recent call last):
File "C:\Python32\lib\site-packages\MySQLdb\cursors.py", line 171, in execute
r = self._query(query)
File "C:\Python32\lib\site-packages\MySQLdb\cursors.py", line 330, in _query
rowcount = self._do_query(q)
File "C:\Python32\lib\site-packages\MySQLdb\cursors.py", line 294, in _do_query
db.query(q)
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax
; check the manual that corresponds to your MySQL server version for the right s
yntax to use near '%s)' at line 1")
Traceback (most recent call last):
File "C:\Python32\lib\site-packages\MySQLdb\cursors.py", line 171, in execute
r = self._query(query)
File "C:\Python32\lib\site-packages\MySQLdb\cursors.py", line 330, in _query
rowcount = self._do_query(q)
File "C:\Python32\lib\site-packages\MySQLdb\cursors.py", line 294, in _do_query
db.query(q)
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax
; check the manual that corresponds to your MySQL server version for the right s
yntax to use near '%s)' at line 1")
Why does this query not work!?
Update: After nearly pulling my hair out I decided to revert back to 2.7.3 and guess what... everything works now :D Should have know it was an error of that type... Thanks for the help none the less everyone!
I just ran across this too. After a lot of digging around, this was found to work (note the change to values):
conn = MySQLdb.connect(user="root", passwd="xxxxx",db="nutrients")
cur = conn.cursor()
test = "Canned Corn"
cur.execute("INSERT INTO food (name) VALUES ({0})", (test,))
conn.commit()
Background info: In the port of MySQLdb to Python 3 posted on an unofficial Python packages site, the execute function in cursors.py was modified to use format() rather than the % operator. Hence why %s remains in the SQL statement rather than being substituted. It looks like these changes were never upstreamed to the official source.
I think you don't need any tricky or advanced SQL; you just need to store and retrieve some stuff. So, I think an ORM might make life a lot easier for you.
I suggest you try using the Autumn ORM:
http://pypi.python.org/pypi/autumn/0.5.1
That is a small and simple ORM, easy to understand and work with.
Another good and popular ORM for Python is SQLAlchemy:
http://www.sqlalchemy.org/
You should write your code like this :
conn = MySQLdb.connect(user="root", passwd="xxxxx",db="nutrients")
cur = conn.cursor()
test = "Canned Corn"
cur.execute("INSERT INTO food (name) VALUES (%s)" % (test,) )
conn.commit()
If you are new to python and have some programming experience , then follow Dive Into Python book. Its free.

Categories

Resources