Develop Reference

Python can't insert multiple rows at once into a MySQL table

I am learning to use the MySQL Connector/Python. When I insert multiple rows together into the table using the code below, it shows an error. But it works when I insert only one row at at time.
Here's the python code I have written:
import mysql.connector as sql
mydb = sql.connect(host = "localhost", user = "root", passwd = "your_password", database = "testdb")
mycursor = mydb.cursor()
str = "insert into test (Name, Address) values (%s, %s)"
val = [
('Foster', 'Moonland'),
('Toblerone', 'Marstown'),
('Hershey', 'Neptunestadt'),
('Cadbury', 'Jupiter DC')
]
mycursor.execute(str, val)
mydb.commit()
This the error message:
Traceback (most recent call last):
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\conversion.py", line 183, in to_mysql
return getattr(self, "_{0}_to_mysql".format(type_name))(value)
AttributeError: 'MySQLConverter' object has no attribute '_tuple_to_mysql'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\cursor.py", line 432, in _process_params
res = [to_mysql(i) for i in res]
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\cursor.py", line 432, in <listcomp>
res = [to_mysql(i) for i in res]
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\conversion.py", line 186, in to_mysql
"MySQL type".format(type_name))
TypeError: Python 'tuple' cannot be converted to a MySQL type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "d:\coding\python\mysql_connector\connectSQL.py", line 24, in <module>
mycursor.execute(str, val)
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\cursor.py", line 557, in execute
psub = _ParamSubstitutor(self._process_params(params))
File "C:\Users\shoun\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\cursor.py", line 437, in _process_params
"Failed processing format-parameters; %s" % err)
mysql.connector.errors.ProgrammingError: Failed processing format-parameters; Python 'tuple' cannot be converted to a MySQL type
But if val is a single tuple, the code executes without producing any error.

Delete and recreate table in Apache Ignite

I am having a problem when I want to DROP a table and recreate it in APACHE IGNITE;
I am using a combination of REST API and PyIgnite to perform the operations.
IGNITE says the table do not exists, however it does not let me recreate it saying that it exists
>>> DROP_QUERY_ALERT="DROP TABLE alerts"
>>> client.sql(DROP_QUERY_ALERT)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/pyignite/client.py", line 404, in sql
raise SQLError(result.message)
pyignite.exceptions.SQLError: Table doesn't exist: ALERTS
>>> CREATE_ALERT_QUERY = '''CREATE TABLE storage.alerts (
... id VARCHAR PRIMARY KEY,
... name VARCHAR,
... address_field VARCHAR,
... create_on TIMESTAMP,
... integration VARCHAR,
... alert VARCHAR,
... ) WITH "CACHE_NAME=storage"'''
>>> client.sql(CREATE_ALERT_QUERY)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/pyignite/client.py", line 404, in sql
raise SQLError(result.message)
pyignite.exceptions.SQLError: Table already exists: ALERTS
>>>
If I try to make a query, it also fails:
>>> N_ALERT_QUERY = '''SELECT * FROM alerts'''
>>> result = client.sql(N_ALERT_QUERY, include_field_names=True)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/pyignite/client.py", line 404, in sql
raise SQLError(result.message)
pyignite.exceptions.SQLError: Failed to parse query. Table "ALERTS" not found; SQL statement:
SELECT * FROM alerts [42102-197]
>>>
I am lost since this seemed to work before, but now I am unable to continue.
Is this a bug, a known behavior? Am I missing something?
Thank you.

It may be a known behavior:
Note, however, that the cache we create can not be dropped with DDL
command. … It should be deleted as any other key-value cache.

After some search and trying, I finally found that there was indeed a table by executing the following query:
SHOW_TABLES_QUERY="SELECT * FROM INFORMATION_SCHEMA.TABLES"
It turns out that IGNITE do not drop a table if it has at least a records, as it was in this case (http://apache-ignite-users.70518.x6.nabble.com/Table-not-getting-dropped-td27957.html).
I deleted the records, and dropped the table.
It took some minutes, but then I was able to recreate the table.
Some of the confusion in my case was related to the fact that TABLE_NAME should have been replaced with <cachename>.TABLE_NAME when performing the drop query:
DROP_QUERY_ALERT="DROP TABLE storage.alerts"

Inserting multiple MySQL records using Python. ERROR: "Python 'tuple' cannot be converted to a MySQL type"

Inserting multiple MySQL records using Python
Error: Python 'tuple' cannot be converted to a MySQL type
ERROR CODE:
Traceback (most recent call last):
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\conversion.py", line 181, in to_mysql
return getattr(self, "_{0}_to_mysql".format(type_name))(value)
AttributeError: 'MySQLConverter' object has no attribute '_tuple_to_mysql'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\cursor.py", line 432, in _process_params
res = [to_mysql(i) for i in res]
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\cursor.py", line 432, in <listcomp>
res = [to_mysql(i) for i in res]
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\conversion.py", line 184, in to_mysql
"MySQL type".format(type_name))
TypeError: Python 'tuple' cannot be converted to a MySQL type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "python_mysql_2.py", line 22, in <module>
my_cursor.execute(mike_placeholders,records_list)
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\cursor.py", line 557, in execute
psub = _ParamSubstitutor(self._process_params(params))
File "C:\Users\POM\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\cursor.py", line 437, in _process_params
"Failed processing format-parameters; %s" % err)
mysql.connector.errors.ProgrammingError: Failed processing format-parameters; Python 'tuple' cannot be converted to a MySQL type
Python Code:
#import sql.connector
import mysql.connector
#Create connection, added db we created#
connection = mysql.connector.connect(
host='localhost',
user='root',
password='123',
database='testdb_1'
)
#Create cursor for the connection
my_cursor = connection.cursor()
#Create SQL statement with placeholders and put in variable
mike_placeholders="INSERT INTO users (name,email,age) VALUES (%s, %s, %s) "
#Create list (array) of records
records_list = [('Tim','Tim#tim.com',32), ('Mary','Mary#mary.com',40), ('Sam','Sam#sam.com',50), ('Fred','Fred#fred.com',22) ]
#Execute cursor, requires SQl statement variable, record variable
my_cursor.execute(mike_placeholders,records_list)
#Commit the connection to make the change on the database
connection.commit()

Ahhh, I used the wrong Python term.
I should have used executemany when working with a tuple.
my_cursor.executemany(mike_placeholders,records_list)

You can't pass a list to my_cursor.execute(), you need to iterate over the list:
for values in records_list:
my_cursor.execute(mike_placeholders, values)
Or you could repeat the (%s, %s, %s) multiple times and do it all in a single query by flattening the list of tuples.
mike_placeholders="INSERT INTO users (name,email,age) VALUES " + ", ".join(["(%s, %s, %s)"] * len(records_list))
my_cursor.execute(mike_placeholders, sum(records_list))

use my_cursor.executemany(mike_placeholders,records_list)
If you have multiple elements which are saved in a list or tuple then use,
cursor.executemany(query,list) or cursor.executemany(query,tuple)

You must use a for loop and INSERT item by item
for x in records_list:
my_cursor.execute(mike_placeholders, x)

select a single column from Mysql DB using sqlalchemy

How do I get values from a single column using sqlalchemy?
In MySQL
select id from request r where r.product_id = 1;
In Python
request = meta.tables['request']
request.select(request.c.product_id==1).execute().rowcount
27L
>>> request.select([request.c.id]).where(request.c.product_id==1).execute()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 2616, in select
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 305, in select
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 5196, in __init__
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 1517, in _literal_as_text
sqlalchemy.exc.ArgumentError: SQL expression object or string expected.

I found the answer, I have to use the general select vs the table select.
Leaving this incase more folks find it useful.
conn = engine.connect()
stmt = select([request.c.id]).where(request.c.product_id==1)
conn.execute(stmt).rowcount
27L

Django: DatabaseError: near "񐁓񐁌��...��1": syntax error

The code:
>>> from django.core import serializers
>>> objects = serializers.deserialize('xml', fixturestr.encode('utf8'))
>>> o = next(objects)
>>> o
<DeserializedObject: countries.Country(pk=AF)>
>>> type(o)
<class 'django.core.serializers.base.DeserializedObject'>
>>> dir(o)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'm2m_data', 'object', 'save']
>>> o.save()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/models/base.py", line 524, in save_base
manager.using(using).filter(pk=pk_val).exists())):
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/models/query.py", line 562, in exists
return self.query.has_results(using=self.db)
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/models/sql/query.py", line 441, in has_results
return bool(compiler.execute_sql(SINGLE))
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 818, in execute_sql
cursor.execute(sql, params)
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/backends/util.py", line 40, in execute
return self.cursor.execute(sql, params)
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 337, in execute
return Database.Cursor.execute(self, query, params)
DatabaseError: near "񐁓񐁌������������𐀠����������������󐁏������������������������񐁈񐁒��������������������������������������������򐁌򐁍����1": syntax error
The query and params:
(Pdb) query
u'SELECT (1) AS "a" FROM "country" WHERE "country"."iso" = ? LIMIT 1'
(Pdb) params
(u'AF',)
To be honest I'm stumped - I don't even know where to go with this one. The query shouldn't even be as long as the horror presented. The error message doesn't decode as utf-8, either.
The underlying task is to read in an xml fixture and push it to the database. Unfortunately, the standard loaddata command can't cope with non-ascii characters in utf-8 xml (see my other recent questions if you're interested). For that reason, I'm trying to do what loaddata does, but manually, so that I can pass the deserializer utf-8 encoded bytes.
Running python 2.7.5 with django 1.4 on linux.
I will be grateful for a way to avoid this problem entirely, or any hints on how to solve or even further diagnose it.
Update: This is the result of trying the query manually:
>>> import sqlite3
>>> conn = sqlite3.connect('database.sqlite3.db')
>>> c = conn.cursor()
>>> c.execute(u'SELECT (1) AS "a" FROM "country" WHERE "country"."iso" = ? LIMIT 1', (u'AF',))
Traceback (most recent call last):
File "<console>", line 1, in <module>
OperationalError: near "񐁓񐁌������������𐀠����������������󐁏������������������������񐁈񐁒��������������������������������������������򐁌򐁍����1": syntax error
>>> c.execute(u'SELECT (1) AS "a" FROM "country" WHERE "country"."iso" = AF LIMIT 1')
Traceback (most recent call last):
File "<console>", line 1, in <module>
OperationalError: near "񐁓񐁌������������𐀠����������������󐁏������������������������񐁈񐁒������������������������������������𐀠����󀀠󐁉��������": syntax error
>>> c.execute(u'SELECT * from "country" WHERE "country"."iso" = AF LIMIT 1')
Traceback (most recent call last):
File "<console>", line 1, in <module>
OperationalError: near "񐁓񐁌����������������������������������������򀁗��������������������������������������������񠁁����򐁌򐁍����1����": syntax error
>>> c.execute(u'SELECT * from "country"')
Traceback (most recent call last):
File "<console>", line 1, in <module>
OperationalError: near "񐁓񐁌������������������������������������": syntax error
>>> c.execute('SELECT * from "country"')
<sqlite3.Cursor object at 0x4123f10>
>>>
My terminal is set to use utf-8. It's not clear why passing a unicode object is going so horribly wrong.
Update 2: This is the version info for sqlite:
>>> sqlite3.version_info
(2, 6, 0)
>>> sqlite3.sqlite_version_info
(3, 7, 11)
>>> sqlite3.sqlite_version
'3.7.11'
>>> sqlite3.x
'11'
>>>
update 3: The same error occurs for every table in the database, if attempting to use unicode strings.
update 4: The same error affects fresh databases. Here's the result of running the example code from the python docs, then trying to do a unicode query:
>>> conn.close()
>>> conn = sqlite3.connect('example.db')
>>> c = conn.cursor()
>>>
>>> # Create table
>>> c.execute('''CREATE TABLE stocks
... (date text, trans text, symbol text, qty real, price real)''')
<sqlite3.Cursor object at 0x43e9180>
>>>
>>> # Insert a row of data
>>> c.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)")
<sqlite3.Cursor object at 0x43e9180>
>>>
>>> # Save (commit) the changes
>>> conn.commit()
>>> c.execute(u'SELECT * FROM "stocks"')
Traceback (most recent call last):
File "<console>", line 1, in <module>
OperationalError: near "񐁓񐁌��������񠀠󰁒��������������������": syntax error
>>>
Update 5: The package which provides the sqlite libraries is sqlite-3.7.11-3.fc17.x86_64.

This was almost certainly caused by my having two different versions of python. After I cleared out the existing python, and reinstalled from source, everything works as it should.
For those who want to recompile from source:
Redhat compiles with UCS-4 enabled, as opposed to the default UCS-2. You probably want to configure with something like ./configure --prefix=/usr --enable-shared --enable-unicode=ucs4.
Python has a lot of dependencies. The package manager commands here are probably the easiest way to figure out what you're missing: http://docs.python.org/devguide/setup.html‎.
Remember if you're using a virtualenv that your environment is likely not picking up changes in your global python.