The code:
>>> from django.core import serializers
>>> objects = serializers.deserialize('xml', fixturestr.encode('utf8'))
>>> o = next(objects)
>>> o
<DeserializedObject: countries.Country(pk=AF)>
>>> type(o)
<class 'django.core.serializers.base.DeserializedObject'>
>>> dir(o)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'm2m_data', 'object', 'save']
>>> o.save()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/models/base.py", line 524, in save_base
manager.using(using).filter(pk=pk_val).exists())):
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/models/query.py", line 562, in exists
return self.query.has_results(using=self.db)
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/models/sql/query.py", line 441, in has_results
return bool(compiler.execute_sql(SINGLE))
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 818, in execute_sql
cursor.execute(sql, params)
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/backends/util.py", line 40, in execute
return self.cursor.execute(sql, params)
File "/home/marcintustin/oneclickrep/oneclickcosvirt/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 337, in execute
return Database.Cursor.execute(self, query, params)
DatabaseError: near "������������𐀠����������������������������������������������������������������������������������������1": syntax error
The query and params:
(Pdb) query
u'SELECT (1) AS "a" FROM "country" WHERE "country"."iso" = ? LIMIT 1'
(Pdb) params
(u'AF',)
To be honest I'm stumped - I don't even know where to go with this one. The query shouldn't even be as long as the horror presented. The error message doesn't decode as utf-8, either.
The underlying task is to read in an xml fixture and push it to the database. Unfortunately, the standard loaddata command can't cope with non-ascii characters in utf-8 xml (see my other recent questions if you're interested). For that reason, I'm trying to do what loaddata does, but manually, so that I can pass the deserializer utf-8 encoded bytes.
Running python 2.7.5 with django 1.4 on linux.
I will be grateful for a way to avoid this problem entirely, or any hints on how to solve or even further diagnose it.
Update: This is the result of trying the query manually:
>>> import sqlite3
>>> conn = sqlite3.connect('database.sqlite3.db')
>>> c = conn.cursor()
>>> c.execute(u'SELECT (1) AS "a" FROM "country" WHERE "country"."iso" = ? LIMIT 1', (u'AF',))
Traceback (most recent call last):
File "<console>", line 1, in <module>
OperationalError: near "������������𐀠����������������������������������������������������������������������������������������1": syntax error
>>> c.execute(u'SELECT (1) AS "a" FROM "country" WHERE "country"."iso" = AF LIMIT 1')
Traceback (most recent call last):
File "<console>", line 1, in <module>
OperationalError: near "������������𐀠����������������������������������������������������������������������������𐀠������������": syntax error
>>> c.execute(u'SELECT * from "country" WHERE "country"."iso" = AF LIMIT 1')
Traceback (most recent call last):
File "<console>", line 1, in <module>
OperationalError: near "��������������������������������������������������������������������������������������������1����": syntax error
>>> c.execute(u'SELECT * from "country"')
Traceback (most recent call last):
File "<console>", line 1, in <module>
OperationalError: near "������������������������������������": syntax error
>>> c.execute('SELECT * from "country"')
<sqlite3.Cursor object at 0x4123f10>
>>>
My terminal is set to use utf-8. It's not clear why passing a unicode object is going so horribly wrong.
Update 2: This is the version info for sqlite:
>>> sqlite3.version_info
(2, 6, 0)
>>> sqlite3.sqlite_version_info
(3, 7, 11)
>>> sqlite3.sqlite_version
'3.7.11'
>>> sqlite3.x
'11'
>>>
update 3: The same error occurs for every table in the database, if attempting to use unicode strings.
update 4: The same error affects fresh databases. Here's the result of running the example code from the python docs, then trying to do a unicode query:
>>> conn.close()
>>> conn = sqlite3.connect('example.db')
>>> c = conn.cursor()
>>>
>>> # Create table
>>> c.execute('''CREATE TABLE stocks
... (date text, trans text, symbol text, qty real, price real)''')
<sqlite3.Cursor object at 0x43e9180>
>>>
>>> # Insert a row of data
>>> c.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)")
<sqlite3.Cursor object at 0x43e9180>
>>>
>>> # Save (commit) the changes
>>> conn.commit()
>>> c.execute(u'SELECT * FROM "stocks"')
Traceback (most recent call last):
File "<console>", line 1, in <module>
OperationalError: near "����������������������������": syntax error
>>>
Update 5: The package which provides the sqlite libraries is sqlite-3.7.11-3.fc17.x86_64.
This was almost certainly caused by my having two different versions of python. After I cleared out the existing python, and reinstalled from source, everything works as it should.
For those who want to recompile from source:
Redhat compiles with UCS-4 enabled, as opposed to the default UCS-2. You probably want to configure with something like ./configure --prefix=/usr --enable-shared --enable-unicode=ucs4.
Python has a lot of dependencies. The package manager commands here are probably the easiest way to figure out what you're missing: http://docs.python.org/devguide/setup.html.
Remember if you're using a virtualenv that your environment is likely not picking up changes in your global python.
Related
I am having a problem when I want to DROP a table and recreate it in APACHE IGNITE;
I am using a combination of REST API and PyIgnite to perform the operations.
IGNITE says the table do not exists, however it does not let me recreate it saying that it exists
>>> DROP_QUERY_ALERT="DROP TABLE alerts"
>>> client.sql(DROP_QUERY_ALERT)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/pyignite/client.py", line 404, in sql
raise SQLError(result.message)
pyignite.exceptions.SQLError: Table doesn't exist: ALERTS
>>> CREATE_ALERT_QUERY = '''CREATE TABLE storage.alerts (
... id VARCHAR PRIMARY KEY,
... name VARCHAR,
... address_field VARCHAR,
... create_on TIMESTAMP,
... integration VARCHAR,
... alert VARCHAR,
... ) WITH "CACHE_NAME=storage"'''
>>> client.sql(CREATE_ALERT_QUERY)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/pyignite/client.py", line 404, in sql
raise SQLError(result.message)
pyignite.exceptions.SQLError: Table already exists: ALERTS
>>>
If I try to make a query, it also fails:
>>> N_ALERT_QUERY = '''SELECT * FROM alerts'''
>>> result = client.sql(N_ALERT_QUERY, include_field_names=True)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/pyignite/client.py", line 404, in sql
raise SQLError(result.message)
pyignite.exceptions.SQLError: Failed to parse query. Table "ALERTS" not found; SQL statement:
SELECT * FROM alerts [42102-197]
>>>
I am lost since this seemed to work before, but now I am unable to continue.
Is this a bug, a known behavior? Am I missing something?
Thank you.
It may be a known behavior:
Note, however, that the cache we create can not be dropped with DDL
command. … It should be deleted as any other key-value cache.
After some search and trying, I finally found that there was indeed a table by executing the following query:
SHOW_TABLES_QUERY="SELECT * FROM INFORMATION_SCHEMA.TABLES"
It turns out that IGNITE do not drop a table if it has at least a records, as it was in this case (http://apache-ignite-users.70518.x6.nabble.com/Table-not-getting-dropped-td27957.html).
I deleted the records, and dropped the table.
It took some minutes, but then I was able to recreate the table.
Some of the confusion in my case was related to the fact that TABLE_NAME should have been replaced with <cachename>.TABLE_NAME when performing the drop query:
DROP_QUERY_ALERT="DROP TABLE storage.alerts"
With Python 3.6.2 and MySQL Connector 2.1.6 package on Windows 10, not calling the execute method of a database cursor, or calling it on a non SELECT statement (CREATE, DROP, ALTER, INSERT, DELETE, UPDATE, etc.) yields the following results:
>>> import mysql.connector
>>> session = mysql.connector.connect(user = "root", database = "mysql")
>>> cursor = session.cursor()
>>> cursor.fetchone()
>>> cursor.fetchmany()
[]
>>> cursor.fetchall()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Maggyero\AppData\Local\Programs\Python\Python36-32\lib\site-packages\mysql\connector\cursor.py", line 891, in fetchall
raise errors.InterfaceError("No result set to fetch from.")
mysql.connector.errors.InterfaceError: No result set to fetch from.
>>> cursor.execute("CREATE TABLE test (x INTEGER)")
>>> cursor.fetchone()
>>> cursor.fetchmany()
[]
>>> cursor.fetchall()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Maggyero\AppData\Local\Programs\Python\Python36-32\lib\site-packages\mysql\connector\cursor.py", line 891, in fetchall
raise errors.InterfaceError("No result set to fetch from.")
mysql.connector.errors.InterfaceError: No result set to fetch from.
PEP 249 explicitly states for the fetchone, fetchmany and fetchall methods:
An Error (or subclass) exception is raised if the previous call to .execute*() did not produce any result set or no call was issued yet.
So why don't fetchone and fetchmany raise an exception like fetchall?
I filed a bug report on bugs.mysql.com and the bug has been fixed in MySQL Connector/Python 8.0.23.
Howdie do,
I'm attempting to use Python 3.6 with SQLAlchemy. I am able to connect to the database, but all reflection attempts are failing:
Traceback (most recent call last):
File "/Users/jw1050/Python/projects/label_automation/generate.py", line 14, in <module>
metadata.reflect(engine, only=['parcel', 'order', 'address', 'document'])
File "/Users/jw1050/.virtualenvs/psd_label_automatiion/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 3874, in reflect
bind.engine.table_names(schema, connection=conn))
File "/Users/jw1050/.virtualenvs/psd_label_automatiion/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2128, in table_names
return self.dialect.get_table_names(conn, schema)
File "<string>", line 2, in get_table_names
File "/Users/jw1050/.virtualenvs/psd_label_automatiion/lib/python3.6/site-packages/sqlalchemy/engine/reflection.py", line 42, in cache
return fn(self, con, *args, **kw)
File "/Users/jw1050/.virtualenvs/psd_label_automatiion/lib/python3.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 1756, in get_table_names
self.identifier_preparer.quote_identifier(current_schema))
File "/Users/jw1050/.virtualenvs/psd_label_automatiion/lib/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 2888, in quote_identifier
self._escape_identifier(value) + \
File "/Users/jw1050/.virtualenvs/psd_label_automatiion/lib/python3.6/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 78, in _escape_identifier
value = value.replace(self.escape_quote, self.escape_to_quote)
TypeError: a bytes-like object is required, not 'str'
My connection information is below:
engine = create_engine('mysql+pymysql://:127.0.0.1:3306/(db_name)?charset=utf8&use_unicode=0')
session = scoped_session(sessionmaker(bind=engine))()
metadata = MetaData()
metadata.reflect(engine, only=['parcel', 'order', 'address', 'document'])
Base = automap_base(metadata=metadata)
Base.prepare()
Everything works fine in Python 2, but I do not want to use Python 2 here. Has anybody else run into this issue and able to resolve?
I've figured this out.
This was due to my connection string having use_unicode=0 in it.
According to the SQLAlchemy Docs this setting should never be used in Python3. In Python 2, it gives superior performance but not in Python 3
Hopefully this helps someone
How do I get values from a single column using sqlalchemy?
In MySQL
select id from request r where r.product_id = 1;
In Python
request = meta.tables['request']
request.select(request.c.product_id==1).execute().rowcount
27L
>>> request.select([request.c.id]).where(request.c.product_id==1).execute()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 2616, in select
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 305, in select
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 5196, in __init__
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 1517, in _literal_as_text
sqlalchemy.exc.ArgumentError: SQL expression object or string expected.
I found the answer, I have to use the general select vs the table select.
Leaving this incase more folks find it useful.
conn = engine.connect()
stmt = select([request.c.id]).where(request.c.product_id==1)
conn.execute(stmt).rowcount
27L
bsddb.db.DBInvalidArgError: (22, 'Invalid argument -- /dbs/supermodels.db: unexpected file type or format')
Is this error a result of incompatible BDB versions (1.85 or 3+)? If so, how do I check the versions, trouble-shoot and solve this error?
Yes, this certainly could be due to older versions of the db file, but it would help if you posted the code that generated this exception and the full traceback.
In the absence of this, are you sure that the database file that you're opening is of the correct type? For example, attempting to open a btree file as if it is a hash raises the exception that you are seeing:
>>> import bsddb
>>> bt = bsddb.btopen('bt')
>>> bt.close()
>>> bsddb.hashopen('bt')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/lib/python2.4/bsddb/__init__.py", line 298, in hashopen
d.open(file, db.DB_HASH, flags, mode)
bsddb.db.DBInvalidArgError: (22, 'Invalid argument -- ./bt: unexpected file type or format')
In *nix you can usually determine the type of db by using the file command, e.g.
$ file /etc/aliases.db cert8.db
/etc/aliases.db: Berkeley DB (Hash, version 8, native byte-order)
cert8.db: Berkeley DB 1.85 (Hash, version 2, native byte-order)
Opening a 1.85 version file fails with the same exception:
>>> db = bsddb.hashopen('/etc/aliases.db') # works, but...
>>> db = bsddb.hashopen('cert8.db')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/lib/python2.4/bsddb/__init__.py", line 298, in hashopen
d.open(file, db.DB_HASH, flags, mode)
bsddb.db.DBInvalidArgError: (22, 'Invalid argument -- ./cert8.db: unexpected file type or format')
If you need to migrate the database files, you should look at the db_dump, db_dump185 and db_load utilities that come with the bdb distribuition.