Delete and recreate table in Apache Ignite - python

I am having a problem when I want to DROP a table and recreate it in APACHE IGNITE;
I am using a combination of REST API and PyIgnite to perform the operations.
IGNITE says the table do not exists, however it does not let me recreate it saying that it exists
>>> DROP_QUERY_ALERT="DROP TABLE alerts"
>>> client.sql(DROP_QUERY_ALERT)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/pyignite/client.py", line 404, in sql
raise SQLError(result.message)
pyignite.exceptions.SQLError: Table doesn't exist: ALERTS
>>> CREATE_ALERT_QUERY = '''CREATE TABLE storage.alerts (
... id VARCHAR PRIMARY KEY,
... name VARCHAR,
... address_field VARCHAR,
... create_on TIMESTAMP,
... integration VARCHAR,
... alert VARCHAR,
... ) WITH "CACHE_NAME=storage"'''
>>> client.sql(CREATE_ALERT_QUERY)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/pyignite/client.py", line 404, in sql
raise SQLError(result.message)
pyignite.exceptions.SQLError: Table already exists: ALERTS
>>>
If I try to make a query, it also fails:
>>> N_ALERT_QUERY = '''SELECT * FROM alerts'''
>>> result = client.sql(N_ALERT_QUERY, include_field_names=True)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/pyignite/client.py", line 404, in sql
raise SQLError(result.message)
pyignite.exceptions.SQLError: Failed to parse query. Table "ALERTS" not found; SQL statement:
SELECT * FROM alerts [42102-197]
>>>
I am lost since this seemed to work before, but now I am unable to continue.
Is this a bug, a known behavior? Am I missing something?
Thank you.


It may be a known behavior:
Note, however, that the cache we create can not be dropped with DDL
command. … It should be deleted as any other key-value cache.


After some search and trying, I finally found that there was indeed a table by executing the following query:
SHOW_TABLES_QUERY="SELECT * FROM INFORMATION_SCHEMA.TABLES"
It turns out that IGNITE do not drop a table if it has at least a records, as it was in this case (http://apache-ignite-users.70518.x6.nabble.com/Table-not-getting-dropped-td27957.html).
I deleted the records, and dropped the table.
It took some minutes, but then I was able to recreate the table.
Some of the confusion in my case was related to the fact that TABLE_NAME should have been replaced with <cachename>.TABLE_NAME when performing the drop query:
DROP_QUERY_ALERT="DROP TABLE storage.alerts"

Related

TypeError not all arguments converted during in Python-mySql

I have a question why I get this error.
I try to insert some data from my python script into mysql.
This is my code:
queryinsertStatus = "INSERT INTO general (status) VALUES (%s, )"
onlinestatus = (y["msg"])
print(onlinestatus)
cursor.execute(queryinsertStatus, onlinestatus)
But as result I get this:
API online
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 204, in execute
query = query % args
TypeError: not all arguments converted during bytes formatting
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "trading.py", line 30, in <module>
cursor.execute(queryinsertStatus, onlinestatus)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 206, in execute
raise ProgrammingError(str(m))
MySQLdb._exceptions.ProgrammingError: not all arguments converted during bytes formatting
What do I wrong? onlinestatus contains a "API online"
mySQL is of type tinytext
Hopefully someone can helpme.

Ironically, you should NOT have a comma in your SQL (VALUES (%s)), but you DO need a comma in your parameter. The execute method expects to get a tuple. (y["msg"]) is just a string. To make it a one-element tuple, use (y["msg"],).
This is just a weird corner of Python.

The SQL statement itself appears to be problematic - that "," doesn't seem to belong in the VALUES section. Maybe try it like this:
queryinsertStatus = "INSERT INTO general (status) VALUES (%s)"

MySQL Connector methods `fetchone` and `fetchmany` are not PEP 249 compliant

With Python 3.6.2 and MySQL Connector 2.1.6 package on Windows 10, not calling the execute method of a database cursor, or calling it on a non SELECT statement (CREATE, DROP, ALTER, INSERT, DELETE, UPDATE, etc.) yields the following results:
>>> import mysql.connector
>>> session = mysql.connector.connect(user = "root", database = "mysql")
>>> cursor = session.cursor()
>>> cursor.fetchone()
>>> cursor.fetchmany()
[]
>>> cursor.fetchall()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Maggyero\AppData\Local\Programs\Python\Python36-32\lib\site-packages\mysql\connector\cursor.py", line 891, in fetchall
raise errors.InterfaceError("No result set to fetch from.")
mysql.connector.errors.InterfaceError: No result set to fetch from.
>>> cursor.execute("CREATE TABLE test (x INTEGER)")
>>> cursor.fetchone()
>>> cursor.fetchmany()
[]
>>> cursor.fetchall()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Maggyero\AppData\Local\Programs\Python\Python36-32\lib\site-packages\mysql\connector\cursor.py", line 891, in fetchall
raise errors.InterfaceError("No result set to fetch from.")
mysql.connector.errors.InterfaceError: No result set to fetch from.
PEP 249 explicitly states for the fetchone, fetchmany and fetchall methods:
An Error (or subclass) exception is raised if the previous call to .execute*() did not produce any result set or no call was issued yet.
So why don't fetchone and fetchmany raise an exception like fetchall?

I filed a bug report on bugs.mysql.com and the bug has been fixed in MySQL Connector/Python 8.0.23.

can't use pony orm on sqlite3 blob fields

Just trying some basic exercises with pony ORM (and python3.5, sqlite3).
I just want to print a select query of some data I have without further processing to start with. Pony orm does not seem to like that at all....
The sqlite db dump
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE sums (t text, path BLOB, name BLOB, sum text, primary key (path,name));
INSERT INTO "sums" VALUES('directory','','','');
INSERT INTO "sums" VALUES('file','','sums-backup-f.db','6859b35f9f026317c5df48932f9f2a91');
INSERT INTO "sums" VALUES('file','','md5-tree.py','c7af81d4aad9d00e88db7af950c264c2');
INSERT INTO "sums" VALUES('file','','test.db','a403e9b46e54d6ece851881a895b1953');
INSERT INTO "sums" VALUES('file','','sirius-alexa.db','22a20434cec550a83c675acd849002fa');
INSERT INTO "sums" VALUES('file','','sums-reseau-y.db','1021614f692b5d7bdeef2a45b6b1af5b');
INSERT INTO "sums" VALUES('file','','.md5-tree.py.swp','1c3c195b679e99ef18b3d46044f6e6c5');
INSERT INTO "sums" VALUES('file','','compare-md5.py','cfb4a5b3c7c4e62346aa5e1affef210a');
INSERT INTO "sums" VALUES('file','','charles.local.db','9c50689e8185e5a79fd9077c14636405');
COMMIT;
Here is the code I try to run on python3.5 interactive shell:
from pony.orm import *
db = Database()
class File(db.Entity) :
_table_ = 'sums'
t = Required(str)
path = Required(bytes)
name = Required(bytes)
sum = Required(str)
PrimaryKey(path,name)
db.bind('sqlite','/some/edited/path/test.db')
db.generate_mapping()
File.select().show()
And it fails like this :
Traceback (most recent call last):
File "/usr/lib/python3.5/site-packages/pony/orm/core.py", line 5149, in _fetch
try: result = cache.query_results[query_key]
KeyError: (('f', 0, ()), (<pony.orm.ormtypes.SetType object at 0x7fd2d2701708>,), False, None, None, None, False, False, False, ())
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 2, in show
File "/usr/lib/python3.5/site-packages/pony/utils/utils.py", line 75, in cut_traceback
raise exc # Set "pony.options.CUT_TRACEBACK = False" to see full traceback
File "/usr/lib/python3.5/site-packages/pony/utils/utils.py", line 60, in cut_traceback
try: return func(*args, **kwargs)
File "/usr/lib/python3.5/site-packages/pony/orm/core.py", line 5256, in show
query._fetch().show(width)
File "/usr/lib/python3.5/site-packages/pony/orm/core.py", line 5155, in _fetch
used_attrs=translator.get_used_attrs())
File "/usr/lib/python3.5/site-packages/pony/orm/core.py", line 3859, in _fetch_objects
real_entity_subclass, pkval, avdict = entity._parse_row_(row, attr_offsets)
File "/usr/lib/python3.5/site-packages/pony/orm/core.py", line 3889, in _parse_row_
avdict[attr] = attr.parse_value(row, offsets)
File "/usr/lib/python3.5/site-packages/pony/orm/core.py", line 1922, in parse_value
val = attr.validate(row[offset], None, attr.entity, from_db=True)
File "/usr/lib/python3.5/site-packages/pony/orm/core.py", line 2218, in validate
val = Attribute.validate(attr, val, obj, entity, from_db)
File "/usr/lib/python3.5/site-packages/pony/orm/core.py", line 1894, in validate
if from_db: return converter.sql2py(val)
File "/usr/lib/python3.5/site-packages/pony/orm/dbapiprovider.py", line 619, in sql2py
if not isinstance(val, buffer): val = buffer(val)
TypeError: string argument without an encoding
Am I using this wrong, or is this a bug ? I don't mind go filing a bug, but it's the first time I'm using this orm, so I thought it might be better to check first ...

SQLite has a (mis)feature, which allows a column to store an arbitrary value disregarding the column type. Instead of rigid data type, each SQLite column has an affinity, while each value has a storage class which can be different within the same column. For example, you can store text value inside an integer column, and vice versa. See Datatypes In SQLite Version 3 for more information.
The reason for the error is that the table contains values of "wrong" type in its BLOB columns. Correct SQLite binary literal looks like x'abcdef'. The INSERT commands that you use insert UTF8 strings instead.
This problem was somewhat fixed in the latest version of Pony which you can take from GitHub. Now if Pony receives a string value from a BLOB column it just keep that value without throwing an exception.
If you populate the table with Pony, it will writes BLOB data as a correct binary values, so it can read them later without any problem.

select a single column from Mysql DB using sqlalchemy

How do I get values from a single column using sqlalchemy?
In MySQL
select id from request r where r.product_id = 1;
In Python
request = meta.tables['request']
request.select(request.c.product_id==1).execute().rowcount
27L
>>> request.select([request.c.id]).where(request.c.product_id==1).execute()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 2616, in select
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 305, in select
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 5196, in __init__
File "build/bdist.freebsd-6.3-RELEASE-i386/egg/sqlalchemy/sql/expression.py", line 1517, in _literal_as_text
sqlalchemy.exc.ArgumentError: SQL expression object or string expected.

I found the answer, I have to use the general select vs the table select.
Leaving this incase more folks find it useful.
conn = engine.connect()
stmt = select([request.c.id]).where(request.c.product_id==1)
conn.execute(stmt).rowcount
27L

Web2py DAL has no attribute?

My web2py application returned me an error today, which is quite odd.
Traceback (most recent call last):
File "/var/www/web2py/gluon/restricted.py", line 212, in restricted
exec ccode in environment
File "/var/www/web2py/applications/1MedCloud/controllers/default.py", line 475, in <module>
File "/var/www/web2py/gluon/globals.py", line 194, in <lambda>
self._caller = lambda f: f()
File "/var/www/web2py/applications/1MedCloud/controllers/default.py", line 63, in patient_register
rows = db(db.patientaccount.email==email).select()
File "/var/www/web2py/gluon/dal.py", line 7837, in __getattr__
return ogetattr(self, key)
AttributeError: 'DAL' object has no attribute 'patientaccount'
I am using Mysql as the database, and the table 'patientaccount' does exist. There is no connection issue as I can create tables but not fetch them from the server.
I have been using the very same code to do the db thing, here is my code
db = DAL('mysql://###:$$$#^^^^^^:3306/account_info', pool_size=0)
rows = db(db.patientaccount.email==email).select()
I did not change any code in my default.py file, but accidentally deleted some files inside "database" folder in my application. But I doubt if that could result the error, since the module is fetching tables on the server rather than using local files.
Please help! Thanks in advance!

The DAL does not inspect the MySQL database to discover its tables and fields. You must define the data models explicitly. So, somewhere in your code, you must do:
db.define_table('patientaccount',
Field('email'),
...)
That will define the db.patientaccount table so the DAL knows it exists and what fields it includes.

Categories

Resources