Develop Reference

Python cassandra update statement error 'mismatched input 're' expecting K_WHERE '?

I am trying to update cassandra database using python client as follows.
def update_content(session, id, content)
update_statement = """
UPDATE mytable SET content='{}' WHERE id={}
"""
session.execute(update_statement.format(content, id))
It works for most of the cases but in some scenarios the content is a string of the form
content = "Content Message -'[re]...)"
which results in error Exception calling application: <Error from server: code=2000 [Syntax error in CQL query] message="line 2:61 mismatched input 're' expecting K_WHERE (
which I am not sure why is it happening?
Is cassandra trying to interpret the string as regex somehow.
I tried printing the data before updation and its seems fine
"UPDATE mytable SET content='Content Message -'[re]...)' WHERE id=2"

To avoid such problems you should stop using the .format to create CQL statements, and start to use prepared statements that allow you to:
avoid problems with not escaped special characters, like, '
allows to do basic type checking
get better performance, because query will be parsed once, and only the data will be sent over the wire
you'll get token-aware query routing, meaning that query will be sent directly to one of the replicas that holds data for partition
Your code need to be modified as following:
prep_statement = session.prepare('UPDATE mytable SET content=? WHERE id=?')
def update_content(session, id, content):
session.execute(prep_statement, [content, id])
Please notice that statement need to be prepared only once, because it includes the round-trip to the cluster nodes to perform parsing of the query

Google bigquery python client library SQL select regex error

I'm trying to query a google bigquery table using the regex from this blog post. Here it is, slightly modified:
pd\.([^”,\.\(\,\`) \’:\[\]\/\\={}]*)
regex101 example of its usage
It does not, however, work in my google bigquery python client SQL query:
query_results = client.run_sync_query(
"""
SELECT
REGEXP_EXTRACT(SPLIT(content, '\n'),
r'pd\.([^”,\.\(\,\`) \’:\[\]\/\\={}]*)')
FROM
[fh-bigquery:github_extracts.contents_py]
LIMIT 10
""")
query_results.run()
data = query_results.fetch_data()
data
BadRequest: BadRequest: 400 Failed to parse regular expression "pd.([^”,.(\,`) \’:[]/\={}]*)": invalid escape sequence: \’

The problem here is that BigQuery uses re2 library for its regex operations.
If you try the same regex but using the golang flavor you will see the exact same error (golang also uses re2).
So maybe if you just remove the escaping of the ' character you'll already have it working for you (as I tested here it seemed to work properly).
Another issue that you might find is that the result of the SPLIT operation is an ARRAY. That means that BigQuery won't process your query saying that the signature of REGEXP_EXTRACT does not allow ARRAY<STRING> as input. You could use REGEXP_REPLACE instead:
"""
SELECT
REGEXP_EXTRACT(REGEXP_REPLACE(content, r'.*(\\n)', ''),
r'pd\.([^”,\.\(\,\`) ’:\[\]\/\\={}]*)')
FROM
[fh-bigquery:github_extracts.contents_py]
LIMIT 10
"""
The character "\n" is replaced by "" in this operation and the result is a STRING.

Django: Using named parameters on a raw SQL query

I'm trying to execute a raw query that is built dynamically.
To assure that the parameters are inserted in the valid position I'm using named parameters.
This seems to work for Sqlite without any problems. (all my tests succeed)
But when I'm running the same code against MariaDB it fails...
A simple example query:
SELECT u.*
FROM users_gigyauser AS u
WHERE u.email like :u_email
GROUP BY u.id
ORDER BY u.last_login DESC
LIMIT 60 OFFSET 0
Parameters are:
{'u_email': '%test%'}
The error I get is a default syntax error as the parameter is not replaced.
I tried using '%' as an indicator, but this resulted in SQL trying to parse
%u[_email]
and that returned a type error.
I'm executing the query like this:
raw_queryset = GigyaUser.objects.raw(
self.sql_fetch, self._query_object['params']
)
Or when counting:
cursor.execute(self.sql_count, self._query_object['params'])
Both give the same error on MariaDB but work on Sqlite (using the ':' indicator)
Now, what am I missing?

edit:
The format needs to have s suffix as following:
%(u_email)s

If you are using SQLite3, for some reason syntax %(name)s will not work.
You have to use :name syntax instead if you want to pass your params as {"name":"value"} dictionary.
It's contrary to the documentation, that states the first syntax should work with all DB engines.
Heres the source of the issue:
https://code.djangoproject.com/ticket/10070#comment:18

Unable to enter data into SQL DB

I am trying to insert values to a SQL DB where I pull data from a dictionary. I ran into a problem when my program tries to enter 0xqb_QWQDrabGr7FTBREfhCLMZLw4ztx into a column named VersionId. The following is my sample code and error.
cursor.execute("""insert into [TestDB].[dbo].[S3_Files] ([Key],[IsLatest],[LastModified],[Size(Bytes)],[VersionID]) values (%s,%s,%s,%s,%s)""",(item['Key'],item['IsLatest'],item['LastModified'],item['Size'],item['VersionId']))
conn_db.commit()
pymssql.ProgrammingError: (102, "Incorrect syntax near 'qb_QWQDrabGr7FTBREfhCLMZLw4ztx'.DB-Lib error message 20018, severity 15:\nGeneral SQL Server error: Check messages from the SQL Server\n")
Based on the error I assume SQL does not like the 0x in the beginning of the VersionId string because of security issues. If my assumption is correct, what are my options? I also cannot change the value of the VersionId.
Edit: This what I get when I print that cursor command
insert into [TestDB].[dbo].[S3_Files] ([Key],[IsLatest],[LastModified],[Size(Bytes)],[VersionID]) values (Docs/F1/Trades/Buy/Person1/Seller_Provided_-_Raw_Data/GTF/PDF/GTF's_v2/NID3154229_23351201.pdf,True,2015-07-22 22:05:38+00:00,753854,0xqb_QWQDrabGr7FTBREfhCLMZLw4ztx)
Edit 2: The odd thing is that when I try to enter the insert command manually on SQL management studio, it doesn't like the (') in the path name in the first parameter, so I escaped the character, added (') to each values except the number and the command worked. At this point I am pretty stumped on why the insert is not working.
Edit 3: I decided to do a try except on every insert and I see that the ones that VersionIds that get caught have the pattern 0x..... Again, does anyone know if my assumption of security correct?

I guess that's what happens when our libraries try to be smarter than us...
No SQL server around to test, but I assume the reason the 0x values are failing is because the way pymssql passes the parameter causes the server to interprete this as a hexadecimal string and the 'q' following the '0x' does not fit its expectations of 0-9 and A-F chars.
I don't have enough information to know if this is a library bug and/or if it can be worked around; the pymssql documentation is not very extensive, but I would try the following:
if you can, check in MSSQL Profiler what command is actually coming in
build your own command as a string and see if the error persists (remember Bobby Tables before putting that in production, though: https://xkcd.com/327/)
try to work around it by adding quotes etc
swith to another library / use SQLAlchemy

getting error during describe a table in vectorwise using python ingresdbi module

I am using python ingress module for connectivity with vectorwise database.
For describe a table I am using the code below:
import ingresdbi
local_db = ingresdbi.connect(database ='x',uid ='y',driver ='z',pwd ='p')
local_db_cursor = local_db.cursor()
local_db_cursor.execute('help tran_applog ; ' )
I am getting this error :
Syntax error. Last symbol read was: 'help'."
Solutions will be appreciated. Thanks

The problem you've got is that 'help' isn't a real SQL statement that's understood by the DBMS server. It's really a terminal monitor command that gets converted into some queries against the system catalogs under the covers.
The alternative depends a little on what you're trying to get from the "describe table". The system catalogs relating to table and column information are iitables and iicolumns and you can do a select against them. Check the documentation or experiment.
Alternatively there appears to be a row descriptor you can get from ingresdbi, see the example here http://community.actian.com/wiki/Python_Row_Description
HTH

I believe you should do it like in any other shell script: echo "help tran_applog;" | sql mydatabase
Reason: "HELP" is not a standard SQL statement.
As suggested by PaulM, your best option to get metadata about tables is to query the system catalogs (iitables, iicolumns, iirelation, etc).
Start with something like:
SELECT C.column_name, C.column_datatype
FROM iitables T, iicolumns C
WHERE T.table_name = C.table_name
AND T.table_name = 'tran_applog';\g