Develop Reference

How to set result of databricks python connector for when explaining SQL queries

My aim is to validate a given SQL string without actually running it.
I thought I could use the EXPLAIN statement to do so.
So I tried using the databricks-sql-connector for python to explain a query, and so determine whether it's valid or not. Example python code:
import databricks.sql
with databricks.sql.connect(...) as connection:
with connection.cursor() as cursor:
cursor.execute("EXPLAIN SELECT BAD-QUERY AS FOO")
r = cursor.fetchall()
The problem with that implementation is that the driver does not throws an error, but instead retrieves me a string containing the error details.
Why it's a problem? I need to parse the string result to distinguish if the explained query was valid or not.
So I was wondering if there some kind of setting / parameter / configuration or so I can use to change the described above result.
Many thanks in Advance!

Python MySQLdb: Syntax Error when create table

The code is very simple, I just directly run it from console, meanwhile, spider.table_name = 'crawler'
import MySQLdb
import scrapy
print (spider.table_name) # >> 'crawler'
db = MySQLdb.connect(........)
db.set_character_set('utf8')
mysql = db.cursor()
sql = "CREATE TABLE %s like r_template;"
mysql.execute(sql, (spider.table_name, ))
db.commit()
But I got Syntax Error:
ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''crawler' like r_template' at line 1")
It seems that the actual sql sentence that being executed was:
CREATE TABLE 'crawler' like r_template
How is that single quote '' generated? How to prevent it from happening?
Then, I tried in more simple way:
mysql.execute(sql, ('crawler', ))
mysql.execute("CREATE TABLE %s like r_template", ('crawler', ))
errors still happened.

You accidentally opened the door to a mysterious and adventurous world. My suggestion is: don't close that door before having a look.
The problem is that your are trying to pass argument for a placeholder "on the left side", but your interface with MySQL works only on the "right side". Placeholders are used for values, not for field names or tables.
The superficial explanation
Let me start with another example.
It is legal to write:
where field = %s
and if the variable is a string, your PEP 249-compliant interface will correctly interpret it: think of it as "putting quotes arounds it" (though it's NOT what it does, otherwise it would open the door to SQL injections; but that will illustrate the point).
That's on the right side of the equality.
But if you write:
where %s = 5
with a value 'my_field', it will not work, because it is on the left side. This is not part of the interface.
As you said if you applied the same logic, it would "put quotes around it", so you would get:
where 'my_field' = 5
it apparently doesn't make sense, because you get quotes where you didn't expect them (caution: again, that's not what happens, but it illustrates the point). It doesn't work, yet those quotes are what you should get if you followed your own logic. There is a contradiction, so something is apparently wrong.
But wait!
A deeper explanation
it is important to understand that with PEP 249 interfaces, the arguments for placeholders are NOT converted into strings and then put into a string query. They are translated into their equivalents (int, etc.) and treated at a lower level (within a parsing tree or some similar structure, I suppose).
The mechanism has been specified for converting the arguments into values. It was not designed for variable identifiers, such as fields or tables (which is a more advanced use case).
Saying that an "identifier is a variable" is quite an advanced idea... I gets you into the wonderful world of higher-order programming.
Could PEP249 be extended to do that? In theory yes, but this is not an open and shut question.
Solution
In the mean time, you are left with only one option: interpolate your string query before you give it to the SQL engine.
sql = "CREATE TABLE %s like r_template;" % 'crawler'
I can imagine the shudders of horror in people around you (don't open that door!). But to the best of my knowledge, that's what you have to do if you really want to have a variable table name.
At that point, you may want to ask yourself why you want to have a variable table name? Did you do that as a lazy workaround for something else? In that case, I would return to the beaten path and forget about making tables or fields variable.
I see, however, two use cases where variable tables or fields are perfectly legitimate: for database administration or for building a database framework.
If that is your case, just use your string interpolation with care, to avoid unintentional SQL injections. You will have to deal wit different issues with whitespaces, special characters, etc.; a good practice will be to "quote" your field or table names, e.g. in standard MySQL:
`table accentuée`
whereas, with ANSI quoting:
"table accentuée"
(As you see, you were not that far off!)
Also be careful to strip off things that could throw the interpreter off, like semicolons.
Anyway, if you want to do that, you will need to navigate out of sight of the coast, straight toward the sunset. You are on the threshold of the hero's journey to the left side. You will enjoy the adventure, as long as you accept that there will be no lifeguard to come to your rescue.

Use .format instead of %s this will allow you to avoid the single quotes in your query.
For Example:
my_sql_query = "CREATE TABLE {}".format(table_name)
mysql.execute(my_sql_query)
That should work :)

Python Database update error

Usually i use Django orm for making database related query in python but now i am using the python itself
I am trying to update a row of my mysql database
query ='UPDATE callerdetail SET upload="{0}" WHERE agent="{1}" AND custid="{2}"AND screenname="{3}" AND status="1"'.format(get.uploaded,get.agent,get.custid,get.screenname)
But i am getting the error
query ='UPDATE callerdetail SET upload="{0}" WHERE agent="{1}" AND custid="{2}"AND screenname="{3}" AND status="1"'.format(get.uploaded,get.agent,get.custid,get.screenname)
AttributeError: 'C' object has no attribute 'uploaded'
Please help me what is wrong with my query ?

Get is probably mapping to a c object. Try renaming your "get" object to something else.
Here is a list of reserved words. I don't see get in there, but it sound like it could be part of a c library that's being included. If you're including something with from x import *, you could be importing it without knowing.

In short - get probably isn't what you think it is.
However, before you go much further building SQL queries with string formatting, I strongly advise you not to! Search for "SQL injection" and you'll see why. Python DB API compliant libraries utilise "placeholders" which the library can use to insert the variables into a query for you providing any necessary escaping/quoting.
So instead of:
query ='UPDATE callerdetail SET upload="{0}" WHERE agent="{1}" AND custid="{2}"AND screenname="{3}" AND status="1"'.format(get.uploaded,get.agent,get.custid,get.screenname)
An example using SQLite3 (using ? as a placeholder - others use %s or :1 or %(name)s - or any/all of the above - but that'll be detailed in the docs of your library):
query = "update callerdetail set upload=? where agent=? and custid=? and screename=? and status=?"
Then when it comes to execute the query, you provide the values to be substituted as a separate argument:
cursor.execute(query, (get.uploaded, get.agent, get.custid, get.screenname))
If you really wanted, you could have a convenience function, and reduce this to:
from operator import attrgetter
get_fields = attrgetter('uploaded', 'agent', 'custid', 'screenname')
cursor.execute(query, get_fields(get))

Really long query

How do u do long query? Is there way to optimize it?
I would do complicated and long query:
all_accepted_parts = acceptedFragment.objects.filter(fragmentID = fragment.objects.filter(categories = fragmentCategory.objects.filter(id=1)))
but it doesn't work, i get:
Error binding parameter 0 - probably unsupported type.
I will be thankful for any hint how i could optimize it or solve of course too - more thankful :)

If it's not working, you can't optimize it. First make it work.
At first glance, it seems that you have really mixed concepts about fields, relationships and equality/membership. First go thought the docs, and build your query piece by piece on the python shell (likely from the inside out).
Just a shot in the dark:
all_accepted_parts = acceptedFragment.objects.filter(fragment__in = fragment.objects.filter(categories = fragmentCategory.objects.get(id=1)))
or maybe:
all_accepted_parts = acceptedFragment.objects.filter(fragment__in = fragment.objects.filter(categories = 1))

As others have said, we really need the models, and some explanation of what you're actually trying to achieve.
But it looks like you want to do a related table lookup. Rather than getting all the related objects in a separate nested query, you should use Django's related model syntax to do the join within your query.
Something like:
acceptedFragment.objects.filter(fragment__categories__id = 1)

Python and psycopg2 - raw sql select from table with integer criteria

It should be simple, bit I've spent the last hour searching for the answer. This is using psycopg2 on python 2.6.
I need something like this:
special_id = 5
sql = """
select count(*) as ct,
from some_table tbl
where tbl.id = %(the_id)
"""
cursor = connection.cursor()
cursor.execute(sql, {"the_id" : special_id})
I cannot get this to work. Were special_id a string, I could replace %(the_id) with %(the_id)s and things work well. However, I want it to use the integer so that it hits my indexes correctly.
There is a surprising lack of specific information on psycopg2 on the internet. I hope someone has an answer to this seemingly simple question.

Per PEP 249, since in psycopg2 paramstyle is pyformat, you need to use %(the_id)s even for non-strings -- trust it to do the right thing.
BTW, internet searches will work better if you use the correct spelling (no h there), but even if you mis-spelled, I'm surprised you didn't get a "did you mean" hint (I did when I deliberately tried!).