I'm trying to return a hard coded value in my SQL query, but when running the query using pyodbc, random records return '\x0e' instead of the hard coded value (in this case '16'). If I run the query on the server (MS SQL Server 2008), the query returns all the correct results and values.
The beginning of the query looks like this:
My SQL Code:
Select '"16","' + S.ShipNum + '","'
My python code:
cursor.execute("""Select '\"16\",\"' + SS.ShipNum + '\",\"'
Is there another way to guarantee a value is returned from a query?
\016 is the oct representation of \x0e
So I would think that it has more to do with the way in which you are escaping your double quotes. In your python you are actually geting \16 and not "16" as you desire.
You should try a prepared statment maybe.
ps = db.prepare("SELECT 16")
ps()
returns:
[(16,)]
Addtional examples can be seen here:
[http://python.projects.pgfoundry.org/docs/0.8/driver.html#parameterized-statements]
You can see all of the ascii and other character sets here
[http://donsnotes.com/tech/charsets/ascii.html]
It looks like you're trying to create a comma-delimited, quoted, string representation of the row. Don't try to do this in the database query, string formatting isn't one of T-SQL's strengths.
Pass the static value using a parameter, then join the row values. Using sys.databases for the example:
params = ("Some value",)
sql = "SELECT ?, name, user_access_desc FROM sys.databases"
for row in cursor.execute(sql):
print(','.join('"{0}"'.format(column) for column in row))
Related
I am trying to execute mysql query from python. I want the output
query = "UPDATE 'college_general' SET 'fees' = '180000' WHERE ('college_id' = '2')"
Below is the snippet of the code
def update(table, column, value):
return f"UPDATE '{table}' SET '{column}' = '{value}' WHERE ('college_id' = '{id}')"
query = update("college_general", "fees", fee)
cursor.execute(query)
Instead Python is storing it like
query = 'UPDATE \'college_general\' SET \'fees\' = \'180000\' WHERE (\'college_id\' = \'2\')'
which is causing the script to fail. How can I achieve the desired output?
Thanks in advance!
You can replace the identifiers single quotes with backticks. For more detailed answers visit this question.
There are two types of quotes in MySQL:
' for enclosing string literals
` for enclosing identifiers such as table and column names
There are multiple issues here:
First, I suspect that the string handling bit of your program is actually working, but you are being confused by the external representation of strings. For example, if you do
x = "O'Reilly"
Python will, in some circumstances, display the string as
'O\'Reilly'
Second, I think you are using the wrong kind of quotes. Single quotes in SQL are for strings; MySQL uses backticks for names when necessary, while other SQL implementations usually use double quotes for this.
Third, AND THIS IS IMPORTANT! Do not use string manipulation for building SQL queries. The database library almost certainly has a feature for parametrized queries and you should be using that. Your query should look something like this:
query = 'UPDATE college_general SET fees = ? WHERE college_ID = ?'
cursor.execute(query, [180000, '2'])
but the details will depend on the DB library you are using. For example, some use %s instead of ?. This saves you from all kinds of headaches with quoting strings.
raw string is the simplest solution to your problem.
I believe the code below will achieve what you wanted.
def update(table, column, value):
return fr"UPDATE '{table}' SET '{column}' = '{value}' WHERE ('college_id' = '{id}')"
query = update("college_general", "fees", fee)
cursor.execute(query)
I am trying to insert data into Mysql in python using the below statements
cursor.execute('INSERT INTO batch_details (batch_id,plant_id,product_code) VALUES ("%s","%s","%s")', (id, plantcode, pcode))
Here id ="75", plantcode="2", pcode="FG"
But my values in database are inserted with single quote in mysql database i.e '75','2','FG'
I do not why the values are inserted with single quotes. The syntax of sql query seems to be right.
Are you making sure you are turning it into a prepared statement first? See the documentation example here -
cursor = cnx.cursor(prepared=True)
stmt = "SELECT fullname FROM employees WHERE id = %s" # (1)
cursor.execute(stmt, (5,))
Otherwise, if you want to just do straight string manipulation, you should do
query = 'INSERT INTO batch_details (batch_id,plant_id,product_code) VALUES ("%s","%s","%s")'%(id, plantcode, pcode)
cursor.execute(query)
%s means the value you're providing is to be interpreted as a string. For the ones you want to be integers, try using %d instead. You might also need to get rid of the quote characters around the %d parts in your VALUES list if you want SQL to interpret the value as a number and not a string.
Also, when different programs or libraries print a string, some will print it with single-quote characters, some will print it with double-quote characters. Either way it's still a string in your database, it's just printed to the screen in a stylistically different way.
My objective is to store a JSON object into a MySQL database field of type json, using the mysql.connector library.
import mysql.connector
import json
jsonData = json.dumps(origin_of_jsonData)
cnx = mysql.connector.connect(**config_defined_elsewhere)
cursor = cnx.cursor()
cursor.execute('CREATE DATABASE dataBase')
cnx.database = 'dataBase'
cursor = cnx.cursor()
cursor.execute('CREATE TABLE table (id_field INT NOT NULL, json_data_field JSON NOT NULL, PRIMARY KEY (id_field))')
Now, the code below WORKS just fine, the focus of my question is the use of '%s':
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES (%s, %s)"
values_to_insert = (1, jsonData)
cursor.execute(insert_statement, values_to_insert)
My problem with that: I am very strictly adhering to the use of '...{}'.format(aValue) (or f'...{aValue}') when combining variable aValue(s) into a string, thus avoiding the use of %s (whatever my reasons for that, let's not debate them here - but it is how I would like to keep it wherever possible, hence my question).
In any case, I am simply unable, whichever way I try, to create something that stores the jsonData into the mySql dataBase using something that resembles the above structure and uses '...{}'.format() (in whatever shape or form) instead of %s. For example, I have (among many iterations) tried
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES ({}, {})".format(1, jsonData)
cursor.execute(insert_statement)
but no matter how I turn and twist it, I keep getting the following error:
ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '[some_content_from_jsonData})]' at line 1
Now my question(s):
1) Is there a way to avoid the use of %s here that I am missing?
2) If not, why? What is it that makes this impossible? Is it the cursor.execute() function, or is it the fact that it is a JSON object, or is it something completely different? Shouldn't {}.format() be able to do everything that %s could do, and more?
First of all: NEVER DIRECTLY INSERT YOUR DATA INTO YOUR QUERY STRING!
Using %s in a MySQL query string is not the same as using it in a python string.
In python, you just format the string and 'hello %s!' % 'world' becomes 'hello world!'. In SQL, the %s signals parameter insertion. This sends your query and data to the server separately. You are also not bound to this syntax. The python DB-API specification specifies more styles for this: DB-API parameter styles (PEP 249). This has several advantages over inserting your data directly into the query string:
Prevents SQL injection
Say you have a query to authenticate users by password. You would do that with the following query (of course you would normally salt and hash the password, but that is not the topic of this question):
SELECT 1 FROM users WHERE username='foo' AND password='bar'
The naive way to construct this query would be:
"SELECT 1 FROM users WHERE username='{}' AND password='{}'".format(username, password)
However, what would happen if someone inputs ' OR 1=1 as password. The formatted query would then become
SELECT 1 FROM users WHERE username='foo' AND password='' OR 1=1
which will allways return 1. When using parameter insertion:
execute('SELECT 1 FROM users WHERE username=%s AND password=%s', username, password)
this will never happen, as the query will be interpreted by the server separately.
Performance
If you run the same query many times with different data, the performance difference between using a formatted query and parameter insertion can be significant. With parameter insertion, the server only has to compile the query once (as it is the same every time) and execute it with different data, but with string formatting, it will have to compile it over and over again.
In addition to what was said above, I would like to add some details that I did not immediately understand, and that other (newbies like me ;)) may also find helpful:
1) "parameter insertion" is meant for only for values, it will not work for table names, column names, etc. - for those, the Python string substitution works fine in the sql syntax defintion
2) the cursor.execute function requires a tuple to work (as specified here, albeit not immediately clear, at least to me: https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html)
EXAMPLE for both in one function:
def checkIfRecordExists(column, table, condition_name, condition_value):
...
sqlSyntax = 'SELECT {} FROM {} WHERE {} = %s'.format(column, table, condition_name)
cursor.execute(sqlSyntax, (condition_value,))
Note both the use of .format in the initial sql syntax definition and the use of (condition_value,) in the execute function.
I trying to get an SQL selection with the LIKE parameter, but it gives my only:
false,"error_code":400,"description":"Bad Request: Message text is empty"
Here is my code:
textvar = message.text
c.execute("SELECT * FROM run WHERE act LIKE '+textvar+'")
res = c.fetchone()
bot.send_message(message.chat.id, res)
print textvar gives me an expected sting, taken from message.text. I've tried to get some data without LIKE and that seems everything else works pretty well.
What am I doing wrong?
You are testing if the string '+textvar+' (not the value from the variable textvar!) is present in your act column. There are no such columns.
Don't use string interpolation to insert variables into SQL queries, because that opens you wide to SQL injection attacks. Use query parameters:
textvar = '%{}%'.format(message.text)
c.execute("SELECT * FROM run WHERE act LIKE ?", (textvar,))
Note that I do use string interpolation to add % wildcards to the string to make sure LIKE searches for text anywhere in the column. Without wildcards LIKE is nothing more than an equality test. % matches 0 or more arbitrary characters, _ matches exactly 1 arbitrary character. See the LIKE operator documentation.
So, for your sample text 'sting', the above code produces the string '%sting%', and then the database adapter takes care of proper quoting and executes a SELECT * FROM run WHERE act LIKE '%sting%' SQL query against your database. Any row where the act column contains the substring sting will match and be returned.
I have a function, where I get a string as parameter. I want to save this string to a database. So I have a command like:
sql_command = """INSERT INTO some_table(some_text_row) VALUE (
'{0}');""".format(some_text)
But the parameter can contain characters like '. So I need to replace this sort of characters. I do this with this function:
some_text = given_parameter.replace("'", r"\'")
But now comes the strange behavior: Sometimes, I get a result of \\' and sometimes I get a result of \'. I want to have the second one.
To give you more information: The given_parameter is the HTML code of a webpage. I get the HTML code from the library called requests
Does anyone have some tipps?
Don't construct the query using string formatting - this is unsafe, you are making it vulnerable to SQL injections.
Instead, parameterize the query and let the mysql driver worry about quotes:
sql_command = """
INSERT INTO
some_table(some_text_row)
VALUES
(%s)"""
cursor.execute(sql_command, (some_text, ))