Solving 'Unrecognized Token' Error While Using SQLite Insert Command - python

I keep getting an OperationalError: Unrecognized Token. The error hapens when I'm attempting to insert data into my SQLite database using an SQLite Insert command. What do I need to do to correct this error or is there a better way I should go about inserting data into my database? The data is water level data measured in meters above chart datum and is gathered from water level gauge data loggers throughout the Great Lakes region of Canada and the US. The script uses the Pandas library and is hardcoded to merge data from water level gauging stations that are located in close proximity to each other. I'd like to use the insert command so I can deal with overlapping data when adding future data to the database. I won't even begin to pretend I know what I'm talking about with databases and programming so any help would be appreciated in how I can solve this error!
I've tried altering my script in the parameterized query to try and solve the problem without any luck as my research has said this is the likely culprit
# Tecumseh. Merges station in steps due to inability of operation to merge all stations at once. Starts by merging PCWL station to hydromet station followed by remaining PCWL station and 3 minute time series
final11975 = pd.merge(hydrometDF["Station11975"], pcwlDF["station11995"], how='outer', left_index=True,right_index=True)
final11975 = pd.merge(final11975, pcwlDF["station11965"], how='outer', left_index=True,right_index=True)
final11975 = pd.merge(final11975, cts, how='outer', left_index=True,right_index=True)
final11975.to_excel("C:/Users/Andrew/Documents/CHS/SeasonalGaugeAnalysis_v2/SeasonalGaugeAnalysis/Output/11975_Tecumseh.xlsx")
print "-------------------------------"
print "11975 - Tecumseh"
print(final11975.info())
final11975.index = final11975.index.astype(str)
#final11975.to_sql('11975_Tecumseh', conn, if_exists='replace', index=True)
#Insert and Ignore data into database to eliminate overlaps
testvalues = (final11975.index, final11975.iloc[:,0], final11975.iloc[:,1], final11975.iloc[:,2])
c.execute("INSERT OR IGNORE INTO 11975_Tecumseh(index,11975_VegaRadar(m),11995.11965), testvalues")
conn.commit()
I'd like the data to insert into the database using the Insert And Ignore command as data is often overlapping when its downloaded. I'm new to databases but I'm under the impression that the Insert and Ignore command will illiminate overlapping data. The message I receive when running my script is:
</> <Exception has occurred: OperationalError
unrecognized token: "11975_Tecumseh"
File "C:\Users\Documents\CHS\SeasonalGaugeAnalysis_v2\SeasonalGaugeAnalysis\Script\CombineStations.py", line 43, in <module>>
c.execute("INSERT OR IGNORE INTO 11975_Tecumseh(index,11975_VegaRadar(m),11995.11965), testvalues") </>

As per SQL Standards, You can create table or column name such as "11975_Tecumseh" and also Tecumseh_11975, but cannot create table or column name begin with numeric without use of double quotes.
c.execute("INSERT OR IGNORE INTO '11975_Tecumseh'(index,'11975_VegaRadar(m)',11995.11965), testvalues")

The error you are getting is because the table name 11975_Tecumseh is invalid as it stands as it is not suitably enclosed.
If you want to use a keyword as a name, you need to quote it. There
are four ways of quoting keywords in SQLite:
'keyword' A keyword in single quotes is a string literal.
"keyword" A keyword in double-quotes is an identifier. [keyword] A
keyword enclosed in square brackets is an identifier.
This is not
standard SQL. This quoting mechanism is used by MS Access and SQL
Server and is included in SQLite for compatibility. keyword A
keyword enclosed in grave accents (ASCII code 96) is an identifier.
This is not standard SQL. This quoting mechanism is used by MySQL and
is included in SQLite for compatibility. For resilience when
confronted with historical SQL statements, SQLite will sometimes bend
the quoting rules above:
If a keyword in single quotes (ex: 'key' or 'glob') is used in a
context where an identifier is allowed but where a string literal is
not allowed, then the token is understood to be an identifier instead
of a string literal.
If a keyword in double quotes (ex: "key" or "glob") is used in a
context where it cannot be resolved to an identifier but where a
string literal is allowed, then the token is understood to be a string
literal instead of an identifier.
Programmers are cautioned not to use the two exceptions described in
the previous bullets. We emphasize that they exist only so that old
and ill-formed SQL statements will run correctly. Future versions of
SQLite might raise errors instead of accepting the malformed
statements covered by the exceptions above.
SQL As Understood By SQLite - SQLite Keywords
The above is applied to invalid names, which includes names that start with numbers and names that include a non numeric inside parenthesises.
If 11975_Tecumseh is the actual table name then it must be enclosed e.g. [11975_Tecumseh]
Likewise the columns
index
11975_VegaRadar(m)
and 11995.11965
Also have to be suitably enclosed.
Doing so you'd end up with
"INSERT OR IGNORE INTO [11975_Tecumseh]([index],[11975_VegaRadar(m)],[11995.11965]), testvalues"
The the issues is that ,testvalues is syntactically incorrect. after the columns to insert into i.e. ([index],[11975_VegaRadar(m)],[11995.11965]) the keyword VALUES with the three values should be used.
An example of a valid statement is :
"INSERT INTO [11975_Tecumseh] ([index],[11975_VegaRadar(m)],[11995.11965]) VALUES('value1','value2','value3')"
As such
c.execute("INSERT INTO [11975_Tecumseh] ([index],[11975_VegaRadar(m)],[11995.11965]) VALUES('value1','value2','value3')")
would insert a new row (unless a constrain conflict occurred)
However, I suspect that you want to insert values according to variables in which case you could use:
"INSERT INTO [11975_Tecumseh] ([index],[11975_VegaRadar(m)],[11995.11965]) VALUES(?,?,?)"
the question marks being place-holders/bind values
SQL As Understood By SQLite- INSERT
The above would then be invoked using :
c.execute("INSERT INTO [11975_Tecumseh] ([index],[11975_VegaRadar(m)],[11995.11965]) VALUES(?,?,?)",testvalues);
#Working Example :
import sqlite3
drop_sql = "DROP TABLE IF EXISTS [11975_Tecumseh]"
crt_sql = "CREATE TABLE IF NOT EXISTS [11975_Tecumseh] ([index],[11975_VegaRadar(m)],[11995.11965])"
testvalues = ("X","Y","Z")
c = sqlite3.connect("test.db")
c.execute(drop_sql)
c.execute(crt_sql)
insert_sql1 = "INSERT INTO [11975_Tecumseh] " \
"([index],[11975_VegaRadar(m)],[11995.11965]) " \
"VALUES('value1','value2','value3')"
c.execute(insert_sql1)
insert_sql2 = "INSERT OR IGNORE INTO '11975_Tecumseh'" \
"('index','11975_VegaRadar(m)',[11995.11965])" \
" VALUES(?,?,?)"
c.execute(insert_sql2,(testvalues))
cursor = c.cursor()
cursor.execute("SELECT * FROM [11975_Tecumseh]")
for row in cursor:
print(row[0], "\n" + row[1], "\n" + row[2])
c.commit()
cursor.close()
c.close()
#Result
##Row 1
value1
value2
value3
##Row 2
X
Y
Z

Related

error in sql syntax when inserting records from dataframe [duplicate]

I am trying to learn the best way to write queries. I also understand the importance of being consistent. Until now, I have randomly used single quotes, double quotes, and backticks without any real thought.
Example:
$query = 'INSERT INTO table (id, col1, col2) VALUES (NULL, val1, val2)';
Also, in the above example, consider that table, col1, val1, etc. may be variables.
What is the standard for this? What do you do?
I've been reading answers to similar questions on here for about 20 minutes, but it seems like there is no definitive answer to this question.
Backticks are to be used for table and column identifiers, but are only necessary when the identifier is a MySQL reserved keyword, or when the identifier contains whitespace characters or characters beyond a limited set (see below) It is often recommended to avoid using reserved keywords as column or table identifiers when possible, avoiding the quoting issue.
Single quotes should be used for string values like in the VALUES() list. Double quotes are supported by MySQL for string values as well, but single quotes are more widely accepted by other RDBMS, so it is a good habit to use single quotes instead of double.
MySQL also expects DATE and DATETIME literal values to be single-quoted as strings like '2001-01-01 00:00:00'. Consult the Date and Time Literals documentation for more details, in particular alternatives to using the hyphen - as a segment delimiter in date strings.
So using your example, I would double-quote the PHP string and use single quotes on the values 'val1', 'val2'. NULL is a MySQL keyword, and a special (non)-value, and is therefore unquoted.
None of these table or column identifiers are reserved words or make use of characters requiring quoting, but I've quoted them anyway with backticks (more on this later...).
Functions native to the RDBMS (for example, NOW() in MySQL) should not be quoted, although their arguments are subject to the same string or identifier quoting rules already mentioned.
Backtick (`)
table & column ───────┬─────┬──┬──┬──┬────┬──┬────┬──┬────┬──┬───────┐
↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
$query = "INSERT INTO `table` (`id`, `col1`, `col2`, `date`, `updated`)
VALUES (NULL, 'val1', 'val2', '2001-01-01', NOW())";
↑↑↑↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑↑↑↑↑
Unquoted keyword ─────┴┴┴┘ │ │ │ │ │ │ │││││
Single-quoted (') strings ───────────┴────┴──┴────┘ │ │ │││││
Single-quoted (') DATE ───────────────────────────┴──────────┘ │││││
Unquoted function ─────────────────────────────────────────┴┴┴┴┘
Variable interpolation
The quoting patterns for variables do not change, although if you intend to interpolate the variables directly in a string, it must be double-quoted in PHP. Just make sure that you have properly escaped the variables for use in SQL. (It is recommended to use an API supporting prepared statements instead, as protection against SQL injection).
// Same thing with some variable replacements
// Here, a variable table name $table is backtick-quoted, and variables
// in the VALUES list are single-quoted
$query = "INSERT INTO `$table` (`id`, `col1`, `col2`, `date`) VALUES (NULL, '$val1', '$val2', '$date')";
Prepared statements
When working with prepared statements, consult the documentation to determine whether or not the statement's placeholders must be quoted. The most popular APIs available in PHP, PDO and MySQLi, expect unquoted placeholders, as do most prepared statement APIs in other languages:
// PDO example with named parameters, unquoted
$query = "INSERT INTO `table` (`id`, `col1`, `col2`, `date`) VALUES (:id, :col1, :col2, :date)";
// MySQLi example with ? parameters, unquoted
$query = "INSERT INTO `table` (`id`, `col1`, `col2`, `date`) VALUES (?, ?, ?, ?)";
Characters requring backtick quoting in identifiers:
According to MySQL documentation, you do not need to quote (backtick) identifiers using the following character set:
ASCII: [0-9,a-z,A-Z$_] (basic Latin letters, digits 0-9, dollar, underscore)
You can use characters beyond that set as table or column identifiers, including whitespace for example, but then you must quote (backtick) them.
Also, although numbers are valid characters for identifiers, identifiers cannot consist solely of numbers. If they do they must be wrapped in backticks.
There are two types of quotes in MySQL:
' for enclosing string literals
` for enclosing identifiers such as table and column names
And then there is " which is a special case. It could be used for one of above-mentioned purposes at a time depending on MySQL server's sql_mode:
By default the " character can be used to enclose string literals just like '
In ANSI_QUOTES mode the " character can be used to enclose identifiers just like `
The following query will produce different results (or errors) depending on SQL mode:
SELECT "column" FROM table WHERE foo = "bar"
ANSI_QUOTES disabled
The query will select the string literal "column" where column foo is equal to string "bar"
ANSI_QUOTES enabled
The query will select the column column where column foo is equal to column bar
When to use what
I suggest that you avoid using " so that your code becomes independent of SQL modes
Always quote identifiers since it is a good practice (quite a few questions on SO discuss this)
(There are good answers above regarding the SQL nature of your question, but this may also be relevant if you are new to PHP.)
Perhaps it is important to mention that PHP handles single and double quoted strings differently...
Single-quoted strings are 'literals' and are pretty much WYSIWYG strings. Double-quoted strings are interpreted by PHP for possible variable-substitution (backticks in PHP are not exactly strings; they execute a command in the shell and return the result).
Examples:
$foo = "bar";
echo 'there is a $foo'; // There is a $foo
echo "there is a $foo"; // There is a bar
echo `ls -l`; // ... a directory list
Backticks are generally used to indicate an identifier and as well be safe from accidentally using the Reserved Keywords.
For example:
Use `database`;
Here the backticks will help the server to understand that the database is in fact the name of the database, not the database identifier.
Same can be done for the table names and field names. This is a very good habit if you wrap your database identifier with backticks.
Check this answer to understand more about backticks.
Now about Double quotes & Single Quotes (Michael has already mentioned that).
But, to define a value you have to use either single or double quotes. Lets see another example.
INSERT INTO `tablename` (`id, `title`) VALUES ( NULL, title1);
Here I have deliberately forgotten to wrap the title1 with quotes. Now the server will take the title1 as a column name (i.e. an identifier). So, to indicate that it's a value you have to use either double or single quotes.
INSERT INTO `tablename` (`id, `title`) VALUES ( NULL, 'title1');
Now, in combination with PHP, double quotes and single quotes make your query writing time much easier. Let's see a modified version of the query in your question.
$query = "INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, '$val1', '$val2')";
Now, using double quotes in the PHP, you will make the variables $val1, and $val2 to use their values thus creating a perfectly valid query. Like
$val1 = "my value 1";
$val2 = "my value 2";
$query = "INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, '$val1', '$val2')";
will make
INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, 'my value 1', 'my value 2')
In MySQL, these symbols are used to delimit a query ` ," ,' and () .
" or ' are used for enclosing string-like values "26-01-2014 00:00:00" or '26-01-2014 00:00:00' . These symbols are only for strings, not aggregate functions like now, sum, or max.
` is used for enclosing table or column names, e.g. select `column_name` from `table_name` where id='2'
( and ) simply enclose parts of a query e.g. select `column_name` from `table_name` where (id='2' and gender='male') or name='rakesh' .
There has been many helpful answers here, generally culminating into two points.
BACKTICKS(`) are used around identifier names.
SINGLE QUOTES(') are used around values.
AND as #MichaelBerkowski said
Backticks are to be used for table and column identifiers, but are
only necessary when the identifier is a MySQL reserved keyword, or
when the identifier contains whitespace characters or characters
beyond a limited set (see below) It is often recommended to avoid
using reserved keywords as column or table identifiers when possible,
avoiding the quoting issue.
There is a case though where an identifier can neither be a reserved keyword or contain whitespace or characters beyond limited set but necessarily require backticks around them.
EXAMPLE
123E10 is a valid identifier name but also a valid INTEGER literal.
[Without going into detail how you would get such an identifier name], Suppose I want to create a temporary table named 123456e6.
No ERROR on backticks.
DB [XXX]> create temporary table `123456e6` (`id` char (8));
Query OK, 0 rows affected (0.03 sec)
ERROR when not using backticks.
DB [XXX]> create temporary table 123451e6 (`id` char (8));
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '123451e6 (`id` char (8))' at line 1
However, 123451a6 is a perfectly fine identifier name (without back ticks).
DB [XXX]> create temporary table 123451a6 (`id` char (8));
Query OK, 0 rows affected (0.03 sec)
This is completely because 1234156e6 is also an exponential number.
The string literals in MySQL and PHP are the same.
A string is a sequence of bytes or characters, enclosed within either
single quote (“'”) or double quote (“"”) characters.
So if your string contains single quotes, then you could use double quotes to quote the string, or if it contains double quotes, then you could use single quotes to quote the string. But if your string contains both single quotes and double quotes, you need to escape the one that used to quote the string.
Mostly, we use single quotes for an SQL string value, so we need to use double quotes for a PHP string.
$query = "INSERT INTO table (id, col1, col2) VALUES (NULL, 'val1', 'val2')";
And you could use a variable in PHP's double-quoted string:
$query = "INSERT INTO table (id, col1, col2) VALUES (NULL, '$val1', '$val2')";
But if $val1 or $val2 contains single quotes, that will make your SQL be wrong. So you need to escape it before it is used in sql; that is what mysql_real_escape_string is for. (Although a prepared statement is better.)
In combination of PHP and MySQL, double quotes and single quotes make your query-writing time so much easier.
$query = "INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, '$val1', '$val2')";
Now, suppose you are using a direct post variable into the MySQL query then, use it this way:
$query = "INSERT INTO `table` (`id`, `name`, `email`) VALUES (' ".$_POST['id']." ', ' ".$_POST['name']." ', ' ".$_POST['email']." ')";
This is the best practice for using PHP variables into MySQL.
If table cols and values are variables then there are two ways:
With double quotes "" the complete query:
$query = "INSERT INTO $table_name (id, $col1, $col2)
VALUES (NULL, '$val1', '$val2')";
Or
$query = "INSERT INTO ".$table_name." (id, ".$col1.", ".$col2.")
VALUES (NULL, '".$val1."', '".$val2."')";
With single quotes '':
$query = 'INSERT INTO '.$table_name.' (id, '.$col1.', '.$col2.')
VALUES (NULL, '.$val1.', '.$val2.')';
Use back ticks `` when a column/value name is similar to a MySQL reserved keyword.
Note: If you are denoting a column name with a table name then use back ticks like this:
`table_name`. `column_name` <-- Note: exclude . from back ticks.
Single quotes should be used for string values like in the VALUES() list.
Backticks are generally used to indicate an identifier and as well be safe from accidentally using the reserved keywords.
In combination of PHP and MySQL, double quotes and single quotes make your query writing time so much easier.
Besides all of the (well-explained) answers, there hasn't been the following mentioned and I visit this Q&A quite often.
In a nutshell; MySQL thinks you want to do math on its own table/column and interprets hyphens such as "e-mail" as e minus mail.
Disclaimer: So I thought I would add this as an "FYI" type of answer for those who are completely new to working with databases and who may not understand the technical terms described already.
SQL servers and MySQL, PostgreySQL, Oracle don't understand double quotes("). Thus your query should be free from double quotes(") and should only use single quotes(').
Back-trip(`) is optional to use in SQL and is used for table name, db name and column names.
If you are trying to write query in your back-end to call MySQL then you can use double quote(") or single quotes(') to assign query to a variable like:
let query = "select id, name from accounts";
//Or
let query = 'select id, name from accounts';
If ther's a where statement in your query and/or trying to insert a value and/or an update of value which is string use single quote(') for these values like:
let querySelect = "select id, name from accounts where name = 'John'";
let queryUpdate = "update accounts set name = 'John' where id = 8";
let queryInsert = "insert into accounts(name) values('John')";
//Please not that double quotes are only to be used in assigning string to our variable not in the query
//All these below will generate error
let querySelect = 'select id, name from accounts where name = "John"';
let queryUpdate = 'update accounts set name = "John" where id = 8';
let queryInsert = 'insert into accounts(name) values("John")';
//As MySQL or any SQL doesn't understand double quotes("), these all will generate error.
If you want to stay out of this confusion when to use double quotes(") and single quotes('), would recommend to stick with single quotes(') this will include backslash() like:
let query = 'select is, name from accounts where name = \'John\'';
Problem with double(") or single(') quotes arise when we had to assign some value dynamic and perform some string concatenation like:
let query = "select id, name from accounts where name = " + fName + " " + lName;
//This will generate error as it must be like name = 'John Smith' for SQL
//However our statement made it like name = John Smith
//In order to resolve such errors use
let query = "select id, name from accounts where name = '" + fName + " " + lName + "'";
//Or using backslash(\)
let query = 'select id, name from accounts where name = \'' + fName + ' ' + lName + '\'';
If need further clearance do follow quotes in JavaScript
It is sometimes useful to not use quotes... because this can highlight issues in the code generating the query... For example:
Where x and y are should always be integers...
SELECT * FROM table WHERE x= AND y=0
Is a SQL syntax error... a little lazy but can be useful...

Syntax error but followed documentation, mysql Python [duplicate]

I am trying to learn the best way to write queries. I also understand the importance of being consistent. Until now, I have randomly used single quotes, double quotes, and backticks without any real thought.
Example:
$query = 'INSERT INTO table (id, col1, col2) VALUES (NULL, val1, val2)';
Also, in the above example, consider that table, col1, val1, etc. may be variables.
What is the standard for this? What do you do?
I've been reading answers to similar questions on here for about 20 minutes, but it seems like there is no definitive answer to this question.
Backticks are to be used for table and column identifiers, but are only necessary when the identifier is a MySQL reserved keyword, or when the identifier contains whitespace characters or characters beyond a limited set (see below) It is often recommended to avoid using reserved keywords as column or table identifiers when possible, avoiding the quoting issue.
Single quotes should be used for string values like in the VALUES() list. Double quotes are supported by MySQL for string values as well, but single quotes are more widely accepted by other RDBMS, so it is a good habit to use single quotes instead of double.
MySQL also expects DATE and DATETIME literal values to be single-quoted as strings like '2001-01-01 00:00:00'. Consult the Date and Time Literals documentation for more details, in particular alternatives to using the hyphen - as a segment delimiter in date strings.
So using your example, I would double-quote the PHP string and use single quotes on the values 'val1', 'val2'. NULL is a MySQL keyword, and a special (non)-value, and is therefore unquoted.
None of these table or column identifiers are reserved words or make use of characters requiring quoting, but I've quoted them anyway with backticks (more on this later...).
Functions native to the RDBMS (for example, NOW() in MySQL) should not be quoted, although their arguments are subject to the same string or identifier quoting rules already mentioned.
Backtick (`)
table & column ───────┬─────┬──┬──┬──┬────┬──┬────┬──┬────┬──┬───────┐
↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
$query = "INSERT INTO `table` (`id`, `col1`, `col2`, `date`, `updated`)
VALUES (NULL, 'val1', 'val2', '2001-01-01', NOW())";
↑↑↑↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑↑↑↑↑
Unquoted keyword ─────┴┴┴┘ │ │ │ │ │ │ │││││
Single-quoted (') strings ───────────┴────┴──┴────┘ │ │ │││││
Single-quoted (') DATE ───────────────────────────┴──────────┘ │││││
Unquoted function ─────────────────────────────────────────┴┴┴┴┘
Variable interpolation
The quoting patterns for variables do not change, although if you intend to interpolate the variables directly in a string, it must be double-quoted in PHP. Just make sure that you have properly escaped the variables for use in SQL. (It is recommended to use an API supporting prepared statements instead, as protection against SQL injection).
// Same thing with some variable replacements
// Here, a variable table name $table is backtick-quoted, and variables
// in the VALUES list are single-quoted
$query = "INSERT INTO `$table` (`id`, `col1`, `col2`, `date`) VALUES (NULL, '$val1', '$val2', '$date')";
Prepared statements
When working with prepared statements, consult the documentation to determine whether or not the statement's placeholders must be quoted. The most popular APIs available in PHP, PDO and MySQLi, expect unquoted placeholders, as do most prepared statement APIs in other languages:
// PDO example with named parameters, unquoted
$query = "INSERT INTO `table` (`id`, `col1`, `col2`, `date`) VALUES (:id, :col1, :col2, :date)";
// MySQLi example with ? parameters, unquoted
$query = "INSERT INTO `table` (`id`, `col1`, `col2`, `date`) VALUES (?, ?, ?, ?)";
Characters requring backtick quoting in identifiers:
According to MySQL documentation, you do not need to quote (backtick) identifiers using the following character set:
ASCII: [0-9,a-z,A-Z$_] (basic Latin letters, digits 0-9, dollar, underscore)
You can use characters beyond that set as table or column identifiers, including whitespace for example, but then you must quote (backtick) them.
Also, although numbers are valid characters for identifiers, identifiers cannot consist solely of numbers. If they do they must be wrapped in backticks.
There are two types of quotes in MySQL:
' for enclosing string literals
` for enclosing identifiers such as table and column names
And then there is " which is a special case. It could be used for one of above-mentioned purposes at a time depending on MySQL server's sql_mode:
By default the " character can be used to enclose string literals just like '
In ANSI_QUOTES mode the " character can be used to enclose identifiers just like `
The following query will produce different results (or errors) depending on SQL mode:
SELECT "column" FROM table WHERE foo = "bar"
ANSI_QUOTES disabled
The query will select the string literal "column" where column foo is equal to string "bar"
ANSI_QUOTES enabled
The query will select the column column where column foo is equal to column bar
When to use what
I suggest that you avoid using " so that your code becomes independent of SQL modes
Always quote identifiers since it is a good practice (quite a few questions on SO discuss this)
(There are good answers above regarding the SQL nature of your question, but this may also be relevant if you are new to PHP.)
Perhaps it is important to mention that PHP handles single and double quoted strings differently...
Single-quoted strings are 'literals' and are pretty much WYSIWYG strings. Double-quoted strings are interpreted by PHP for possible variable-substitution (backticks in PHP are not exactly strings; they execute a command in the shell and return the result).
Examples:
$foo = "bar";
echo 'there is a $foo'; // There is a $foo
echo "there is a $foo"; // There is a bar
echo `ls -l`; // ... a directory list
Backticks are generally used to indicate an identifier and as well be safe from accidentally using the Reserved Keywords.
For example:
Use `database`;
Here the backticks will help the server to understand that the database is in fact the name of the database, not the database identifier.
Same can be done for the table names and field names. This is a very good habit if you wrap your database identifier with backticks.
Check this answer to understand more about backticks.
Now about Double quotes & Single Quotes (Michael has already mentioned that).
But, to define a value you have to use either single or double quotes. Lets see another example.
INSERT INTO `tablename` (`id, `title`) VALUES ( NULL, title1);
Here I have deliberately forgotten to wrap the title1 with quotes. Now the server will take the title1 as a column name (i.e. an identifier). So, to indicate that it's a value you have to use either double or single quotes.
INSERT INTO `tablename` (`id, `title`) VALUES ( NULL, 'title1');
Now, in combination with PHP, double quotes and single quotes make your query writing time much easier. Let's see a modified version of the query in your question.
$query = "INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, '$val1', '$val2')";
Now, using double quotes in the PHP, you will make the variables $val1, and $val2 to use their values thus creating a perfectly valid query. Like
$val1 = "my value 1";
$val2 = "my value 2";
$query = "INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, '$val1', '$val2')";
will make
INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, 'my value 1', 'my value 2')
In MySQL, these symbols are used to delimit a query ` ," ,' and () .
" or ' are used for enclosing string-like values "26-01-2014 00:00:00" or '26-01-2014 00:00:00' . These symbols are only for strings, not aggregate functions like now, sum, or max.
` is used for enclosing table or column names, e.g. select `column_name` from `table_name` where id='2'
( and ) simply enclose parts of a query e.g. select `column_name` from `table_name` where (id='2' and gender='male') or name='rakesh' .
There has been many helpful answers here, generally culminating into two points.
BACKTICKS(`) are used around identifier names.
SINGLE QUOTES(') are used around values.
AND as #MichaelBerkowski said
Backticks are to be used for table and column identifiers, but are
only necessary when the identifier is a MySQL reserved keyword, or
when the identifier contains whitespace characters or characters
beyond a limited set (see below) It is often recommended to avoid
using reserved keywords as column or table identifiers when possible,
avoiding the quoting issue.
There is a case though where an identifier can neither be a reserved keyword or contain whitespace or characters beyond limited set but necessarily require backticks around them.
EXAMPLE
123E10 is a valid identifier name but also a valid INTEGER literal.
[Without going into detail how you would get such an identifier name], Suppose I want to create a temporary table named 123456e6.
No ERROR on backticks.
DB [XXX]> create temporary table `123456e6` (`id` char (8));
Query OK, 0 rows affected (0.03 sec)
ERROR when not using backticks.
DB [XXX]> create temporary table 123451e6 (`id` char (8));
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '123451e6 (`id` char (8))' at line 1
However, 123451a6 is a perfectly fine identifier name (without back ticks).
DB [XXX]> create temporary table 123451a6 (`id` char (8));
Query OK, 0 rows affected (0.03 sec)
This is completely because 1234156e6 is also an exponential number.
The string literals in MySQL and PHP are the same.
A string is a sequence of bytes or characters, enclosed within either
single quote (“'”) or double quote (“"”) characters.
So if your string contains single quotes, then you could use double quotes to quote the string, or if it contains double quotes, then you could use single quotes to quote the string. But if your string contains both single quotes and double quotes, you need to escape the one that used to quote the string.
Mostly, we use single quotes for an SQL string value, so we need to use double quotes for a PHP string.
$query = "INSERT INTO table (id, col1, col2) VALUES (NULL, 'val1', 'val2')";
And you could use a variable in PHP's double-quoted string:
$query = "INSERT INTO table (id, col1, col2) VALUES (NULL, '$val1', '$val2')";
But if $val1 or $val2 contains single quotes, that will make your SQL be wrong. So you need to escape it before it is used in sql; that is what mysql_real_escape_string is for. (Although a prepared statement is better.)
In combination of PHP and MySQL, double quotes and single quotes make your query-writing time so much easier.
$query = "INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, '$val1', '$val2')";
Now, suppose you are using a direct post variable into the MySQL query then, use it this way:
$query = "INSERT INTO `table` (`id`, `name`, `email`) VALUES (' ".$_POST['id']." ', ' ".$_POST['name']." ', ' ".$_POST['email']." ')";
This is the best practice for using PHP variables into MySQL.
If table cols and values are variables then there are two ways:
With double quotes "" the complete query:
$query = "INSERT INTO $table_name (id, $col1, $col2)
VALUES (NULL, '$val1', '$val2')";
Or
$query = "INSERT INTO ".$table_name." (id, ".$col1.", ".$col2.")
VALUES (NULL, '".$val1."', '".$val2."')";
With single quotes '':
$query = 'INSERT INTO '.$table_name.' (id, '.$col1.', '.$col2.')
VALUES (NULL, '.$val1.', '.$val2.')';
Use back ticks `` when a column/value name is similar to a MySQL reserved keyword.
Note: If you are denoting a column name with a table name then use back ticks like this:
`table_name`. `column_name` <-- Note: exclude . from back ticks.
Single quotes should be used for string values like in the VALUES() list.
Backticks are generally used to indicate an identifier and as well be safe from accidentally using the reserved keywords.
In combination of PHP and MySQL, double quotes and single quotes make your query writing time so much easier.
Besides all of the (well-explained) answers, there hasn't been the following mentioned and I visit this Q&A quite often.
In a nutshell; MySQL thinks you want to do math on its own table/column and interprets hyphens such as "e-mail" as e minus mail.
Disclaimer: So I thought I would add this as an "FYI" type of answer for those who are completely new to working with databases and who may not understand the technical terms described already.
SQL servers and MySQL, PostgreySQL, Oracle don't understand double quotes("). Thus your query should be free from double quotes(") and should only use single quotes(').
Back-trip(`) is optional to use in SQL and is used for table name, db name and column names.
If you are trying to write query in your back-end to call MySQL then you can use double quote(") or single quotes(') to assign query to a variable like:
let query = "select id, name from accounts";
//Or
let query = 'select id, name from accounts';
If ther's a where statement in your query and/or trying to insert a value and/or an update of value which is string use single quote(') for these values like:
let querySelect = "select id, name from accounts where name = 'John'";
let queryUpdate = "update accounts set name = 'John' where id = 8";
let queryInsert = "insert into accounts(name) values('John')";
//Please not that double quotes are only to be used in assigning string to our variable not in the query
//All these below will generate error
let querySelect = 'select id, name from accounts where name = "John"';
let queryUpdate = 'update accounts set name = "John" where id = 8';
let queryInsert = 'insert into accounts(name) values("John")';
//As MySQL or any SQL doesn't understand double quotes("), these all will generate error.
If you want to stay out of this confusion when to use double quotes(") and single quotes('), would recommend to stick with single quotes(') this will include backslash() like:
let query = 'select is, name from accounts where name = \'John\'';
Problem with double(") or single(') quotes arise when we had to assign some value dynamic and perform some string concatenation like:
let query = "select id, name from accounts where name = " + fName + " " + lName;
//This will generate error as it must be like name = 'John Smith' for SQL
//However our statement made it like name = John Smith
//In order to resolve such errors use
let query = "select id, name from accounts where name = '" + fName + " " + lName + "'";
//Or using backslash(\)
let query = 'select id, name from accounts where name = \'' + fName + ' ' + lName + '\'';
If need further clearance do follow quotes in JavaScript
It is sometimes useful to not use quotes... because this can highlight issues in the code generating the query... For example:
Where x and y are should always be integers...
SELECT * FROM table WHERE x= AND y=0
Is a SQL syntax error... a little lazy but can be useful...

Python pyodbc is not recognizing parameter marker. I've supplied 2 markers and two values but its not recognizing them

Any idea why the code below would not recognize the first placeholder? I'm assuming I have to put a special character in front of it but i've been unable to find any documentation around it. I've also tried just a simple "create table ?" with no success.
for champ in champion_list:
UPDATE_SQL = """\
if not exists (select * from sysobjects where name=? and xtype='U')
CREATE TABLE [dbo].[?](
[champId] [varchar](50) NOT NULL,
[championName] [varchar] NOT NULL,
[version] [varchar](50) NOT NULL
) ON [PRIMARY]
"""
values=(champ,champ)
try:
cursorprod.execute(UPDATE_SQL, values)
print str(champ),'table added.'
except Exception as e:
print(e)
I get the error
The SQL contains 1 parameter markers, but 2 parameters were supplied
Query parameters are for specifying column values in DML statements; they cannot be used to specify object (e.g., column or table) names in DDL statements. You will need to use dynamic SQL (string substitution) for that ...
... assuming that you really want to create separate tables for each item in the list. If the structure of those tables is identical then that is a bad design. You'd be better served with one table that includes an extra column to identify the list item associated with each row.

How to handle apostrophes in MySQL-Python?

A Python API is giving back u"'HOPPE'S No. 9'" as a value for a particular product attribute. I'm then looking to insert it into the DB, also using Python (python-mysqldb), with the following query:
INSERT INTO mytable (rating, Name) VALUES('5.0 (7)', 'HOPPE'S No. 9';
MySQL rejects this, and the suggested approach to handling a single quote in MySQL is to escape it first. This I need to do in Python, so I try:
In [5]: u"'HOPPE'S No. 9'".replace("'", "\'")
Out[5]: u"'HOPPE'S No. 9'"
When I incorporate this in my program, MySQL still rejects it. So I double-escape the apostrophe, and then an insert happens successfully. Thing is, it contains the escape character (so what gets written is 'HOPPE\'S No. 9').
If I need the second escape character, but when I add it gets left in, then how can I handle the escaping without having the escape character included in the string that gets inserted?
Edit: Based on theBjorn's suggestion, tried:
actualSQL = "INSERT INTO %s (%s) VALUES(%s);"
#cur.execute(queryString)
cur.execute(actualSQL,
(configData["table"], sqlFieldMappingString, sqlFieldValuesString))
but it looks like I'm back to where I was when I was trying to escape using the single escape with .replace():
Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''mytable' ('rating, Name, Image, mfg, price, URL') VALUES('\'5.0 (3)\', \'AR-1' at line 1
You should never construct sql that way. Use parameterized code instead:
cursor.execute(
"insert into mytable (rating, name) values (%s, %s);",
("5.0 (7)", "HOPPE'S No. 9")
)
your latest problem is due to the misconception that this is string interpolation, which it isn't (the use of %s is confusing), thus:
actualSQL = "INSERT INTO %s (%s) VALUES(%s);"
will be wrong. It is possible to construct your sql string, but probably easier to do so in two steps so we don't trip over sql parameter markers looking like string interpolation markers. Assuming you have the values in a tuple named field_values:
params = ["%s"] * len(field_values) # create a list with the correct number of parameter markers
sql = "insert into %s (%s) values (%s)" % ( # here we're using string interpolation, but not with the values
configData["table"],
sqlFieldMappingString,
', '.join(params)
)
if you print sql it should look like my example above. Now you can execute it with:
cursor.execute(sql, field_values)

python sqlite named parameter containing dash

I'm trying to insert a record into an sqlite database using named parameters in python (with the sqlite3 module).
The values I want to insert are in a dictionary, but the dictionary keys might contain dashes, for example {'request-id': 100, 'year': '2015'}.
I'm trying to execute the following:
import sqlite3
conn = sqlite3.connect('database.db')
cursor = conn.cursor()
cursor.execute('''CREATE TABLE IF NOT EXISTS requests (request_id text, year text)''')
query = '''INSERT INTO requests (request_id, year) VALUES (:request-id, :year)'''
cursor.execute(query, {'request-id': 100, 'year': '2015'})
conn.commit()
conn.close()
I get this error during the insert statement:
sqlite3.OperationalError: no such column: id
It seems like dashes are not well accepted as named parameters.
There are many workarounds for this, like creating a new dictionary where dashes in the keys are replaced by underscores, but I'd like to know if I could use some escaping technique or something else to avoid that.
Thanks for your help
The documentation for sqlite3_bind_* states that parameter names must be composed of alphanumeric characters, and doesn't mention a way of escaping them.
Your query is probably being parsed as :request - id, i.e. :request minus id, and since there's no such column id, SQLite throws an error.
(Also, as Prerak Sola points out, you create the table with a date column but try to insert to a year column which doesn't exist.)
SQL parameter names have no quoting or escaping mechanism; you have to use the same rules as for an unquoted identifier.

Categories

Resources