Develop Reference

Why is my psycopg2 query failing with `psycopg2.errors.InternalError_: Assert`?

I'm running psycopg2 on Python 3.9, querying a Redshift table.
I'm querying a table with a SELECT * passed to the cursor.execute() method:
query = 'select * from my_schema.my_table;'
cursor = connection.cursor(cursor_factory=RealDictCursor) # psycopg2 connection
cursor.execute(query)
data = cursor.fetchall()
And, I'm getting back the following:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mypython.py", <lines>
File "/usr/local/lib/python3.9/site-packages/psycopg2/extras.py", line 236, in execute
return super().execute(query, vars)
psycopg2.errors.InternalError_: Assert
DETAIL:
-----------------------------------------------
error: Assert
code: 1000
context: status == 0 - timestamp: '-9223372036854775808'
query: <query #>
location: cg_util.cpp:705
process: <process>
-----------------------------------------------
I can query other tables in different schema and the same schema, but not this table.
The table has varchar and timestamp column types. I'm using the -infinity Special Date/Time Input in one of the columns.

The issue is the -infinity timestamp value, psycopg2 doesn't seem to be agreeable to it.
In this case, I'm able to use a different built-in value, since my actual reason for using -infinity is to have a value that means "long ago".
update my_schema.my_table
set ts_column = timestamp 'epoch'
where my_table.ts_column = timestamp '-infinity';

python 2.7 variable substitution issue in MYsql statement

Thank you for reading. I have some experience with SQL, very new to python.
In the below code, i am accessing 2 databases in python 2.7
The connections work. I can query a tables that has a serial #s for devices in one statement with no issue. I then want to query a table which name matches that serial number in another database, pulling the latest value of the "Stamp" field. All of this works when i explictly name the table ccnbsc00000001, but when using variable subsitution, it fails.
When the variable currentdevice is substituted, extras characters are included. When i print that variable, those character are not present in that output. here is the code, and the error result at the bottom
#!/usr/bin/python
### Imports
import datetime
import mysql.connector
#Connect to heartbeat results database
hb_db = mysql.connector.connect(
host="localhost",
user="otheruser",
passwd="******",
database="active_devices"
)
#Connect to heartbeat results database
device_Settings_db = mysql.connector.connect(
host="localhost",
user="otheruser",
passwd="******",
database="active_devices"
)
device_settings_cursor = device_settings_db.cursor()
hb_cursor = hb_db.cursor()
## Get deviuce serial#
device_settings_cursor.execute('select device_serial from devices')
active_devices = device_settings_cursor.fetchall()
print ("these are the current devices:")
print (active_devices)
for device in active_devices:
currentdevice = device[0]
print(currentdevice)
print ("SELECT MAX(stamp) FROM (%s)" , (currentdevice,) )
hb_cursor.execute('SELECT MAX(stamp) FROM (%s)' , (currentdevice,) )
laststamp = hb_cursor.fetchone
laststamp = laststamp[0]
print("Last time stamp is:")
print(laststamp)
*
Output of print(active_devices)
[(u'ccnbsc00000001',), (u'ccnbsc00000002',)]
output of print(currentdevice)
ccnbsc00000001
(This is the correct output/value)
but I get this error in the SQL query that implies it has kept the surrounding characters ' and ')
Traceback (most recent call last):
File "./hb_notify.py", line 61, in <module>
hb_cursor.execute('SELECT MAX(stamp) FROM (%s)' , (currentccn,) )
File "/usr/lib/python2.7/site-packages/mysql/connector/cursor.py", line 551, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "/usr/lib/python2.7/site-packages/mysql/connector/connection.py", line 490, in cmd_query
result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
File "/usr/lib/python2.7/site-packages/mysql/connector/connection.py", line 395, in _handle_result
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your **SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''ccnbsc00000001')' at line 1**

Python MySQL libraries commonly insert quotation marks when you pass string arguments to them as arguments, because usually you do actually want those quotation marks. This is why you're seeing quotation marks.
The fix here is easy: instead of passing those values as arguments to your cursor, you can just insert those values directly into the string like you would if it were any other Python string. Like so:
hb_cursor.execute('SELECT MAX(stamp) FROM {0}'.format(currentdevice))
Python string arguments will remove quotes around a string, MySQL cursor arguments will keep the quotes.

Testing Python to MySQL error

I have created a little script that allows me to save data to MySQLdb. At first it was working fine when I was using:
cursor.execute('INSERT INTO people (name, text) VALUES ("dan", "test2")')
The above would save "dan" into the title and "test2" into the text. I wanted to test to see if I was able to define something and fill it in this way. For example if I was to scrape a site and say (dan = soup.title.string) or something like that it would be able to populate this data into the database. I have tried to have a look around but cannot seem to find anything.
import MySQLdb
import sys
try:
db = MySQLdb.connect(
host = 'localhost',
user = 'root',
passwd = '',
db = 'python',
)
except:
print "db not found"
dan = "dandandan"
test2 = "testing101"
cursor = db.cursor()
cursor.execute('INSERT INTO people (name, text) VALUES (dan, test2)')
cursor.execute('SELECT * FROM people')
result = cursor.fetchall()
db.commit()
db.close()
The error I am receiving is:
C:\Users\********\Desktop>python mysqltest.py
Traceback (most recent call last):
File "mysqltest.py", line 18, in <module>
cursor.execute('INSERT INTO people (name) VALUES (dan)')
File "C:\Python27\lib\site-packages\MySQLdb\cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "C:\Python27\lib\site-packages\MySQLdb\connections.py", line 36, in defau
lterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.OperationalError: (1054, "Champ 'dan' inconnu dans field list"
)

You need to use parameters.
cursor.execute('INSERT INTO people (name, text) VALUES (%s,%s)', (dan, test2))

Use prepared statements:
cursor.execute("INSERT INTO people (name, text) VALUES (%s,%s)", (dan, test2))
From the documentation :
paramstyle
String constant stating the type of parameter marker formatting
expected by the interface. Set to 'format' = ANSI C printf format
codes, e.g. '...WHERE name=%s'. If a mapping object is used for
conn.execute(), then the interface actually uses 'pyformat' = Python
extended format codes, e.g. '...WHERE name=%(name)s'. However, the API
does not presently allow the specification of more than one style in
paramstyle.
Note that any literal percent signs in the query string passed to
execute() must be escaped, i.e. %%.
Parameter placeholders can only be used to insert column values. They
can not be used for other parts of SQL, such as table names,
statements, etc.

Python using mysql connector list databases LIKE and then use those databases in order and run query

I'm trying to write a script using pythong and the mysql-connector library. The script should connect to the mysql server do a "SHOW DATABASES LIKE 'pdns_%' and then using the results returned by the query use each database and then run another query while using that database.
Here is the code
import datetime
import mysql.connector
from mysql.connector import errorcode
cnx = mysql.connector.connect (user='user', password='thepassword',
host='mysql.server.com',buffered=True)
cursor = cnx.cursor()
query = ("show databases like 'pdns_%'")
cursor.execute(query)
databases = query
for (databases) in cursor:
cursor.execute("USE %s",(databases[0],))
hitcounts = ("SELECT Monthname(hitdatetime) AS 'Month', Count(hitdatetime) AS 'Hits' WHERE hitdatetime >= Date_add(Last_day(Date_sub(Curdate(), interval 4 month)), interval 1 day) AND hitdatetime < Date_add(Last_day(Date_sub(Curdate(), interval 1 month)), interval 1 day) GROUP BY Monthname(hitdatetime) ORDER BY Month(hitdatetime)")
cursor.execute(hitcounts)
print(hitcounts)
cursor.close()
cnx.close()
When running the script it stops with the following error'd output
Traceback (most recent call last):
File "./mysql-test.py", line 18, in <module>
cursor.execute("USE %s",(databases[0],))
File "/usr/lib/python2.6/site-packages/mysql/connector/cursor.py", line 491, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "/usr/lib/python2.6/site-packages/mysql/connector/connection.py", line 635, in cmd_query
statement))
File "/usr/lib/python2.6/site-packages/mysql/connector/connection.py", line 553, in _handle_result
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''pdns_382'' at line 1
Based on the error I'm guessing there is an issue with how its doing the datbase name from the first query. Any pointers in the correct direction would be very helpful as I'm very much a beginner. Thank you very much.

Alas, the two-args form of execute does not support "meta" parameters, such as names of databases, tables, or fields (roughly, think of identifiers you wouldn't quote if writing the query out manually). So, the failing statement:
cursor.execute("USE %s",(databases[0],))
needs to be re-coded as:
cursor.execute("USE %s" % (databases[0],))
i.e, the single arg form of execute, with a string interpolation. Fortunately, this particular case does not expose you to SQL injection risks, since you're only interpolating DB names coming right from the DB engine.

How to insert a table consist of several dictionaries into MySQL tables?

I have the below result from my python code:
[
{filename:'1,2',Name:'Gorge',registration number: '6657', registration date: '2012-09-10 14:31:13'},
{filename:'5,43',Name:'mazu',registration number:'45', registration date:'2012-10-08 17:28:47'}]
and as soon as I want to put it in a MySQL table, I got this error:
Traceback (most recent call last):
File "C:\Users\jio\Datasets 1\MyTable_info.py", line 63, in <module>
cur.executemany(query,records)
File "C:\Python27\lib\site-packages\MySQLdb\cursors.py", line 243, in executemany
self.errorhandler(self, ProgrammingError, msg.args[0])
File "C:\Python27\lib\site-packages\MySQLdb\connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: not enough arguments for format string
My python code to insert the result in MySQL table is the below code:
con = MySQLdb.connect(host = "******", port=***, user = "***", passwd="*****", db="****")
with con:
cur=con.cursor()
cur.execute('''CREATE TABLE IF NOT EXISTS info(id INT(10) auto_increment primary key,file_name VARCHAR(10),
Name VARCHAR(50),Registration ID INT(50),registration time INT(50))''')
query= "INSERT INTO info (file_name, Name, Registration ID, registration time) VALUES ( %s, %s, %s, %s )"
cur.executemany(query,records)
con.commit()
Does anyone has an idea why I get this error and what does the error mean?

You are trying to insert a String into a field INT(50).
Take a look in the last field of the table, registration time field. It is an Integer and you are trying to insert values like '2012-10-08 17:28:47' or '2012-09-10 14:31:13'.
For a quick fix just change registration time field type as a VARCHAR(50).
But maybe, for perfomance issues, you should think to use some kind of TIMESTAMP field instead of a VARCHAR for this kind of purposes.
Apart from don't use INT type for field where you would like to add some kind of String.
Modify the records variable from
[ {filename:'1,2',Name:'Gorge',registration number: '6657', registration date: '2012-09-10 14:31:13'},
{filename:'5,43',Name:'mazu',registration number:'45', registration date:'2012-10-08 17:28:47'}]
to
[('1,2','Gorge','6657', '2012-09-10 14:31:13'),('5,43','mazu','45','2012-10-08 17:28:47')]
And avoid use whitespaces for the columns names as well

Try to quote field names. What the contents of records?

Don't use spaces in field names.
If you are required to, quote them with the ` character.