Develop Reference

Errors adding a new column to a table in MySQL with Python

I'm new to MySQL and database in general, however I'm having some issue when I try to add a new column of integer inside my table. To add a new column I'm doing so:
import mysql.connector
mydb = mysql.connector.connect(
# host, user, password and database
)
mycursor = mydb.cursor(buffered = True)
# some stuff to get the variable domain
mycursor.execute('ALTER TABLE domainsMoreUsed ADD {} INTEGER(10)'.format(domain)) # domain is a string
but i get this error:
raise errors.get_mysql_exception(exc.errno, msg=exc.msg,
mysql.connector.errors.ProgrammingError: 1064 (42000): You
have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right
syntax to use near 'in INTEGER(10)' at line 1
I get the same error above also trying:
mycursor.execute('ALTER TABLE domainsMoreUsed ADD %s INTEGER(10)' % domain)
Instead when I use:
mycursor.execute('ALTER TABLE domainsMoreUsed ADD %s INTEGER(10)', (domain))
i get:
raise ValueError("Could not process parameters")
ValueError: Could not process parameters
I read some post of other users about the same error, but I couldn't find what I need. I'm pretty sure about the SQL syntax being correct.
I'm using MySQL 8.0 with Python 3.8.3 on Windows 10.
Thank you in advance for your help.

What is the string domain set to? The error message syntax to use near 'in INTEGER(10)' at line 1, implies "in", which is a reserved word. If you want to use that for a table or column name, you need to add backticks: " ` " (left of '1' on the top row of your keyboard) around them.
Change your queries like this:
mycursor.execute('ALTER TABLE domainsMoreUsed ADD `{}` INTEGER(10)'.format(domain))
mycursor.execute('ALTER TABLE domainsMoreUsed ADD `%s` INTEGER(10)', (domain))

Store numpy array in SQL Server

I try to store pickled numpy array in SQL Server as a VARBINARY(MAX) object using pyodbc. INSERT statement from SQL Server looks like this:
INSERT INTO [dbo].[Images]
([UserId]
,[FileName]
,[FeaturesVector])
VALUES
(<UserId, int,>
,<FileName, nchar(100),>
,<FeaturesVector, varbinary(max),>)
In my python code I build query as a fstring:
query = f"INSERT INTO Images(UserID, FileName, FeaturesVector) \
VALUES ('{user_id}', '{file_name}', '{features_vector}')"
When I try to insert object to database:
features_vector = np.arange(1)
features_vector.astype('float32')
features_vector = pickle.dumps(features_vector)
query = f"INSERT INTO Images(UserID, FileName, FeaturesVector) \
VALUES ('{user_id}', '{file_name}', '{features_vector}')"
cnxn.execute(query)
cnxn.commit()
I get an error:
('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Incorrect syntax near '\'. (102) (SQLExecDirectW)")

Reliable Insert Method With pyodbc - Why?
#quarkpol, I feel your pain! I ran into this a while back, and I don't appreciate why I had to do it the way that I had to do it, but I had to do it this way to get it to work, and I noticed others doing this too.
First, when you do something like the following on reading data, the following ...
query_string = f'SELECT {col_name_1}, {col_name_2} FROM {some_table} WHERE {other_col} = {some_val}
cursor.execute(query_string)
# do some stuff with what the cursor got
the python f'stuff' type statements work great!
When INSERT'ing however, I have found that I must do the following for it to work ...
command_string = f'''INSERT INTO My_Table
(Col_Name_1, Col_Name_2, Col_Name_3) VALUES (?, ?, ?);'''
cursor.execute(command_string, val_1, val_2, val_3)
conn.commit()
Again, I don't know why, but at least this works.

Dremio ODBC with Python

Getting below error while running this code in Python, If anyone could advise me on this that would be appreciated. Thanks
dataframe = pandas.read_sql(sql,cnxn)
DatabaseError: Execution failed on sql 'SELECT * FROM train_data': ('HY000', "[HY000] [Dremio][Connector] (1040) Dremio failed to execute the query: SELECT * FROM train_data\n[30038]Query execution error. Details:[ \nVALIDATION ERROR: Table 'train_data' not found\n\nSQL Query SELECT * FROM train_data\nstartLine 1\nstartColumn 15\nendLine 1\nendColumn 24\n\n[Error Id: 24c7de0e-6e23-44c6-8cb6-b0a110bbd2fd on user:31010]\n\n (org.apache.calcite.runtime.CalciteContextException) From line 1, column 15 to line 1, column 24: ...[see log] (1040) (SQLExecDirectW)")

You only need to provide your Space name, just before your table name.
for example:
SELECT * FROM
SpaceName.train_data
This is a query to fetch data from Dremio Space, Dremio source cannot be used for data ingestion. Dremio Source only be used to establish a connection between database and Dremio.

this is being solved, it says that table does not exist, should give a valid table, in dremio it can be inside a specific space

sqlalchemy syntax for using mysql constants

So i am trying to call the mysql function TIMEDATEDIFF with the SECOND constant as the first parameter like so
query = session.query(func.TIME(Log.IncidentDetection).label('detection'), func.TIMESTAMPDIFF(SECOND,Log.IncidentDetection, Log.IncidentClear).label("duration")).all()
print(query)
I have tried it as a string and I get a mysql/mariadb error:
query = session.query(func.TIME(Log.IncidentDetection).label('detection'), func.TIMESTAMPDIFF("SECOND",Log.IncidentDetection, Log.IncidentClear).label("duration")).all()
print(query)
Gives me this
sqlalchemy.exc.ProgrammingError: (mysql.connector.errors.ProgrammingError) 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''SECOND', log.`IncidentDetection`, log.`IncidentClear`) AS duration
FROM log' at line 1 [SQL: 'SELECT TIME(log.`IncidentDetection`) AS detection, TIMESTAMPDIFF(%(TIMESTAMPDIFF_1)s, log.`IncidentDetection`, log.`IncidentClear`) AS duration \nFROM log'] [parameters: {'TIMESTAMPDIFF_1': 'SECOND'}]
I am sure it is something simple, some sort of escape sequence or import that I am missing. I have looked through the sqlalchemy documentation to no avail.

To get sqlalchemy to parse the string exactly into the query I used the _literal_as_text() function
Working solution
from sqlalchemy.sql.expression import func, _literal_as_text
# ...
query = session.query(func.TIME(Log.IncidentDetection).label('detection'), func.TIMESTAMPDIFF(_literal_as_text("SECOND"),Log.IncidentDetection, Log.IncidentClear).label("duration")).all()
print(query)

Python how to know if a record inserted successfully or not

I'm using Python MySQL Connector, I inserted a record into database, and it was successful. But in Python code, how can I know if it is inserted or not?
My Table does not have a primary key.
def insert(params) :
db_connection = Model.get_db_connection()
cursor = db_connection.cursor()
try :
cursor.execute("""INSERT INTO `User`(`UID`, `IP`) VALUES(%s,%s);""", (params))
db_connection.commit()
except :
db_connection.rollback()
Model.close_db(db_connection)
return result

You can use .rowcount attribute:
cursor.execute("""INSERT INTO `User`(`UID`, `IP`) VALUES(%s,%s);""", params)
print("affected rows = {}".format(cursor.rowcount))
.rowcount This read-only attribute specifies the number of rows that
the last .execute*() produced (for DQL statements like SELECT) or
affected (for DML statements like UPDATE or INSERT). [9]