Develop Reference

Why df.to_sql not work with single store, but cur.executemany works with entire data frame?

I am struggling to understand why df.to_sql is throwing an error while cur.executemany works with the dataframe. I have tried both SqlAlchemy and SinglestoreDB package, but both are throwing similar error. Below is the code to load data to singlestore db with cur.executemany.
conn = s2.connect(host=host, port=port, user=user, password=password, database=db)
cur = conn.cursor()
cur.executemany(r'insert into salaries (employeeId, salary) '
r'values (:1, :2)', df)
While it throws an error when using df.to_sql to load data to singlestore.
df.to_sql("salaries", conn, index=False)
Below is the error message:
Traceback (most recent call last):
File "/Users/abhishekkumaryadav/Documents/offc_work/LogGold/test_load_pd.py", line 50, in
df.to_sql("salaries", conn, index=False)
File "/Users/abhishekkumaryadav/Documents/offc_work/LogGold/venv/lib/python3.9/site-packages/pandas/core/generic.py", line 2951, in to_sql
return sql.to_sql(
File "/Users/abhishekkumaryadav/Documents/offc_work/LogGold/venv/lib/python3.9/site-packages/pandas/io/sql.py", line 697, in to_sql
return pandas_sql.to_sql(
File "/Users/abhishekkumaryadav/Documents/offc_work/LogGold/venv/lib/python3.9/site-packages/pandas/io/sql.py", line 2189, in to_sql
table.create()
File "/Users/abhishekkumaryadav/Documents/offc_work/LogGold/venv/lib/python3.9/site-packages/pandas/io/sql.py", line 831, in create
if self.exists():
File "/Users/abhishekkumaryadav/Documents/offc_work/LogGold/venv/lib/python3.9/site-packages/pandas/io/sql.py", line 815, in exists
return self.pd_sql.has_table(self.name, self.schema)
File "/Users/abhishekkumaryadav/Documents/offc_work/LogGold/venv/lib/python3.9/site-packages/pandas/io/sql.py", line 2197, in has_table
return len(self.execute(query, [name]).fetchall()) > 0
File "/Users/abhishekkumaryadav/Documents/offc_work/LogGold/venv/lib/python3.9/site-packages/pandas/io/sql.py", line 2032, in execute
raise ex from exc
pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT name FROM sqlite_master WHERE type='table' AND name=?;': 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '?' at line 1
While it throws different error when I try sqlalchemy to create a connection. Below is the error message.
Traceback (most recent call last):
File "/Users/abhishekkumaryadav/Documents/offc_work/LogGold/venv/lib/python3.9/site-packages/pymysql/connections.py", line 613, in connect
sock = socket.create_connection(
File "/opt/brew/Cellar/python#3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 823, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/opt/brew/Cellar/python#3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 954, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
Please help. Thanks in advance!

How to Check if database exists in PyMySQL

I'm trying to check if database exists in Python. I found that it can be done using the following sql statement: cur.execute(f'SHOW DATABASES LIKE {event["db_name"]}') However,after trying to execute it I get the following errors:
[ERROR] ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'TestLAU' at line 1")
Traceback (most recent call last):
File "/var/task/handler.py", line 63, in handler
cur.execute(f'SHOW DATABASES LIKE {event["db_name"]}')
File "/var/task/pymysql/cursors.py", line 170, in execute
result = self._query(query)
File "/var/task/pymysql/cursors.py", line 328, in _query
conn.query(q)
File "/var/task/pymysql/connections.py", line 517, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/var/task/pymysql/connections.py", line 732, in _read_query_result
result.read()
File "/var/task/pymysql/connections.py", line 1075, in read
first_packet = self.connection._read_packet()
File "/var/task/pymysql/connections.py", line 684, in _read_packet
packet.check_error()
File "/var/task/pymysql/protocol.py", line 220, in check_error
err.raise_mysql_exception(self._data)
File "/var/task/pymysql/err.py", line 109, in raise_mysql_exception
raise errorclass(errno, errval)

You miss an apostrophe in your statement and you can use % in your LIKE statement in order to match more results, try the following:
cur.execute(f"SHOW DATABASES LIKE '%{event["db_name"]}%;'")
more about LIKE operator:
LIKE 'a%' Finds any values that start with "a"
LIKE '%a' Finds any values that end with "a"
LIKE '%or%' Finds any values that have "or" in any position
LIKE '_r%' Finds any values that have "r" in the second position
LIKE 'a__%' Finds any values that start with "a" and are at least 3 characters in length
LIKE 'a%o' Finds any values that start with "a" and ends with "o"

Turns out the error was caused by the fact that I needed to surround the database name with quotes so the following SQL statement worked: cur.execute(f'SHOW DATABASES LIKE "{event["db_name"]}"')

1241, 'Operand should contain 1 column(s)' when saving

I am trying to save some items to mysql with python and keep getting this error
I have connected to the db and then do a query on table1 and then later am trying to save to Resturants like this
cursor.execute("SELECT * FROM `table1`")
#do some stuff
cursor.execute("INSERT INTO Resturants (School,Resturant_Name,Phone,Street,City, State,Postal_Code,Country,Lat,Lng, Rating,Review_Count,Mobile_Url,Url) VALUES (%s,%s,%s,%s,%s, %s,%s,%s,%s,%s, %s,%s,%s,%s)", (SCHOOL,bus_name,bus_phone,bus_address,bus_city, bus_state,bus_post,bus_country,bus_lat,bus_lng, bus_rating,bus_rating_count,bus_mobile_url,bus_url))
here is error
File "build/bdist.linux-x86_64/egg/MySQLdb/cursors.py", line 174, in execute
File "build/bdist.linux-x86_64/egg/MySQLdb/connections.py", line 36, in defaulterrorhandler
_mysql_exceptions.OperationalError: (1241, 'Operand should contain 1 column(s)')
And then I get the error on the INSERT line why?

python mysqldb how to correctly escape urls

I'm trying to correctly escape urls to enter into a mysql connection, but apparently I'm doing it wrong:
>>> import MySQLdb
>>> db = MySQLdb.connect(host="localhost",user="...",passwd="...",db="...")
>>> cur = db.cursor()
>>> cmd = "insert into S3_data (url) VALUES ('http://google.com')"
>>> cur.execute(cmd)
1
>>> cur.execute(MySQLdb.escape_string(cmd))
Traceback (most recent call last):
File "<pyshell#49>", line 1, in <module>
cur.execute(MySQLdb.escape_string(cmd))
File "C:\Python34\lib\site-packages\MySQLdb\cursors.py", line 220, in execute
self.errorhandler(self, exc, value)
File "C:\Python34\lib\site-packages\MySQLdb\connections.py", line 36, in defaulterrorhandler
raise errorvalue
File "C:\Python34\lib\site-packages\MySQLdb\cursors.py", line 209, in execute
r = self._query(query)
File "C:\Python34\lib\site-packages\MySQLdb\cursors.py", line 371, in _query
rowcount = self._do_query(q)
File "C:\Python34\lib\site-packages\MySQLdb\cursors.py", line 335, in _do_query
db.query(q)
File "C:\Python34\lib\site-packages\MySQLdb\connections.py", line 280, in query
_mysql.connection.query(self, query)
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '\\'http://google.com\\')' at line 1")
As you can see the command works ok, but the escaping fails. What am I missing here?
Also, does escape string handle multi-byte encodings?
Thanks

I believe if you use parameters you shouldn't have a problem.
cmd = "INSERT INTO S3_data (url) VALUES (%s)"
args = 'http://google.com'
cur.execute(cmd, args)

Lost connection to MySQL server during Python connection

Relatively new to Python and MySQL, but I'm performing a simple query of a DB in a dev environment using the MySQL Python Connector. I've created a buffered cursor to return results as dictionaries. When I perform the simple query:
family_query = ("SELECT * FROM family as FF")
...I get a list of errors, all around this idea of error 2013: Lost Connection to MySQL server.
>python "FitMatch v1.5.py"
Traceback (most recent call last):
File "FitMatch v1.5.py", line 505, in <module>
fit_match_cursor.execute(fit_family_query, () )
File "C:\Python34\lib\site-packages\mysql\connector\cursor.py", line 507, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "C:\Python34\lib\site-packages\mysql\connector\cursor.py", line 421, in _handle_result
self._handle_resultset()
File "C:\Python34\lib\site-packages\mysql\connector\cursor.py", line 895, in _handle_resultset
(self._rows, eof) = self._connection.get_rows()
File "C:\Python34\lib\site-packages\mysql\connector\connection.py", line 671, in get_rows
rows = self._protocol.read_text_result(self._socket, count)
File "C:\Python34\lib\site-packages\mysql\connector\protocol.py", line 309, in read_text_result
packet = sock.recv()
File "C:\Python34\lib\site-packages\mysql\connector\network.py", line 226, in recv_plain
raise errors.InterfaceError(errno=2013)
mysql.connector.errors.InterfaceError: 2013: Lost connection to MySQL server during query
I've tried increasing my connection_timeout to well over 10,000 (as I read on other stack overflow posts that could be the issue) but it had no effect.
Any ideas what could be causing the "Lost connection to MySQL server" error?

As a print of nested query result I face this problem
my Python query would fail with the error described in the question after returning just a subset of results.
As my way you Switched to PyMySQL and things work as you like
PyMySQL Link
My Sample Code For printing data
import pymysql
connection = pymysql.connect(user='XYZ', passwd='XYZ',host='XYZ',database='XYZ')
cursor = connection.cursor()
query = ("YOUR_QUERY")
cursor.execute(query)
for item in cursor:
print item
If any problem with this answer must comment me....:)