Develop Reference

How do I load a .sql file in a python environment?

I have a .sql file which I'm trying to load in an online Python environment (JupyterHub) but other code I've found online has just left me confused. I've gotten as far as:
import sqlite3
from sqlite3 import connect
sqlite_uri = "sqlite:///basketball.db"
sqlite_engine = sqlalchemy.create_engine(sqlite_uri)
connection = sqlite3.connect(":memory:")
cursor = connection.cursor()
sql_file = open("travel-times.sql")
travel = sql_file.read()
travel
sql_expr = """
SELECT *
FROM travel;
"""
pd.read_sql(sql_expr, sqlite_engine)
and calling the 'travel' object does at least print the data in raw form, but from there I'm at a loss to actually load the table from here. What commands would accomplish this?

Importing .bak MySQL database with Python using pymssql

The title is pretty self explanotory.
I've tried the following code :
import _mssql
conn = _mssql.connect(server='', user='', password='', database='')
conn.execute_non_query("IF EXISTS (SELECT 0 FROM sys.databases WHERE name = 'mydb') BEGIN ALTER DATABASE mydb MODIFY NAME = mydb_old END")
conn.execute_non_query("RESTORE DATABASE mydb FROM DISK='C:\mydb.bak'")
But I get the following error : No module named '_mssql'.
I have the version 2.2.2 of pymssql and I use Python 3.9.
I'm just trying to write and read from this database, and I only have the .bak file. I'm quite new to SQL, so I might be doing it the wrong way ? I only have a .bak file though, nothing else.
Thank you for your time.

As mentioned in the comments:
MySQL <> MS SQL
And I have to fill the server = "", user = "", etc ...

Connecting and testing a JDBC driver from Python

I'm trying to do some testing on our JDBC driver using Python.
Initially figuring out JPype, I eventually managed to connect the driver and execute select queries like so (reproducing a generalized snippet):
from __future__ import print_function
from jpype import *
#Start JVM, attach the driver jar
jvmpath = 'path/to/libjvm.so'
classpath = 'path/to/JDBC_Driver.jar'
startJVM(jvmpath, '-ea', '-Djava.class.path=' + classpath)
# Magic line 1
driver = JPackage('sql').Our_Driver
# Initiating a connection via DriverManager()
jdbc_uri = 'jdbc:our_database://localhost:port/database','user', 'passwd')
conn = java.sql.DriverManager.getConnection(jdbc_uri)
# Executing a statement
stmt = conn.createStatement()
rs = stmt.executeQuery ('select top 10 * from some_table')
# Extracting results
while rs.next():
''' Magic #2 - rs.getStuff() only works inside a while loop '''
print (rs.getString('col_name'))
However, I've failed to to batch inserts, which is what I wanted to test. Even when executeBatch() returned a jpype int[], which should indicate a successful insert, the table was not updated.
I then decided to try out py4j.
My plight - I'm having a hard time figuring out how to do the same thing as above. It is said py4j does not start a JVM on its own, and that the Java code needs to be prearranged with a GatewayServer(), so I'm not sure it's even feasible.
On the other hand, there's a library named py4jdbc that does just that.
I tinkered through the dbapi.py code but didn't quite understand the flow, and am pretty much jammed.
If anyone understands how to load a JDBC driver from a .jar file with py4j and can point me in the right direction, I'd be much grateful.

add a commit after adding the records and before retrieving.
conn.commit()

I have met a similar problem in airflow, I used teradata jdbc jars and jaydebeapi to connect teradata database and execute sql:
[root#myhost transfer]# cat test_conn.py
import jaydebeapi
from contextlib import closing
jclassname='com.teradata.jdbc.TeraDriver'
jdbc_driver_loc = '/opt/spark-2.3.1/jars/terajdbc4-16.20.00.06.jar,/opt/spark-2.3.1/jars/tdgssconfig-16.20.00.06.jar'
jdbc_driver_name = 'com.teradata.jdbc.TeraDriver'
host='my_teradata.address'
url='jdbc:teradata://' + host + '/TMODE=TERA'
login="teradata_user_name"
psw="teradata_passwd"
sql = "SELECT COUNT(*) FROM A_TERADATA_TABLE_NAME where month_key='202009'"
conn = jaydebeapi.connect(jclassname=jdbc_driver_name,
url=url,
driver_args=[login, psw],
jars=jdbc_driver_loc.split(","))
with closing(conn) as conn:
with closing(conn.cursor()) as cur:
cur.execute(sql)
print(cur.fetchall())
[root#myhost transfer]# python test_conn.py
[(7734133,)]
[root#myhost transfer]#

In py4j, with your respective JDBC uri:
from py4j.java_gateway import JavaGateway
# Open JVM interface with the JDBC Jar
jdbc_jar_path = '/path/to/jdbc_driver.jar'
gateway = JavaGateway.launch_gateway(classpath=jdbc_jar_path)
# Load the JDBC Jar
jdbc_class = "com.vendor.VendorJDBC"
gateway.jvm.class.forName(jdbc_class)
# Initiate connection
jdbc_uri = "jdbc://vendor:192.168.x.y:zzzz;..."
con = gateway.jvm.DriverManager.getConnection(jdbc_uri)
# Run a query
sql = "select this from that"
stmt = con.createStatement(sql)
rs = stmt.executeQuery()
while rs.next():
rs.getInt(1)
rs.getFloat(2)
.
.
rs.close()
stmt.close()

Write to MS Access table, python win32com

I'm playing around with win32com.client for python to try to write/insert a row to a MS Access table. I've found an example of how to connect and query an Access table here. Basically, their code slightly modified for my own use is:
import win32com.client
connection = win32com.client.Dispatch(r'ADODB.Connection')
DSN = 'PROVIDER=Microsoft.Jet.OLEDB.4.0;DATA SOURCE=c:\\testdb.mdb;'
connection.Open(DSN)
recordset = win32com.client.Dispatch(r'ADODB.Recordset')
recordset.Open('SELECT * FROM Table1', connection, 1, 3)
fields_dict = {}
for x in range(recordset.Fields.Count):
fields_dict[x] = recordset.Fields.Item(x).Name
print fields_dict[x], recordset.Fields.Item(x).Value
So this tells me how to execute a select statement on the Access table. I'd like to be able to write rows and data to the table. When using win32com for MS Office products, I tend to dive into the MSDN pages and try to interpret the VBA code for python code, but this one has me a bit handcuffed. Couple that with no examples found on the internet after lengthy searches has made me second guess whether or not this is possible? hopefully someone out there has played with this before and has a suggestion.

As I mentioned in my comment to the question, using pyodbc (or pypyodbc) and the Access ODBC driver is a more common way of doing CRUD operations, but if you really want to use win32com and OLEDB then you could do an UPDATE like this:
import win32com.client
# ADODB constants
adVarWChar = 202
adInteger = 3
adParamInput = 1
connection = win32com.client.Dispatch(r'ADODB.Connection')
DSN = (
r'PROVIDER=Microsoft.Jet.OLEDB.4.0;'
r'DATA SOURCE=C:\Users\Public\mdbTest.mdb;'
)
connection.Open(DSN)
cmd = win32com.client.Dispatch(r'ADODB.Command')
cmd.ActiveConnection = connection
cmd.CommandText = "UPDATE Donors SET LastName = ? WHERE ID = ?"
cmd.Parameters.Append(cmd.CreateParameter("?", adVarWChar, adParamInput, 255))
cmd.Parameters.Append(cmd.CreateParameter("?", adInteger, adParamInput))
cmd.Parameters(0).Value = "Thompson"
cmd.Parameters(1).Value = 10
cmd.Execute()
connection.Close()

Fast MySQL Import

Writing a script to convert raw data for MySQL import I worked with a temporary textfile so far which I later imported manually using the LOAD DATA INFILE... command.
Now I included the import command into the python script:
db = mysql.connector.connect(user='root', password='root',
host='localhost',
database='myDB')
cursor = db.cursor()
query = """
LOAD DATA INFILE 'temp.txt' INTO TABLE myDB.values
FIELDS TERMINATED BY ',' LINES TERMINATED BY ';';
"""
cursor.execute(query)
cursor.close()
db.commit()
db.close()
This works but temp.txt has to be in the database directory which isn't suitable for my needs.
Next approch is dumping the file and commiting directly:
db = mysql.connector.connect(user='root', password='root',
host='localhost',
database='myDB')
sql = "INSERT INTO values(`timestamp`,`id`,`value`,`status`) VALUES(%s,%s,%s,%s)"
cursor=db.cursor()
for line in lines:
mode, year, julian, time, *values = line.split(",")
del values[5]
date = datetime.strptime(year+julian, "%Y%j").strftime("%Y-%m-%d")
time = datetime.strptime(time.rjust(4, "0"), "%H%M" ).strftime("%H:%M:%S")
timestamp = "%s %s" % (date, time)
for i, value in enumerate(values[:20], 1):
args = (timestamp,str(i+28),value, mode)
cursor.execute(sql,args)
db.commit()
Works as well but takes around four times as long which is too much. (The same for construct was used in the first version to generate temp.txt)
My conclusion is that I need a file and the LOAD DATA INFILE command to be faster. To be free where the textfile is placed the LOCAL option seems useful. But with MySQL Connector (1.1.7) there is the known error:
mysql.connector.errors.ProgrammingError: 1148 (42000): The used command is not allowed with this MySQL version
So far I've seen that using MySQLdb instead of MySQL Connector can be a workaround. Activity on MySQLdb however seems low and Python 3.3 support will probably never come.
Is LOAD DATA LOCAL INFILE the way to go and if so is there a working connector for python 3.3 available?
EDIT: After development the database will run on a server, script on a client.

I may have missed something important, but can't you just specify the full filename in the first chunk of code?
LOAD DATA INFILE '/full/path/to/temp.txt'
Note the path must be a path on the server.

To use LOAD DATA INFILE with every accessible file you have to set the
LOCAL_FILES client flag while creating the connection
import mysql.connector
from mysql.connector.constants import ClientFlag
db = mysql.connector.connect(client_flags=[ClientFlag.LOCAL_FILES], <other arguments>)