Over a year ago someone asked this question: Execute .sql files that are used to run in SQL Management Studio in python.
I am writing a script in python that connects to a SQL server and creates and populates a database based on SQL commands in a large (several GBs) .sql file.
It looks like SQLCMD requires a download and install of SQL Server Express. Are there other ways to execute a .sql file from python without requiring everyone who uses my script to download and install SQL Server? Does pyodbc have this capability?
EDIT:
Here's another similar question:
execute *.sql file with python MySQLdb
Here, again, the solution is to call a utility from command (in this case, mysql.exe) with the file listed as an argument.
It seems to me that there should be a way to do this using one of Python's DB API libraries, but I haven't found it so I'm looking for an *.exe like SQLCMD or MYSQL that I can use to run the file from command line.
P.S. Please feel free to correct me if I'm not looking at this correctly. Maybe the code below is just as efficient as running from command line:
for line in open('query.sql','r'):
cursor.execute(line)
I found it's actually faster to read the file in python and execute in batches using pyodbc than it is to use the SQLCMD utility externally (and I don't have to install SQLCMD on every computer I run the scripts on!).
Here is the code I used (because pyodbc doesn't seem to have an executescript() method):
with open(scriptPath, 'r') as inp:
for line in inp:
if line == 'GO\n':
c.execute(sqlQuery)
sqlQuery = ''
elif 'PRINT' in line:
disp = line.split("'")[1]
print(disp, '\r')
else:
sqlQuery = sqlQuery + line
inp.close()
Not sure if this is what you are asking but why not use MS SQL directly from Python? There are libraries like pymssql that will allow you to do that. Or you can use ODBC.
See http://wiki.python.org/moin/SQL%20Server for a list of Python MS SQL drivers
SQLCMD and other management utilities are freely available for download. They are part of what Microsoft calls "Feature pack for SQL Server".
I understand you want to perform large bulk imports. To do this you may want to check out the the BCP utility. Download http://www.microsoft.com/en-us/download/details.aspx?id=16978
Using SQL you may also perform a BULK INSERT. This command can insert data from a file to a SQL db.
Another way is of course to use SQL Server Integration Services for pure ETL jobs.
Related
I am trying to trigger a SQL Server Agent Job (takes a backup of the db and places into a directory) from python. Unfortunately, I haven't found anything in regards to python triggering a SQL Server Agent Job (only the other way around, SQL Server Agent Job triggering a python script).
Once I get that backup, I want to restore this db into a different SQL Server using the same python script.
Thanks for any help!!
You can run a job from Transact-SQL from Python:
EXEC dbo.sp_start_job N'My Job Name';
GO
See documentation for more information.
Windows 10 Pro, Python 3.9, SQL Server 2012.
I have written a bunch of Windows-based Python scripts over the years. One aspect, I use the pypyodbc library to insert data harvested via Python into SQL Server. This has been working well for years whether my Python script is run from the Python IDLE or if the Python script is run from the command line via a Windows batch (.bat) file for automation.
I recently introduced calling a, fully operational, SQL Server stored procedure from my Windows-based Python script that is already doing SQL Server inserts successfully. When I run my Python script from the Python IDLE, the SQL Server inserts and the stored procedure runs successfully.
PROBLEM: when I run my Python script via a Windows batch (.bat) file, the SQL Server inserts are successful but the stored procedure does not run. I have also tried "Run as administrator" for the .bat file invoking my Python script. Still, the stored procedure will not run.
Given the SQL Server inserts are successful no matter what and the stored procedure WILL ONLY run when originating the Python script via the Python IDLE, perhaps this problem is a Windows configuration issue?
Thanks in advance for any guidance.
I located the problem. It was a permissions problem in SQL Server. This is now closed. Thanks.
I'm developing on an environment which I'm not allowed to install anything. It's a monitoring server and I'm making a script to work with logs and etc.
So, I need to connect to a SQL Server with Python 2.7 without any lib like pyodbc installed. Is it possible to make this? I've found nothing I could use to connect to that database.
There are certain things you can do to run sql from the command line from python:
import subprocess
x = subprocess.check_output('sqlcmd -Q "SELECT * FROM db.table"')
print x
I'm trying to install MYSQLdb on a windows client. The goal is, from the Windows client, run a python script that connects to a MySQL server on a LINUX client. Looking at the setup code (and based on the errors I am getting when I try to run setup.py for mysqldb, it appears that I have to have my own version of MySQL on the windows box. Is there a way (perhaps another module) that will let me accomplish this? I need to have people on multiple boxes run a script that will interact with a MySQL database on a central server.
you could use a pure python implementation of the mysql client like
pymysql
(can be used as a dropin-replacement for MySQLdb by calling pymysql.install_as_MySQLdb())
MySql-Connector
You don't need the entire MySQL database server, only the MySQL client libraries.
It's been a long time since I wrote python db code for windows...but I think something like this should still work.
If you're running the client only on windows machines, install the pywin32 package. This should have an odbc module in it.
Using the windows control / management tools, create an odbc entry for either the user or the system. In that entry, you'll give the connection parameter set a unique name, then select the driver (in this case MySQL), and populate the connection parameters (e.g. host name, etc.) See PyWin32 Documentation for some notes on the odbc module in pywin32.
Also, see this post: Common ways to connect to odbc from python on windows.
Periodically when using pyodbc to create a connection to an oracle database, it kills the interpreter.
import pyodbc
connectString = 'Driver={Microdsoft ODBC for Oracle};Server=<host>:<port>/<db>.<host>;uid=<username>;pwd=<password>'
cnxn = pyodbc.connect(connectString)
Say 1/5 of the time this will just drop me from the Python prompt (>>>) down to my dos prompt (C:)
Any ideas of why this happens or how to fix it?
We could not discover what was exactly causing this problem. Research leads us to think that there may be a problem with Python 2.7.1, PyODBC, and Win 2003 server.
To fix this problem we created a jar file that executed the permission check and then had the python script call that file and return the result