I need to connect to a test server and run a sql query using pyodbc. For that I first need to set ODBCINI environment variable. I tried including that in python code using subprocess.
import subprocess
import pyodbc
bashCommand = "export ODBCINI=~/odbc.ini"
process = subprocess.Popen(bashCommand.split(), stdout=subprocess.PIPE, shell=True)
pyodbc.pooling = False
conn = pyodbc.connect(DSN='test',autocommit=True,ansi=True)
cur = conn.cursor()
Below is the error I get
Traceback (most recent call last):
File "my_si_web_impressions_bcookie_dealid.py", line 15, in <module>
conn = pyodbc.connect(DSN='tdwb',autocommit=True,ansi=True)
pyodbc.Error: ('IM002', '[IM002] [DataDirect][ODBC lib] System information file not found. Please check the ODBCINI environment variable. (0) (SQLDriverConnect)')
I guess it implies that it din't export the ODBCINI variable so it couldn't connect to server. Can anyone help?
This doesn't work as you intend because a subprocess that is spawned can only affect it's own environment, and thus the one of the processes it spawns (as they inherit it).
So your bash-command is essentially a NOP.
Instead, you need to manipulate your own environment, by using os.environ:
import os
os.environ["ODBCINI"]="~/odbc.ini"
Then your connect should work.
Related
I am a complete noob when it comes to staging connections using python. How I understand it is, python already comes pre-packaged with sqlite3. I am using python 3.8.5.
Here is my attempt
connection = sqlite3.connect("\python\Lib\sqlite3")
It returns error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
sqlite3.OperationalError: unable to open database file
I got it now. So I just need to supply a filename where the database objects are stored.
I was incorrectly thinking that I would need to supply the folder location to the database.
connection = sqlite3.connect('test.db')
which stores the database objects in the test.db file within the python folder.
You don't connect to the package, you connect to a database file.
import sqlite3
con = sqlite3.connect('path/to/database.db')
Full info here.
I have a bunch of local Python scripts that I need to execute from a local MS SQL Server. There is no problem if I try to import standard Python libraries (for example, pandas). I get an error, when I try to import a Python file, called Simulator.py located in the C:/Users/amusaeva/PyCharmProjects/ARW/WorkforceModel folder.
EXEC sp_execute_external_script #language =N'Python',
#script=N'
import sys
sys.path.insert(0, "C:/Users/amusaeva/PyCharmProjects/ARW/WorkforceModel")
import Simulator
'
Here is the error message I get when running this SQL script:
Msg 39004, Level 16, State 20, Line 0
A 'Python' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 2, Line 0
An external script error occurred:
Error in execution. Check the output for more information.
Traceback (most recent call last):
File "", line 5, in
File "C:\ProgramData\MSSQLSERVER\Temp-PY\Appcontainer1\D4294516-2993-475D-9F61-DF7C5AF4FE69\sqlindb_0.py", line 35, in transform
import Simulator
ModuleNotFoundError: No module named 'Simulator'
SqlSatelliteCall error: Error in execution. Check the output for more information.
STDOUT message(s) from external script:
SqlSatelliteCall function failed. Please see the console output for more information.
Traceback (most recent call last):
File "C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\computecontext\RxInSqlServer.py", line 605, in rx_sql_satellite_call
rx_native_call("SqlSatelliteCall", params)
File "C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\RxSerializable.py", line 375, in rx_native_call
ret = px_call(functionname, params)
RuntimeError: revoscalepy function failed.
I know there is definitely a Python script called Simulator.py in that WorkforceModel directory and I don't think I have any syntax mistakes because when I run the exact three lines in Python through a PS window, the module gets imported with no problem. Why does my stored procedure not see the Python script?
It turns out that I needed to sort out some permissions. I used this article as an inspiration: https://www.red-gate.com/simple-talk/sql/data-science-sql/sql-server-machine-learning-2019-working-with-security-changes/ and here are the exact steps I followed.
The application containers are created during the SQL Server MLS setup. They are objects inside the Windows local directory, and such as all the objects in the local directory, they have a unique SID to identify the application container.
The All Application Container object has a fixed SID in the local directory, which is S-1-15-2-1 . You can use the application icacls to grant permission.
Open a command prompt as an administrator.
Run the following command:
icacls C:\Users\amusaeva\PyCharmProjects\ARW\WorkforceModel /grant *S-1-15-2-1:(OI)(CI)F /t
Open the SQL Server Configuration Manager, select ‘SQL Server Services’. Find ‘SQL Server Launchpad’, right-click on it and select ‘Restart’.
After this I ran my code with no issues (the only thing I needed to change was the slashes - using C:\\Users\\amusaeva\\PyCharmProjects\\ARW\\WorkforceModel).
I issue this statement: db = cx_Oracle.connect("user/pass#IP/BKTDW")
and I get this error:
Traceback (most recent call last):
File "", line 1, in
cx_Oracle.DatabaseError: Error while trying to retrieve text for error ORA-01804
It seems that connect method doen't work at all. I have installed the Oracle Client and I am connecting normally via Toad or Sql Developer.
Please Help!
I had to set the ORACLE_HOME variable in the system variables and also add the bin directory in the PATH system variable. THNX
Ensure that sqlplus is working from cmd line. It could be if on 64bit windows in has in PATH there is a target to non-64 bin version of oracle bin folder. In our case we ensured that 64bit location is placed in PATH . For instance place c:\Oracle\Ora11g_r2_x64\bin\ and remove c:\Oracle\ora11g_2\bin\, it was not related with ORACLE_HOME .
I have a .py file that i am trying to execute through Talend using tSystem module. I am getting below error.
Starting job MyJob at 07:06 20/02/2017.
[statistics] connecting to socket on port 3364
[statistics] connected
Traceback (most recent call last):
File "H:\Talend POC\NameParsingUSData.py", line 7, in <module>
from unidecode import unidecode
ImportError: No module named 'unidecode'
[statistics] disconnected
Job MyJob ended at 07:06 20/02/2017. [exit code=0]
This is how the job looks like.
Its strange, because when i double click the .py file, it executes and gives the result successfully. I also tried executing the file through cmd command line in command prompt, outside talend and it runs successfully as well. Not sure what might be causing the error. For reference below is my Python script.
import os
from unidecode import unidecode
import pandas as pd
dat = pd.read_csv("H:\\Talend POC\\1M_dummyData.csv")
df1.myvar = df1.myvar.str.replace(r"[^a-zA-Z., ]",r' ')
df1.myvar = df1.myvar.str.replace('\s+', ' ')
df1.myvar = df1.myvar.str.lstrip(".,' ")
df=df1.as_matrix()
df1.to_csv("H:\\Talend POC\\results_USvalidemail_parse.csv",header=True,index=False, encoding='utf-8')
Note that it successfully imports the first library, error starts from the second library import 'unicode'
Any help?
I don't know the reason why it works, but this is what my hack was. I searched for any .py file that exists in the Talend data integration dump and i found only one in the directory
C:\Users\user\Downloads\TOS_DI-20161216_1026-V6.3.1\plugins\org.apache.ant_1.9.2.v201404171502\bin
I tried installing the library in this directory in the command line.
Firstly i ran the below code in command line to go to the directory.
cd /d C:\Users\user\Downloads\TOS_DI-20161216_1026-V6.3.1\plugins\org.apache.ant_1.9.2.v201404171502\bin
Secondly, i installed the unidecode library using command pip install unidecode
I re-executed the talend job and it executed with success!!
tarting job MyJob at 08:05 20/02/2017.
[statistics] connecting to socket on port 3983
[statistics] connected
[statistics] disconnected
Job MyJob ended at 08:05 20/02/2017. [exit code=0]
I've been connecting to a MSSQL SERVER 2008 database using Microsoft SQL Server Management Studio graphic interface, as depicted in this screenshot
I would like to connect to this database using a python script. I installed pymssql and I've failed to connect to the database. I tried the following command:
import _mssql
conn = _mssql.connect(server="POLIVEIRA-PC\\MSSQLSERVER2008", user="POliveira-PC\\POliveira", password="my_password", database="database_name")
with and without the user and password flags. I always end up with this error:
Traceback (most recent call last):
File "", line 1, in
File "_mssql.pyx", line 1887, in _mssql.connect (_mssql.c:20477)
File "_mssql.pyx", line 632, in _mssql.MSSQLConnection.init (_mssql.c:6169)
_mssql.MSSQLDriverException: Connection to the database failed for an unknown reason.
Can you help me connect to this database using Python (either using pymssql module or not). I'm not experienced with Python, nor do I with SQL, therefore I would like to do it in the simplest manner possible.
I'm running Windows 7 64 bit. Pyhton v2.7.9
I recommend you to use pyodbc, if you're using anaconda, use the 3.0.10 version of pyodbc, example:
import pyodbc
from urllib.parse import quote_plus
params = quote_plus("DRIVER={SQL Server};SERVER=POLIVEIRA-PC\\MSSQLSERVER2008;DATABASE=dbname;UID=userid;PWD=password")
try: cnxn = create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
except Exception as e:
raise SystemExit('Error: Conexion Base de Datos SQL %s' % e)
And if the problem is a remote connection in this links they talk about it
https://blogs.msdn.microsoft.com/walzenbach/2010/04/14/how-to-enable-remote-connections-in-sql-server-2008/
http://akawn.com/blog/2012/01/configuring-sql-server-2008-r2-express-edition-for-remote-access/
Hope this works
You need to create a login with SQL Server authentication and then connect with this user:
(you'll need to connect this login to a user etc., but that's unrelated to logging in), then use
import pymssql
pymssql.connect(host=r"POLIVEIRA-PC\MSSQLSERVER2008", user='logintest', password='secret', database='database_name')
don't use the _mssql module directly.