I want to connect to the Microsoft Analysis Server via Python. I have seen you can do this by the package XML or olapy, but both of them required the Analysis Server to be in HTTP, which is not applicable in my case. Is it possible to connect to Analysis Server using a connection string, that is similar to Microsoft's OLAP in R?
i.e. the connection will be something like:
connection_string = "Provider=MSOLAP.8;Integrated Security=SSPI;Persist Security Info=True;Initial Catalog="Database Name";Data Source="Server Name";MDX Compatibility=1;Safety Options=2;MDX Missing Member Mode=Error;Update Isolation Level=2"
After connecting to the Analysis Server via this connection string, I expect to query the Server by some MDX/DAX code.
Thanks in advance!
Well just Googling around it seems that the library IronPython will be useful
Execute query on SQL Server Analysis Services with IronPython
I have never used WebDav before but recently my client asked me to upload some files to his server. This proccess should be automated so I decided to use python to do this
My client has given me the info about the server in the following format:
Server location: \123.456.789.012\Something
Username: user
Password: pass
Domain: somedomain
I am trying to use easywebdav framework to do the job, however I get the following results:
Code:
webdav = easywebdav.connect(
host='123.456.789.012/Something',
username='user',
port=80,
protocol="http",
password='pass'
)
print(webdav.ls())
And the the exception I get is
Operation : PROPFIND .
Expected code : 207 Multi-Status, 301 Moved Permanently
Actual code : 401 Unauthorized
I might not be understanding everything correctly since I already tried multiple frameworks and they all fail the same way, but I HAVE successfully connected to the server using the same credentials via MAC OS X webdav client built-in Finder so it does work correctly.
I am sorry for the format of the question and probably lack of details, I am currently desperate after several hours trying to fix this problem. Feel free to ask anything in comments!
Found the solution to my problem. After using Wireshark to find out how my OS X(WebdavFS) connects I found out that the server is Microsoft IIS 7.5 and requires NTLM Auth instead of Basic Auth. Haven't found any python libraries that support NTLM Auth so luckily due to the fact the app is relatively small I swithed to C++ using neon library.
I'm trying to talk to an AS400 in Python. The goal is to use SQLAlchemy, but when I couldn't get that to work I stepped back to a more basic script using just ibm_db instead of ibm_db_sa.
import ibm_db
dbConnection = ibm_db.pconnect("DATABASE=myLibrary;HOSTNAME=1.2.3.4;PORT=8471;PROTOCOL=TCPIP;UID=username;PWD=password", "", "") #this line is where it hangs
print ibm_db.conn_errormsg()
The problem seems to be the port. If I use the 50000 I see in all the examples, I get an error. If I use 446, I get an error. The baffling part is this: if I use 8471, which IBM says to do, I get no error, no timeout, no response whatsoever. I've left the script running for over twenty minutes, and it just sits there, doing nothing. It's active, because I can't use the command prompt at all, but it never gives me any feedback of any kind.
This same 400 is used by the company I work for every day, for logging, emailing, and (a great deal of) database usage, so I know it works. The software we use, which talks to the database behind the scenes, runs just fine on my machine. That tells me my driver is good, the network settings are right, and so on. I can even telnet into the 400 from here.
I'm on the SQLAlchemy and ibm_db email lists, and have been communicating with them for days about this problem. I've also googled it so much I'm starting to run out of un-visited links in my search results. No one seems to have the problem of the connection hanging indefinitely. If there's anything I can try in Python, I'll try it. I don't deal with the 400 directly, but I can ask the guy who does to check/configure whatever I need to. As I said though, several workstations can talk to the 400's database with no problems, and queries run against the library I want to access work fine, if run from the 400 itself. If anyone has any suggestions, I'd greatly appreciate hearing them. Thanks!
The README for ibm_db_sa only lists DB2 for Linux/Unix/Windows in the "Supported Database" section. So it most likely doesn't work for DB2 for i, at least not right out of the box.
Since you've stated you have IBM System i Access for Windows, I strongly recommend just using one of the drivers that comes with it (ODBC, OLEDB, or ADO.NET, as #Charles mentioned).
Personally, I always use ODBC, with either pyodbc or pypyodbc. Either one works fine. A simple example:
import pyodbc
connection = pyodbc.connect(
driver='{iSeries Access ODBC Driver}',
system='11.22.33.44',
uid='username',
pwd='password')
c1 = connection.cursor()
c1.execute('select * from qsys2.sysschemas')
for row in c1:
print row
Now, one of SQLAlchemy's connection methods is pyodbc, so I would think that if you can establish a connection using pyodbc directly, you can somehow configure SQLAlchemy to do the same. But I'm not an SQLAlchemy user myself, so I don't have example code for that.
UPDATE
I managed to get SQLAlchemy to connect to our IBM i and execute straight SQL queries. In other words, to get it to about the same functionality as simply using PyODBC directly. I haven't tested any other SQLAlchemy features. What I did to set up the connection on my Windows 7 machine:
Install ibm_db_sa as an SQLAlchemy dialect
You may be able to use pip for this, but I did it the low-tech way:
Download ibm_db_sa from PyPI.
As of this writing, the latest version is 0.3.2, uploaded on 2014-10-20. It's conceivable that later versions will either be fixed or broken in different ways (so in the future, the modifications I'm about to describe might be unnecessary, or they might not work).
Unpack the archive (ibm_db_sa-0.3.2.tar.gz) and copy the enclosed ibm_db_sa directory into the sqlalchemy\dialects directory.
Modify sqlalchemy\dialects\ibm_db_sa\pyodbc.py
Add the initialize() method to the AS400Dialect_pyodbc class
The point of this is to override the method of the same name in DB2Dialect, which AS400Dialect_pyodbc inherits from. The problem is that DB2Dialect.initialize() tries to set attributes dbms_ver and dbms_name, neither of which is available or relevant when connecting to IBM i using PyODBC (as far as I can tell).
Add the module-level name dialect and set it to the AS400Dialect_pyodbc class
Code for the above modifications should go at the end of the file, and look like this:
def initialize(self, connection):
super(DB2Dialect, self).initialize(connection)
dialect = AS400Dialect_pyodbc
Note the indentation! Remember, the initialize() method needs to belong to the AS400Dialect_pyodbc class, and dialect needs to be global to the module.
Finally, you need to give the engine creator the right URL:
'ibm_db_sa+pyodbc://username:password#host/*local'
(Obviously, substitute valid values for username, password, and host.)
That's it. At this point, you should be able to create the engine, connect to the i, and execute plain SQL through SQLAlchemy. I would think a lot of the ORM stuff should also work at this point, but I have not verified this.
The way to find out what port is needed is to look at the service table entries on the IBM i.
Your IBM i guy can use the iNav GUI or the green screen Work with Service Table Entry (WRKSRVTBLE) command
Should get a screen like so:
Service Port Protocol
as-admin-http 2001 tcp
as-admin-http 2001 udp
as-admin-https 2010 tcp
as-admin-https 2010 udp
as-central 8470 tcp
as-central-s 9470 tcp
as-database 8471 tcp
as-database-s 9471 tcp
drda 446 tcp
drda 446 udp
The default port for the DB is indeed 8471. Though drda is used for "distributed db" operations.
Based upon this thread, to use ibm_db to connect to DB2 on an IBM i, you need the IBM Connect product; which is a commercial package that has to be paid for.
This thread suggests using ODBC via the pyodbc module. It also suggests that JDBC via the JT400 toolkit may also work.
Here is an example to work with as400, sqlalchemy and pandas.
This exammple take a bunch of csv files and insert with pandas/sqlalchemy.
Only works for windows, on linux the i series odbc driver segfaults (Centos 7 and Debian 9 x68_64)
Client is Windows 10.
My as400 version is 7.3
Python is 2.7.14
installed with pip: pandas, pyodbc, imb_db_sa, sqlalchemy
You need to install i access for windows from ftp://public.dhe.ibm.com/as400/products/clientaccess/win32/v7r1m0/servicepack/si66062/
Aditionally the modifications by #JohnY on pyodbc.py
C:\Python27\Lib\site-packages\sqlalchemy\dialects\ibm_db_sa\pyodbc.py
Change line 99 to
pyodbc_driver_name = "IBM i Access ODBC Driver"
The odbc driver changed it's name.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import pandas as pd
import numpy as np
from sqlalchemy import create_engine
import glob
csvfiles=(glob.glob("c:/Users/nahum/Documents/OUT/*.csv"))
df_csvfiles = pd.DataFrame(csvfiles)
for index, row in df_csvfiles.iterrows():
datastore2=pd.read_csv(str(row[0]), delimiter=',', header=[0],skipfooter=3)
engine = create_engine('ibm_db_sa+pyodbc://DB2_USER:PASSWORD#IP_SERVER/*local')
datastore2.to_sql('table', engine, schema='SCHEMA', chunksize=1000, if_exists='append', index=False)
Hope it helps.
If you don't need Pandas/SQLAlchemy, just use pyodbc as suggested in John Y's answer. Otherwise, you can try doing what worked for me, below. It's taken from my answer to my own, similar question, which you can check out for more detail on what doesn't work (I tried and failed in so many ways before getting it working).
I created a blank file in my project to appease this message that I was receiving:
Unable to open 'hashtable_class_helper.pxi': File not found
(file:///c:/git/dashboards/pandas/_libs/hashtable_class_helper.pxi).
(My project folder is C:/Git/dashboards, so I created the rest of the path.)
With that file present, the code below now works for me. For the record, it seems to work regardless of whether the ibm_db_sa module is modified as suggested in John Y's answer, so I would recommend leaving that module alone. Note that although they aren't imported by directly, you need these modules installed: pyodbc, ibm_db_sa, and possibly future (if using Python 2...I forget if it's necessary). If you are using Python 3, I you'll need urllib.parse instead of urllib. I also have i Access 7.1 drivers installed on my computer, which probably came into play.
import urllib
import pandas as pd
from sqlalchemy import create_engine
CONNECTION_STRING = (
"driver={iSeries Access ODBC Driver};"
"system=ip_address;"
"database=database_name;"
"uid=username;"
"pwd=password;"
)
SQL= "SELECT..."
quoted = urllib.quote_plus(CONNECTION_STRING)
engine = create_engine('ibm_db_sa+pyodbc:///?odbc_connect={}'.format(quoted))
df = pd.read_sql_query(
SQL,
engine,
index_col='some column'
)
print df
I have a python application designed to pull data from a remote database server using pyodbc, then organize and display the data in a spreadsheet. I've had it working fine for several months now, with multiple coworkers in my department using it through a shared network folder.
My connection:
pyodbc.connect('DRIVER={SQL Server};
SERVER=<myServer_name>;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword')
A different employee within our same network recently tried to use the program and got this error:
pyodbc.Error: ('08001','[08001][Microsoft][ODBC SQL Server Driver]
[DBNETLIB]SQL Server does not exist or access denied. (17) (SQLDriverConnect)')
It looked like a simple permissions issue to me, so to confirm I replaced the userID and password with my own hardcoded in, but it gave the same error. Furthermore the same employee can log in and execute queries through SQL Server Management Studio without issue.
Since everyone else in our department can still use the application fine, I know it must be a client-side issue, but I just can't pinpoint the problem. Any input would be greatly appreciated, Thanks!
Updates:
Per flipperPA's answer below, I updated my connection string to include the port:
con = pyodbc.connect('''DRIVER={SQL Server};
SERVER=<myServer_name>;
PORT=1433;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword;''')
Unfortunately we still got the same error.
He is running 32-bit Windows 7 on an HP machine, the same setup as the rest of the group so it shouldn't to be an os-level issue.
He does operate SSMS on the same machine, but I ran through the telnet check just be sure - no issue there.
I've taught myself the pyodbc API and basic SQL, but I'm still relatively new to the underlying concepts of databases and remote connections. Could you explain the TDS driver a little more?
When including SERVER, I've found you often need to include the PORT as well; this is the most likely problem:
pyodbc.connect('DRIVER={SQL Server};
SERVER=<myServer_name>;
PORT=1433;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword')
I connect mostly from Linux, however. Could it be the other person is connecting from Mac OS/X or Linux? If so, they'll need to use the FreeTDS driver (MS provides one as well, but it is flaky, at best). If you continue to have problems, from the coworkers machine, make sure you can connect from the machine you're having issues with (unless its the same machine they can connect SSMS from):
telnet <myServer_name> 1433
If it connects, you're good, if it hangs on connecting, you're most likely looking at a firewall issue. Good luck!
After talking with a knowledgeable friend I was finally able to figure out my issue!
For some reason, the user's system was configured to connect using named pipes, but the server I was connecting to only had TCP/IP protocol enabled. The solution was to force the application to use TCP/IP by adding "tcp:" to the front of the server name.
The fixed connection string:
pyodbc.connect('''DRIVER={SQL Server};
SERVER=tcp:<myServer_name>;
PORT=1433;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword
''')
If for any of you still doesn't work you can try to refer the Localdb (if that's the case) by its pipe address.
If the DB name is LocalDBA, in cmd type
SqlLocalDB LocalDBA v
Get the instance pipe name and then put it on the server name:
conn_str = (
r'DRIVER={SQL Server};'
r'SERVER=np:\\.\pipe\LOCALDB#ECE5B7EE\tsql\query;'
r'PORT=1433;'
r'DATABASE=VT-DE;'
r'trusted_connection=yes;'
)
I am using Tally.ERP9, I haven't found anything about Tally ODBC connection on net. I want a python script to connect with Tally database. Can I get some links for that and some python scripts to connect tally database using Tally ODBC?
Since it supports ODBC, you might try pyodbc: http://pyodbc.sourceforge.net/
I am not aware of direct connection engine is available or not but you can have a look at this post may it will help
Specification for Tally SOAP API