Develop Reference

Cannot connect AS400 by python

I am using idb_bm module to build the connection to the AS400 by using the following code.
from ibm_db import connect
connection = connect('DATABASE=DATABASE;'
'HOSTNAME=xxx.xxx.xxx.xxx;'
'PORT=446;'
'PROTOCOL=TCPIP;'
'UID=uid;'
'PWD=pwd;', '', '')
After execute the code, it shows the following error,
connection = connect('DATABASE=DATABASE;'
SQLCODE=-30061][CLI Driver] SQL30061N The database alias or database name "DATABASE " was not found at the remote node. SQLSTATE=08004
The AS400 structure is complicated with tons of library and tons of table in each library. What's the database alias or database name actually? I have stuck to it for few days....Thanks all.

Assuming you actually have a modern POWER server running IBM i and not a 20 year old AS/400...
Download IBM Access Client Solutions (ACS)..
Set up a connection to your IBM i.
The "Schemas" component of ACS will show you a list of databases on the server.

Assuming you're connecting to the local database and not an iASP, you should be able to use *LOCAL.

When sharing a Python .exe with colleagues that uses PyODBC to query a database, is an ODBC Driver for SQL Server the only download required?

I have a python script that I have made and developed an .exe with Pyinstaller that uses python's pyodbc library to query and looks like the fake code below. It works on my computer perfectly. I tried to share my .exe with a couple co-workers, who I have double-checked have access to query our databases within Excel after providing them login information. However, they are not able to access our database from within my python .exe.
After testing once, I know that they are unable to access, because the .exe I created outputted error (No suitable driver found. Cannot connect.) as you can see in the code below, because it could not find ODBC Driver 17 for SQL Server on the colleague's computer. I believe Excel uses PowerQuery, so it doesn't have a reliance on an ODBC Driver, but python will require the driver. My question is this -- should simply installing the ODBC Driver 17 for SQL Server from this link work? https://www.microsoft.com/en-us/download/details.aspx?id=56567
The user doesn't have admin rights to download the driver (and I can't test this myself on another computer) and I have limited time and and access to IT resources; otherwise, I would troubleshoot with the user myself, so I'm eager to hear if anyone has any experience with this or has found a similar post/documentation that could be helpful to me. Again, the code is below for reference, but I don't need assistance with that. I need to explain to IT what I need them to do for the users, which is why I was wondering if Microsoft® ODBC Driver 17 for SQL Server® - Windows will be enough.
import pandas as pd
import pyodbc
#Connection and credentials
driver_name = ''
driver_names = [x for x in pyodbc.drivers() if x.endswith(' for SQL Server')]
driver_names
if driver_names:
driver_name = driver_names[-1]
if driver_name:
conn_str = f'''DRIVER={driver_name};SERVER='''
else:
print('(No suitable driver found. Cannot connect.)')
server = '111.111.11.111'
database = 'database'
username = 'username'
password = 'password'
cnxn = pyodbc.connect(conn_str+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
try:
account_id_input = input("Select Account ID: ").strip()
print("Confirming Account ID...")
#SQL Query
df = pd.read_sql_query("""
Select [Account ID], [Year-Month], [Revenue] from database.dbo.tblaccount
""",
cnxn, params=[account_id_input]
)
print("CONFIRMED")
else: print("Incorrect ID or database connection error")
df.to_csv(f'{account_id_input}_data.csv', index=False)

In short, yes ODBC driver would be only additional download required for pyodbc.
PyODBC is an open-ended Python DB-API that accepts any compliant ODBC driver from any data source (Oracle, SQL Server, PostgreSQL, SalesForce, Quickbooks, etc.). ODBC drivers serve as a middle-layer between client (i.e., Python) and database server and, like any software, they can be developed by proprietary vendors (e.g., Oracle, Microsoft), open-source developers/groups, or third-parties. In addition, ODBC is a recognized technology among many platforms like Excel and languages.
Unlike other Python DB-APIs (pymysql, cxOracle, psycopg2), pyodbc as you recognize has one requirement: the specified ODBC driver in code must be installed beforehand on the client machine (not simply the Python library installed). By the way, this is very similar to JDBC drivers with Python's jaydebeapi and Microsoft does maintain JDBC drivers for SQL Server.
Your colleagues receive the following error:
No suitable driver found. Cannot connect.
because that specific ODBC driver is not installed on their machines but is installed on your machine. Have them check what versions are available to them with the command:
pyodbc.drivers()
As background Microsoft maintains multiple ODBC drivers (of varying versions that are backwards compatible) for Windows, Linux, and MacOS machines. Because MS Excel does not have any native database layer, it usually connects with ODBC or OLEDB but with a library like DAO or ADO (counterpart to Python's library, pydobc). Possibly, your other colleagues maintain other versions such as:
'ODBC Driver 11 for SQL Server'
'ODBC Driver 13 for SQL Server'
'SQL Server'
'SQL Native Client'
'SQL Server Native Client 11.0'
With that said, if you are going to circulate your code integrated into an executable to other users, do not hard-code any ODBC credentials since as you demonstrate, CPU environments vary considerably. Instead, consider below three solutions:
Data Source Name: Create a Data Source Name (DSN) individually tailored to each user machine that can vary between different drivers, servers, username, passwords, etc.
cnxn = pyodbc.connect(dsn="myDatabase")
Configuration File: Use configuration files like yaml or json with all credentials listed in a secured folder and then integrated into connection. Many online tutorials on this.
YAML
db:
driver: 'ODBC Driver Name'
server: '111.111.11.111'
database: 'database'
username: 'username'
password: 'password'
Python
import yaml
with open('file.yaml') as f:
db_creds = yaml.load(f)
...
cnxn = pyodbc.connect(driver=db_creds['driver'], host=db_creds['server'],
uid=db_creds['username'], pwd=db_creds['password'],
database=db_creds['database'])
Environment Variables: Use environment variables permanently or temporarily set on user's machine before connection.
import os
...
cnxn = pyodbc.connect(driver = os.environ.get('MSSQL_DRIVER'),
host = os.environ.get('MSSQL_SERVER'),
uid = os.environ.get('MSSQL_USER'),
pwd = os.environ.get('MSSQL_PWD'),
database = os.environ.get('MSSQL_DB'))
Browse the web for documentation, tutorials, blogs, even other SO posts on how to troubleshoot. Above are simple examples that may need adjustment.

In short, the answer to this question turned out to be "Yes", but I am not deleting as this could be beneficial and useful for people who are researching how to share python/SQL .exe's with other users similar to how you might share an Excel Macro.
Assuming the user is on our company network, the only requirements were that the user:
1) downloads my python .exe file that utilizes the libraries: a) pyodbc and b) pandas
2) downloads an ODBC Driver for SQL Server, depending on your company's SQL Server setup. In my case, I had the user download ODBC Driver 17 for SQL Server: https://www.microsoft.com/en-us/download/details.aspx?id=56567
This means that they don't have to fully install SQL Server, just the driver!

JDBC Spark connection

I'm looking into establishing a JDBC Spark connection to use from R/python. I know that pyspark and SparkR are both available - but these seem more appropriate for interactive analysis, particularly since they reserve cluster resources for the user. I'm thinking of something more analogous to the Tableau ODBC Spark connection - something more light-weight (as I understand it) for supporting simple random access. While this seems possible and there is some documentation it isn't clear (to me) what the JDBC driver requirements are.
Should I use the org.apache.hive.jdbc.HiveDriver like I do to establish a Hive connection since Hive and Spark SQL via thrift seem closely linked? Should I swap out the hadoop-common dependency needed for my Hive connection (using HiveServer2 Port) for some spark-specific dependency (when using hive.server2.thrift.http.port)?
Also, since most of the connection functionality seems to leverage Hive, what is the key thing that causes Spark SQL to be used as the query engine instead of Hive?

As it turned out the URL that I needed to use did not match the Hive database host URL listed in the ambari. I came across the correct URL in an example for how to connect (to my cluster specifically). Given the proper URL I was able to establish a connection using the HiveDriver without issue.

Where does pyodbc get its user and pwd from when none are provided in the connection string

I inherited a project and am having what seems to be a permissions issue when trying to interact with the database. Basically we have a two step process of detach and then delete.
Does anyone know where the user would come from if the connection string only has driver, server, and database name.
EDIT
I am on Windows Server 2008 standard
EDIT
"DRIVER={%s};SERVER=%s;DATABASE=%s;" Where driver is "SQL Server"

I just did a few tests and the {SQL Server} ODBC driver apparently defaults to using Windows Authentication if the Trusted_connection and UID options are both omitted from the connection string. So, your Python script must be connecting to the SQL Server instance using the Windows credentials of the user running the script.
(On the other hand, the {SQL Server Native Client 10.0} driver seems to default to SQL Authentication unless Trusted_connection=yes is included in the connection string.)

Since you're on Windows, a few things you should know:
Using the Driver={SQL Server} only enables features and data types
supported by SQL Server 2000. For features up through 2005, use {SQL
Native Client} and for features up through 2008 use {SQL Server
Native Client 10.0}.
To view your ODBC connections, go to Start and search for "ODBC" and
bring up Data Sources (ODBC). This will list User, System, and File
DSNs in a GUI. You should find the DSN with username and password
filled in there.

Client-side pyodbc error: "Server does not exist or access denied."

I have a python application designed to pull data from a remote database server using pyodbc, then organize and display the data in a spreadsheet. I've had it working fine for several months now, with multiple coworkers in my department using it through a shared network folder.
My connection:
pyodbc.connect('DRIVER={SQL Server};
SERVER=<myServer_name>;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword')
A different employee within our same network recently tried to use the program and got this error:
pyodbc.Error: ('08001','[08001][Microsoft][ODBC SQL Server Driver]
[DBNETLIB]SQL Server does not exist or access denied. (17) (SQLDriverConnect)')
It looked like a simple permissions issue to me, so to confirm I replaced the userID and password with my own hardcoded in, but it gave the same error. Furthermore the same employee can log in and execute queries through SQL Server Management Studio without issue.
Since everyone else in our department can still use the application fine, I know it must be a client-side issue, but I just can't pinpoint the problem. Any input would be greatly appreciated, Thanks!
Updates:
Per flipperPA's answer below, I updated my connection string to include the port:
con = pyodbc.connect('''DRIVER={SQL Server};
SERVER=<myServer_name>;
PORT=1433;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword;''')
Unfortunately we still got the same error.
He is running 32-bit Windows 7 on an HP machine, the same setup as the rest of the group so it shouldn't to be an os-level issue.
He does operate SSMS on the same machine, but I ran through the telnet check just be sure - no issue there.
I've taught myself the pyodbc API and basic SQL, but I'm still relatively new to the underlying concepts of databases and remote connections. Could you explain the TDS driver a little more?

When including SERVER, I've found you often need to include the PORT as well; this is the most likely problem:
pyodbc.connect('DRIVER={SQL Server};
SERVER=<myServer_name>;
PORT=1433;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword')
I connect mostly from Linux, however. Could it be the other person is connecting from Mac OS/X or Linux? If so, they'll need to use the FreeTDS driver (MS provides one as well, but it is flaky, at best). If you continue to have problems, from the coworkers machine, make sure you can connect from the machine you're having issues with (unless its the same machine they can connect SSMS from):
telnet <myServer_name> 1433
If it connects, you're good, if it hangs on connecting, you're most likely looking at a firewall issue. Good luck!

After talking with a knowledgeable friend I was finally able to figure out my issue!
For some reason, the user's system was configured to connect using named pipes, but the server I was connecting to only had TCP/IP protocol enabled. The solution was to force the application to use TCP/IP by adding "tcp:" to the front of the server name.
The fixed connection string:
pyodbc.connect('''DRIVER={SQL Server};
SERVER=tcp:<myServer_name>;
PORT=1433;
DATABASE=<myDB_name>;
UID=personsUser;
PWD=personsPassword
''')

If for any of you still doesn't work you can try to refer the Localdb (if that's the case) by its pipe address.
If the DB name is LocalDBA, in cmd type
SqlLocalDB LocalDBA v
Get the instance pipe name and then put it on the server name:
conn_str = (
r'DRIVER={SQL Server};'
r'SERVER=np:\\.\pipe\LOCALDB#ECE5B7EE\tsql\query;'
r'PORT=1433;'
r'DATABASE=VT-DE;'
r'trusted_connection=yes;'
)