My goal is to display SQL queries via the Python console using the Win32com module. I am able to use a comobject to access and successfully display the fields of a SQL query however when i try to display rows i am falling into Exceptions.
Background
I have used PYODBC which works great however there is a limitation based off what SQL providers are installed and whether TLS 1.2 is enforced. The software sometimes is installed to an external SQL server and therefore no provider on the software's server that can always establish a connection. This is why i am now using the kernal of the software via com objects to access the DB as this circumvents the pitfalls of the provider needing to be installed or the latest Windows update to allow TLS 1.2 connections etc.
Win32com Code that works for me using Fields
import win32com.client
import win32com
"""Connection"""
objprox = win32com.client.Dispatch("****.DbObjectProxy") #Blanked out for security of Software
"""Set Query"""
sql1 = "select * from ServiceConsumer_t"
dbq1 = objprox.DoDatabaseQuery(sql1)
dbq1.Open(sql1)
"""Specify & Print result"""
while not dbq1.EOF:
col1 = dbq1.Fields("ID").Value
dbq1.MoveNext()
print(col1)
"""Close Session"""
dbq1.Close()
Output of the above is:
{9CAFD41E-D322-4234-BF80-CF6E11A724A0}
{CE4AAE72-0889-41E8-BDB2-ED96696DDB91}
{DC18008F-2C84-4EB4-BCCB-D94FF96E0564}
{1AAB143C-8393-4C1E-BE94-7AB44788D4E4}
This is correct as I am specifying the ID column to output and using MoveNext() to iterate. This shows I am close to my goal, however, the below code for displaying the rows never appears to work, I am now lost on why?
Win32com Code that does not work for me to display rows:
import win32com.client
import win32com
"""Connection"""
objprox = win32com.client.Dispatch("*****.DbObjectProxy") #Blanked out for security of Software
"""Set Query"""
sql2 = "select * from ServiceConsumer_t"
dbq2 = objprox.DoDatabaseQuery(sql2)
dbq2.Open(sql2)
"""Specify & Print result"""
while not dbq2.EOF:
dbq2.MoveFirst()
res = dbq2.GetRows()
dbq2.MoveNext()
print(res)
"""Close Session"""
dbq2.Close()
From this, I simply get the exception that the object GetRows has no attribute. Looking online there is very little surrounding this. Please can you suggest why the code is not working for displaying all row results? Ideally, I would like the column names displayed too.
Assuming your COM object aligns to the ADODB object, GetRows does the following:
Retrieves multiple records of a Recordset object into an array.
In Python, this array or multidimensional object translates as a nested tuple without metadata like columns names:
rst.MoveFirst()
res = rst.GetRows() # TUPLE OF TUPLES
# LOOP THROUGH ROWS
for i in range(len(res)):
# LOOP THROUGH COLUMNS
for j in range(len(res[i])):
print(res[i][j])
ADODB example:
import win32com.client
# OPEN SQL SERVER DATABASE CONNECTION
conn = win32com.client.gencache.EnsureDispatch("ADODB.Connection")
conn.Open(
"Driver={ODBC Driver 17 for SQL Server};"
f"Server=myServer;Database=myDatabase;"
"Trusted_Connection=Yes;"
)
# OPEN RECORDSET
rst = win32com.client.gencache.EnsureDispatch("ADODB.Recordset")
rst.Open("SELECT * FROM myTable", conn)
rst.MoveFirst()
res = rst.GetRows()
# LOOP THROUGH ROWS
for i in range(len(res)):
# LOOP THROUGH COLUMNS
for j in range(len(res[i])):
print(res[i][j])
# CLOSE AND RELEASE OBJECTS
rst.Close(); conn.Close()
rst = None; conn = None
del rst; del conn
Simple answer to this is I added each column manually to create the full row output like below:
col1 = dbq1.Fields("ID").Value
col2 = dbq1.Fields("Name").Value
col3 = dbq1.Fields("Login").Value
col4 = dbq1.Fields("Email").Value
print(str(col1) + " " + str(col2) + " " str(col3) + " " str(col4))
Related
I have a pretty weird problem, I am trying to extract with Python a SQL database in Azure.
Within this database, there are several tables (I explain this because you gonna see a "for" loop in the code).
I can import some tables without problem, others (the ones that take the longest, I suppose it is because size) fail.
Not only does it throw an error ( [1] 25847 killed / usr / bin / python3 ), but it directly kicks me out of the console.
Does anyone know why? Is there an easier way to calculate the size of the database without import the entire database with pd.read_sql ()?
code:
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
query = "SELECT * FROM INFORMATION_SCHEMA.TABLES"
df = pd.read_sql(query, cnxn)
df
DataConContenido = pd.DataFrame({'Nombre':[], 'TieneCon?':[],'Size':[]})
for tablas in df['TABLE_NAME']:
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
query = "SELECT * FROM " + tablas
print("vamos con "+ str(tablas))
try:
df = pd.read_sql(query, cnxn)
size=df.shape
if size[0] > 0:
DataConContenido= DataConContenido.append(dict(zip(['Nombre','TieneCon?','Size'],[tablas,True,size])),ignore_index=True)
else:
DataConContenido= DataConContenido.append(dict(zip(['Nombre','TieneCon?','Size'],[tablas,False,size])),ignore_index=True)
except:
pass
Could it be that the connection drops when it takes so long and that is why the error named above?
I think the process is getting killed in the below line :
DataConContenido= DataConContenido.append(dict(zip(['Nombre','TieneCon?','Size'],[tablas,True,size])),ignore_index=True)
You could double confirm by adding a print statement just above it.
print("Querying Completed...")
You are getting KILLED mainly because there is a probability that your process crossed some limit in the amount of system resources that you are allowed to use. This specific operation to me appears like one.
If possible you could query and append in batches rather than doing in one shot.
I'm connecting to an influx db using python. Using built in dataframe tools I am successfully accessing data and am able to do everything I'd like, accept I can't access the timestamp values. For example:
import sys
from influxdb import DataFrameClient
reload(sys)
sys.setdefaultencoding('utf-8')
user = 'reader'
password = 'oddstringtoconfusebadguys'
dbname = 'autoweights'
host = '55.777.244.112'
protocol = 'line'
port = 8086
client = DataFrameClient(host,port=8086,username=user,password=password,
database=dbname,verify_ssl=False,ssl=True)
results = client.query("select * from measurementname")
df = results['measurementname']
for index, row in df.iterrows():
print row
results look like this
Name: 2017-11-14 22:11:23.534395882+00:00, dtype: object
host C4:27:EB:D7:D9:70
value 327
I can easily access row['host'] and row['value']. The date/time stamp is obviously important but try as I might I can't find an approach to get the values.
You can get the timestamp using the index not the row parameter
for index, row in df.iterrows():
print(index)
print(row)
You can also use Pinform library, an ORM for InfluxDB to easily get timestamp, fields and tags.
I'm playing around with win32com.client for python to try to write/insert a row to a MS Access table. I've found an example of how to connect and query an Access table here. Basically, their code slightly modified for my own use is:
import win32com.client
connection = win32com.client.Dispatch(r'ADODB.Connection')
DSN = 'PROVIDER=Microsoft.Jet.OLEDB.4.0;DATA SOURCE=c:\\testdb.mdb;'
connection.Open(DSN)
recordset = win32com.client.Dispatch(r'ADODB.Recordset')
recordset.Open('SELECT * FROM Table1', connection, 1, 3)
fields_dict = {}
for x in range(recordset.Fields.Count):
fields_dict[x] = recordset.Fields.Item(x).Name
print fields_dict[x], recordset.Fields.Item(x).Value
So this tells me how to execute a select statement on the Access table. I'd like to be able to write rows and data to the table. When using win32com for MS Office products, I tend to dive into the MSDN pages and try to interpret the VBA code for python code, but this one has me a bit handcuffed. Couple that with no examples found on the internet after lengthy searches has made me second guess whether or not this is possible? hopefully someone out there has played with this before and has a suggestion.
As I mentioned in my comment to the question, using pyodbc (or pypyodbc) and the Access ODBC driver is a more common way of doing CRUD operations, but if you really want to use win32com and OLEDB then you could do an UPDATE like this:
import win32com.client
# ADODB constants
adVarWChar = 202
adInteger = 3
adParamInput = 1
connection = win32com.client.Dispatch(r'ADODB.Connection')
DSN = (
r'PROVIDER=Microsoft.Jet.OLEDB.4.0;'
r'DATA SOURCE=C:\Users\Public\mdbTest.mdb;'
)
connection.Open(DSN)
cmd = win32com.client.Dispatch(r'ADODB.Command')
cmd.ActiveConnection = connection
cmd.CommandText = "UPDATE Donors SET LastName = ? WHERE ID = ?"
cmd.Parameters.Append(cmd.CreateParameter("?", adVarWChar, adParamInput, 255))
cmd.Parameters.Append(cmd.CreateParameter("?", adInteger, adParamInput))
cmd.Parameters(0).Value = "Thompson"
cmd.Parameters(1).Value = 10
cmd.Execute()
connection.Close()
I am using cx_Oracle to fetch some data stored in Arabic characters from an Oracle database. Below is how I try to connect to the database. When I try to print the results, specially those columns stored in Arabic, I get something like "?????" which seems to me that the data was not coded properly.
I tried to print random Arabic string in Python it went alright, which indicates the problem is in the manner in which I am pulling data from the database.
connection = cx_Oracle.connect(username, password, instanceName)
wells = getWells(connection)
def getWells(conn):
cursor = conn.cursor()
wells = []
cursor.execute(sql)
clmns = len(cursor.description)
for row in cursor.fetchall():
print row
well = {}
for i in range(0, clmns):
if type(row[i]) is not datetime.datetime:
well[cursor.description[i][0]] = row[i]
else:
well[cursor.description[i][0]] = row[i].isoformat()
wells.append(well)
cursor.close()
connection.close()
return wells
In order to force a reset of the default encoding from the environment, you can call the setdefaultencoding method in the sys module.
As this is not recommended, it is not visible by default and a reload is required.
It is recommended that you attempt to fix the encoding set in the shell for the user on the host system rather than modifying in a script.
import sys
reload(sys)
sys.setdefaultencoding('utf-8')
I am currently connecting to a Sybase 15.7 server using sybpydb. It seems to connect fine:
import sys
sys.path.append('/dba/sybase/ase/15.7/OCS-15_0/python/python26_64r/lib')
sys.path.append('/dba/sybase/ase/15.7/OCS-15_0/lib')
import sybpydb
conn = sybpydb.connect(user='usr', password='pass', servername='serv')
is working fine. Changing any of my connection details results in a connection error.
I then select a database:
curr = conn.cursor()
curr.execute('use db_1')
however, now when I try to run queries, it always returns None
print curr.execute('select * from table_1')
I have tried running the use and select queries in the same execute, I have tried including go commands after each, I have tried using curr.connection.commit() after each, all with no success. I have confirmed, using dbartisan and isql, that the same queries I am using return entries.
Why am I not getting results from my queries in python?
EDIT:
Just some additional info. In order to get the sybpydb import to work, I had to change two environment variables. I added the lib paths (the same ones that I added to sys.path) to $LD_LIBRARY_PATH, i.e.:
setenv LD_LIBRARY_PATH "$LD_LIBRARY_PATH":dba/sybase/ase/15.7/OCS-15_0/python/python26_64r/lib:/dba/sybase/ase/15.7/OCS-15_0/lib
and I had to change the SYBASE path from 12.5 to 15.7. All this was done in csh.
If I print conn.error(), after every curr.execute(), I get:
("Server message: number(5701) severity(10) state(2) line(0)\n\tChanged database context to 'master'.\n\n", 5701)
I completely understand where you might be confused by the documentation. Its doesn't seem to be on par with other db extensions (e.g. psycopg2).
When connecting with most standard db extensions you can specify a database. Then, when you want to get the data back from a SELECT query, you either use fetch (an ok way to do it) or the iterator (the more pythonic way to do it).
import sybpydb as sybase
conn = sybase.connect(user='usr', password='pass', servername='serv')
cur = conn.cursor()
cur.execute("use db_1")
cur.execute("SELECT * FROM table_1")
print "Query Returned %d row(s)" % cur.rowcount
for row in cur:
print row
# Alternate less-pythonic way to read query results
# for row in cur.fetchall():
# print row
Give that a try and let us know if it works.
Python 3.x working solution:
import sybpydb
try:
conn = sybpydb.connect(dsn="Servername=serv;Username=usr;Password=pass")
cur = conn.cursor()
cur.execute('select * from db_1..table_1')
# table header
header = tuple(col[0] for col in cur.description)
print('\t'.join(header))
print('-' * 60)
res = cur.fetchall()
for row in res:
line = '\t'.join(str(col) for col in row)
print(line)
cur.close()
conn.close()
except sybpydb.Error:
for err in cur.connection.messages:
print(f'Error {err[0]}, Value {err[1]}')