Hi I have a simple python script that uses an odbc driver to connect to a database get a dataframe and store it/overwrite an excel file. When I run the script using eclipse, it works just fine. However, when I run it by right clicking the .py file and open with py.exe, the excel file is not being overwritten/saved.
Ultimately, I want other users to be able to install python and just double click a .py script to update an excel file. Does anyone know why it is not working with the right click method? They should both be using the same interpreter when I checked.
import pyodbc
import pandas as pd
cnxn = pyodbc.connect('Driver={ODBC Driver (x64)};'
'DSN=MyDSN;'
'Server=ServerAddress;'
'Database=Stuff;')
t1 = "table1"
sql = ("select * " + "from " + t1)
writer = pd.ExcelWriter("MyExcelFile.xlsx")
dframe = pd.read_sql(sql,cnxn)
aggDf = dframe.groupby(['DEPARTMENT']).sum()
dframe.to_excel(writer,"RawSalesData", index = False)
aggDf.to_excel(writer, "SalesStats")
writer.save()
writer.close()
Below is the results of running the sys code suggested by Jacob in the comment. Seems like both methods match.
sys.version_info(major=3, minor=6, micro=5, releaselevel='final', serial=0)
[
'C:\\Users\\persona\\PythonWorkSpace\\TestPython',
'C:\\Users\\persona\\anaconda3',
'C:\\Users\\persona\\anaconda3\\DLLs',
'C:\\Users\\persona\\anaconda3\\libs',
'C:\\Users\\persona\\anaconda3\\pkgs',
'C:\\Users\\persona\\anaconda3\\conda-meta',
'C:\\Users\\persona\\anaconda3\\envs',
'C:\\Users\\persona\\anaconda3\\etc',
'C:\\Users\\persona\\anaconda3\\include',
'C:\\Users\\persona\\anaconda3\\Lib',
'C:\\Users\\persona\\anaconda3\\Library',
'C:\\Users\\persona\\anaconda3\\man',
'C:\\Users\\persona\\anaconda3\\Menu',
'C:\\Users\\persona\\anaconda3\\Scripts',
'C:\\Users\\persona\\anaconda3\\share',
'C:\\Users\\persona\\anaconda3\\sip',
'C:\\Users\\persona\\anaconda3\\tcl',
'C:\\Users\\persona\\anaconda3\\Tools',
'C:\\WINDOWS\\system32',
'C:\\Users\\persona\\Anaconda3\\python36.zip',
'C:\\Users\\persona\\Anaconda3\\lib\\site-packages',
'C:\\Users\\persona\\Anaconda3\\lib\\site-packages\\win32',
'C:\\Users\\persona\\Anaconda3\\lib\\site-packages\\win32\\lib',
'C:\\Users\\persona\\Anaconda3\\lib\\site-packages\\Pythonwin'
]
^ right click method
--------------------
sys.version_info(major=3, minor=6, micro=5, releaselevel='final', serial=0)
[
'C:\\Users\\persona\\PythonWorkSpace\\TestPython',
'C:\\Users\\persona\\PythonWorkSpace\\TestPython',
'C:\\Users\\persona\\anaconda3',
'C:\\Users\\persona\\anaconda3\\DLLs',
'C:\\Users\\persona\\anaconda3\\libs',
'C:\\Users\\persona\\anaconda3\\pkgs',
'C:\\Users\\persona\\anaconda3\\conda-meta',
'C:\\Users\\persona\\anaconda3\\envs',
'C:\\Users\\persona\\anaconda3\\etc',
'C:\\Users\\persona\\anaconda3\\include',
'C:\\Users\\persona\\anaconda3\\Lib',
'C:\\Users\\persona\\anaconda3\\Library',
'C:\\Users\\persona\\anaconda3\\man',
'C:\\Users\\persona\\anaconda3\\Menu',
'C:\\Users\\persona\\anaconda3\\Scripts',
'C:\\Users\\persona\\anaconda3\\share',
'C:\\Users\\persona\\anaconda3\\sip',
'C:\\Users\\persona\\anaconda3\\tcl',
'C:\\Users\\persona\\anaconda3\\Tools',
'C:\\Program Files\\eclipse',
'C:\\Users\\persona\\Anaconda3\\lib\\site-packages',
'C:\\Users\\persona\\Anaconda3\\lib\\site-packages\\win32',
'C:\\Users\\persona\\Anaconda3\\lib\\site-packages\\win32\\lib',
'C:\\Users\\persona\\Anaconda3\\lib\\site-packages\\Pythonwin',
'C:\\Users\\persona\\Anaconda3\\python36.zip'
]
Make a python file like this:
from sys import version_info, path
print(version_info)
print(path)
Then try opening with both methods you mentioned. The results should match, and if they don't then there is your issue. If they do match, you should add more logging/exception handling so we can see why this fails.
Related
I am trying to run a python script from VB.Net using IronPython. So far, I have installed Python and IronPython. I have the ExecPython method shown below. It works fine when I call a simple print/hello world type of script. This DBTest.py script is just using pyodbc and connecting to the database and executing a basic select query.
The error I get at the source.Execute(scope) line is "IronPython.Runtime.Exceptions.ImportException: 'No module named 'pyodbc''"
I've installed pyodbc using pip install pyodbc. The DBTest.py script runs fine when I run it with IDLE.
I'm not sure if this is a limitation or if there's something I'm missing in the setup.
Thanks in advance for your help!!
Sub ExecPython(ByVal argv As List(Of String))
Dim engine As ScriptEngine = Python.CreateEngine
Dim scriptPath As String = "C:\scripts\DBTest.py"
Dim source As ScriptSource = engine.CreateScriptSourceFromFile(scriptPath)
argv.Add("")
engine.GetSysModule.SetVariable("argv", argv)
engine.SetSearchPaths({"C:\Users\MYUSERNAME\AppData\Local\Programs\Python\Python39\Scripts",
"C:\Users\MYUSERNAME\AppData\Local\Programs\Python\Python39\include",
"C:\Users\MYUSERNAME\AppData\Local\Programs\Python\Python39\Lib",
"C:\Program Files\IronPython 3.4\Lib"})
Dim eIO As ScriptIO = engine.Runtime.IO
Dim errors As New MemoryStream
eIO.SetErrorOutput(errors, Text.Encoding.Default)
Dim results As New MemoryStream
eIO.SetOutput(results, Text.Encoding.Default)
Dim scope As ScriptScope = engine.CreateScope
source.Execute(scope)
Console.WriteLine("ERRORS:")
Console.WriteLine(FormatResult(errors.ToArray))
Console.WriteLine("")
Console.WriteLine("RESULTS:")
Console.WriteLine(FormatResult(results.ToArray))
End Sub
Here is the python script that I am calling. It runs when I run the module from IDLE.
import pyodbc
conn = pyodbc.connect('Driver={SQL Server};'
'Server=MYSERVERNAME;'
'Database=MYDBNAME;'
'Trusted_Connection=yes;')
cursor = conn.cursor() cursor.execute('SELECT * FROM dbo.TABLENAME')
for row in cursor:
print(row)
It's been a while, probably you already found a solution.
But in general, for being able to use the 'module' you need to import clr and use the addReference function. Then you must do an import of the namespace with the python style.
Example:
import clr
clr.AddReference('System')
from System import *
I have run into a peculiar issue while using the teradatasql package (installed from pypi). I use the following code (let's call it pytera.py) to query a database:
from dotenv import load_dotenv
import pandas as pd
import teradatasql
# Load the database credentials from .env file
_ = load_dotenv()
db_host = os.getenv('db_host')
db_username = os.getenv('db_username')
db_password = os.getenv('db_password')
def run_query(query):
"""Run query string on teradata and return DataFrame."""
if query.strip()[-1] != ';':
query += ';'
with teradatasql.connect(host=db_host, user=db_username,
password=db_password) as connect:
df = pd.read_sql(query, connect)
return df
When I import this function in the IPython/Python interpreter or in Jupyter Notebook, I can run queries just fine like so:
import pytera as pt
pt.run_query('select top 5 * from table_name;')
However, if I save the above code in a .py file and try to run it, I get an error message most of the time (not all the time). The error message is below.
E teradatasql.OperationalError: [Version 16.20.0.49] [Session 0] [Teradata SQL Driver] Hostname lookup failed for None
E at gosqldriver/teradatasql.(*teradataConnection).makeDriverError TeradataConnection.go:1046
E at gosqldriver/teradatasql.(*Lookup).getAddresses CopDiscovery.go:65
E at gosqldriver/teradatasql.discoverCops CopDiscovery.go:137
E at gosqldriver/teradatasql.newTeradataConnection TeradataConnection.go:133
E at gosqldriver/teradatasql.(*teradataDriver).Open TeradataDriver.go:32
E at database/sql.dsnConnector.Connect sql.go:600
E at database/sql.(*DB).conn sql.go:1103
E at database/sql.(*DB).Conn sql.go:1619
E at main.goCreateConnection goside.go:229
E at main._cgoexpwrap_e6e101e164fa_goCreateConnection _cgo_gotypes.go:214
E at runtime.call64 asm_amd64.s:574
E at runtime.cgocallbackg1 cgocall.go:316
E at runtime.cgocallbackg cgocall.go:194
E at runtime.cgocallback_gofunc asm_amd64.s:826
E at runtime.goexit asm_amd64.s:2361
E Caused by lookup None on <ip address redacted>: server misbehaving
I am using Python 3.7.3 and teradatasql 16.20.0.49 on Ubuntu (WSL) 18.04.
Perhaps not coincidentally, I run into a similar issue when trying a similar workflow on Windows (using the teradata package and the Teradata Python drivers installed). Works when I connect inside the interpreter or in Jupyter, but not in a script. In the Windows case, the error is:
E teradata.api.DatabaseError: (10380, '[08001] [Teradata][ODBC] (10380) Unable to establish connection with data source. Missing settings: {[DBCName]}')
I have a feeling that there's something basic that I'm missing, but I can't find a solution to this anywhere.
Thanks ravioli for the fresh eyes. Turns out the issue was loading in the environment variables using dotenv. My module is in a Python package (separate folder), and my script and .env files are in the working directory.
dotenv successfully reads in the environment variables (.env in my working directory) when I run the code in my original post, line by line, in the interpreter or in Jupyter. However, when I run the same code in a script, it does not find in the .env file in my working directory. That will be a separate question I'll have to find the answer to.
import teradatasql
import pandas as pd
def run_query(query, db_host, db_username, db_password):
"""Run query string on teradata and return DataFrame."""
if query.strip()[-1] != ';':
query += ';'
with teradatasql.connect(host=db_host, user=db_username,
password=db_password) as connect:
df = pd.read_sql(query, connect)
return df
The code below runs fine in a script now:
import pytera as pt
from dotenv import load_dotenv()
_ = load_dotenv()
db_host = os.getenv('db_host')
db_username = os.getenv('db_username')
db_password = os.getenv('db_password')
data = pt.run_query('select top 5 * from table_name;', db_host, db_username, db_password)
It looks like your client can't find the Teradata server, which is why you see that DBCName missing error. This should be the "system name" of your Teradata server (i.e. TDServProdA).
A couple things to try:
If you are trying to connect directly with a hostname, try disabling COP discovery in your connection with this flag: cop = false. More info
Try updating your hosts file on your local system. From the documentation:
Modifying the hosts File
If your site does not use DNS, you must define the IP address and the
Teradata Database name to use in the system hosts file on the
computer.
Locate the hosts file on the computer. This file is typically located in the following folder: %SystemRoot%\system32\drivers\etc
Open the file with a text editor, such as Notepad.
Add the following entry to the file: xxx.xx.xxx.xxx sssCOP1 where xxx.xx.xxx.xxx is the IP address and where sss is the Teradata
Database name.
Save the hosts file.
Link 1
Link 2
It is the first time I write as I really didn't find any solution to my issue.
I want to allow my user to launch some Python program from Excel.
So i have this VBA code at some point:
lg_ErrorCode = wsh.Run(str_PythonPath & " " & str_PythonArg, 1, bl_StopVBwhilePython)
If lg_ErrorCode <> 0 Then
MsgBox "Couldn't run python script! " _
+ vbCrLf + CStr(lg_ErrorCode)
Run_Python = False
End If
str_PythonPath = "C:\Python34\python.exe C:\Users\XXXX\Documents\4_Python\Scan_FTP\test.py"
str_PythonArg = "arg1 arg2"
After multiple testing, the row in error in Python is when I try to import another module (I precise that this VBA code is working without the below row in Python):
import fct_Ftp as ftp
The architecture of the module is as follow:
4_Python
-folder: Scan_FTP
- file: test.py (The one launch from VBA)
-file: fct_Ftp.py
(For information, I change the architecture of the file, and try to copy the file at some other position just to test without success)
The import has no problem when I launch Test.py directly with:
import sys, os
sys.path.append('../')
But from VBA, this import is not working.
So I figured out this more generic solution, that dont work as well from Excel/VBA
import sys, os
def m_importingFunction():
str_absPath = os.path.abspath('')
str_absPathDad = os.path.dirname(str_absPath)
l_absPathSons = [os.path.abspath(x[0]) for x in os.walk('.')]
l_Dir = l_absPathSons + [str_absPathDad]
l_DirPy = [Dir for Dir in l_Dir if 'Python' in Dir]
for Dir in l_DirPy:
sys.path.append(Dir)
print(Dir)
m_importingFunction()
try:
import fct_Ftp as ftp
# ftp = __import__ ("fct_Ftp")
write += 'YAAAA' # write a file YAAAA from Python
except:
write += 'NOOOOOO' # write a file NOOOOO from VBA
f= open(write + ".txt","w+")
f.close()
Can you please help me as it is a very tricky questions ?
Many thanks to you guys.
You are able to start your program from the command line?
Why not create a batch file with excel which you then start in a shell?
I try to use win32com in GIMP python script.
Script should appear under tab: /Grzegorz.
I noticed that "IMPORT FROM EXCEL..." shortcut appears on tab only when I delete import declaration "import win32com.client as win32". So I assume win32com causes problem.
Win32com works when I try it outside gimp as a python file so win32com is ok.
#!/usr/bin/env python
from gimpfu import *
import win32com.client as win32
def import_text_my(img):
Excel = win32.gencache.EnsureDispatch("Excel.Application")
Excel.Visible = True
title = Excel.ActiveSheet.Cells(Excel.Selection.Row, 1).Value
description = Excel.ActiveSheet.Cells(Excel.Selection.Row, 1).Value
param = pdb.gimp_image_get_active_layer(img)
ret = param.Split(" ")
if len(ret) == 1:
ret = [title, title, 'textZwyklyLeft']
output = param.replace(ret[1], title)
layerMy = img.active_layer
layerMy.Name = output
pdb.gimp_text_layer_set_text(layerMy, description)
register(
"python_import_excel",
"IMPORT FROM EXCEL",
"import_text",
"Grzegorz Mackiewicz",
"Grzegorz Mackiewicz",
"2019",
"IMPORT FROM EXCEL...",
"",
[
(PF_IMAGE, "img", "Input image", None)
],
[],
import_text_my, menu="<Image>/Grzegorz")
main()
Does anyone know if there is possiblility to use win32com in gimp script?
pypiwin32Yes, but you would have to add the win32com module to the python runtime used by Gimp.
Some indications to install things here (this is for numpy, but holds for anything installed with PIP).
This post hints that you should install pypiwin32 with PIP.
I have recently installed xlwings on my Mac and am currently trying to write a small programme to update some data(via requests). As a test, I tried to update the cryptocurrency prices via an API and write them into excel.
Without using runpython, the code works. However as soon as I run my VBA code,
I get this error:
File "<string>", line 1
import sys, os;sys.path.extend(os.path.normcase(os.path.expandvars('/Users/Dennis/Documents/crypto;
^
SyntaxError: EOL while scanning string liberal
I have searched numerous threads and forums, but can't seem to find an answer to my problem.
For a better understanding,
my python code:
import requests, json
from datetime import datetime
import xlwings as xw
def do():
parameter = {'convert' : 'EUR'}
#anfrage über API
query_ticker = requests.get('https://api.coinmarketcap.com/v1/ticker', params = parameter)
#anfragedaten in JSON-format
data_ticker = query_ticker.json()
wb = xw.Book.caller()
ws0 = wb.sheets['holdings']
for entry in data_ticker:
# update eth price
if entry['symbol'] == 'ETH':
ws0.range('B14').value = float(entry['price_eur'])
#update btc price
if entry['symbol'] == 'BTC':
ws0.range('B15').value = float(entry['price_eur'])
if entry['symbol'] == 'NEO':
ws0.range('B16').value = float(entry['price_eur'])
if entry['symbol'] == 'XRP':
ws0.range('B17').value = float(entry['price_eur'])
now = datetime.now()
write_date = '%s.%s.%s' %(now.day, now.month, now.year)
write_time = '%s:%s:%s' %(now.hour, now.minute,now.second)
ws0.range('B2').value = write_date
ws0.range('B3').value = write_time
wb.save('holdings.xlsm')
#wb.close()
this is my vba code:
Sub update_holdings()
RunPython ("import update_holdings; update_holdings.do()")
End Sub
Solved this. I just wanted to post the solution for anyone who might be confronted with the same issue.
I went to check my xlwings.conf file, in order to see the setup for "INTERPRETER" and "PYTHONPATH". I never did editing on this, however, it was formatted incorrectly.
The correct format is:
"INTERPRETER","pythonw"
"PYTHONPATH",""
My config file was setup this way:
"PYTHONPATH","
"
"INTERPRETER","Python"
Also, the path to my python was set incorrectly by default. Even though my command line works with Anaconda python 3.6, "pythonw" used the interpreter set in .bash_profile referenced to python 2.7 which came pre-installed with macOS.
Editing the config file "INTERPRETER" solved this issue.
Thanks everyone.