Develop Reference

Output of Python script differ depending on way of activation

hei guys,
I have an executable python script, say get_data.py (located in project_x/src/) which is working properly, when started by: python get_data.py . It gets data (a list of id's which are necessary for further calculations) from a database via mysql.connector and then processes these data in parallel (via multiprocessing) using pool.map.
BUT it is supposed to be started by an .exe-file (located in project_x/exec/)[EDIT: This .exe uses the php command exec() to directly addresses my python script]. This is not working properly but ending in the try-except-block (in wrapper_fun) when catching the (unknown) mistake and not terminating when deleting the try-except-commands.
Do you have any idea what could be going wrong? I would appreciate any idea. I tried logging but there seems to be a permission problem. My idea is that the connection the db cannot be established and therefore there are no id's.
def calculations():
do_something...
def wrapper_fun(id):
try:
calculations(id)
except Exception:
return(False)
if __name__ == "__main__":
import multiprocessing
import mysql.connector
from mysql.connector import Error
host_name = <secret_1>
user_name = <secret_2>
passt = <secret_3>
connection = None
try:
connection = mysql.connector.connect(
host=host_name,
user=user_name,
passwd=user_password
)
except Error as err:
print(f"Error: '{err}'")
d = pd.read_sql_query(query, connection,coerce_float=False)
connection.close()
id_s = list(d.ids)
results = [pool.map(wrapper_fun,id_s)]
...

RTD client in Python

my goal is to get the updates of an rtd server in python
I've following call in excel which is working:
=RTD("xrtd.xrtd";;"EUCA")
For python I've found following client library: https://github.com/brotchie/pyrtd/blob/master/rtd/client.py
I tried to get a simple example where I can connect to the server
import sys
sys.path.append(".")
from client import RTDClient
name = "xrtd.xrtd"
try:
client = RTDClient(name)
client.connect(False)
client.register_topic('EUCA')
except Exception as identifier:
print(str(name) + " error : " + str(identifier))
My first problem was that I've used 64bit python, but after I solved this I receive following exception from the connect():
xrtd.xrtd error : This COM object can not automate the makepy process
please run makepy manually for this object
I've no idea what I've to do now. I've python experience but no experience with COM Objects

Try this
import pythoncom
from rtd import RTDClient
if __name__ == '__main__':
time = RTDClient('xrtd.xrtd')
time.connect()
time.register_topic('EUCA')
while 1:
pythoncom.PumpWaitingMessages()
if time.update():
print time.get('EUCA')

Why is an unknown line being printed to stdout by sshtunnel module?

If I run this code:
import sshtunnel
try:
with sshtunnel.open_tunnel("hello", ssh_username="user",
ssh_password="PASSWORD",
remote_bind_address=("1.2.3.4", 23)) as server:
pass
except:
pass
I get this:
2016-04-06 10:47:53,006 | ERROR | Could not resolve IP address for hello, aborting!
I am ignoring the exception, but some random line is showing up for some reason. Why? Is this just some random print statement in some library somewhere? Is this common? Seems like libraries shouldn't really be printing anything to the screen directly. How do I suppress this line?
PS. Code meant to simply replicate the error - obviously using a catch-all for exceptions and doing nothing with them is bad

That looks like a logging statement, specifically logging.error().
It's going to the screen because you haven't set up a log handler which would send it somewhere else. See https://docs.python.org/2/library/logging.html for more information.
It's going to the standard error output (which on a terminal window looks the same as the regular output.) If your code were part of a web service, it would go to the web server's error log.

The first non-keyword argument you pass to open_tunnel is expected to be the destination server (either a string, or an (ip, port) tuple (see the function's docstring).
Eventually, this leads to ssh_host being set to "hello" in the example you gave, which logs an error message in this except block.

"some random line is showing up for some reason." - its a straight-forward error message... the remote server couldn't find a host called "hello". As for why you see it, sshtunnel creates a console logger for error messages if you don't pass a logger yourself. That is a strange thing to do, IMHO. open_tunnel accepts two keywoard arguments: logger is a standard python logger and debug_level is the level to log. See python logging for details on setting up a logger.

The part where you have "Hello" should ideally have the IP address of the SSH Server that you are connecting to. The random line that you mentioned is the ERROR statement that explains it.
Although the program does not have any stdout statements, the ERROR line is from the library sshtunnel.
The statement block in the library uses a raise(exeption) statement with this particular error message. raise function is used to populate messages that can be captured by except statements.

TL;DR
add threaded=False
Hello I had exactly the same ERROR
and it appears after the closing of the tunnel (release):
2018-01-16 10:52:58,685| INFO | Shutting down tunnel ('0.0.0.0', 33553)
2018-01-16 10:52:58,764| INFO | Tunnel: 0.0.0.0:33553 <> sql_database_resolved_on_remote:1433 released
2018-01-16 10:52:58,767| ERROR | Could not establish connection from ('127.0.0.1', 33553) to remote side of the tunnel
2018-01-16 10:52:58,889| DEBUG | Transport is closed
That's somewhat normal, but why it's there?
"Solution-ish"
Add threaded=False.
Both tested with open_tunnel() and SSHTunnelForwarder in with ... as tunnel: context:
import time
import socket
from sshtunnel import SSHTunnelForwarder, open_tunnel
def test_tunnel(tunnel):
# Wait for tunnel to be established ?
#tunnel.check_tunnels()
#time.sleep(0.5)
#print tunnel.tunnel_is_up
s = socket.socket()
s.settimeout(2)
for i in range(0, 10):
"""
I create a new socket each time otherwise I get
the error 106 (errno.EISCONN):
'Transport endpoint is already connected'
"""
s = socket.socket()
s.settimeout(2)
state = s.connect_ex(('localhost', tunnel.local_bind_port))
s.close()
okoko = "OK" if state == 0 else "NO"
print "%s (%s)" % (okoko, state)
with open_tunnel(
'server_i_can_ssh_with_mykey',
ssh_username='myuser',
ssh_pkey='/home/myuser/.ssh/id_rsa',
ssh_private_key_password=None, #no pwd on my key
remote_bind_address=('sql_database_resolved_on_remote', 1433), #e.g.
debug_level=10, # remove this if you test with SSHTunnelForwarder
threaded=False,
) as tunnel:
test_tunnel(tunnel)
hth

Python, Linux: How to execute multiple commands using pysftp

Here I am trying to execute commands on remote server and getting back its output, for every command, but it need a saperate connection, but if I create saperate connection or session, its responding me very slowly,
The code that I am trying run is, But its not working :
import pysftp
import paramiko
commandlist = ("ls", "uname -a","whoami")
ans = []
sftp = pysftp.Connection("localhost",username="root",password="123456",port=22)
for i in commandlist:
try:
out = sftp.execute(i)
ans.append(out)
except (paramiko.ssh_exception.AuthenticationException,paramiko.ssh_exception.SSHException), e:
ans.append(e)
return ans
working code is:
import pysftp
import paramiko
commandlist = ("ls", "uname -a")
ans = []
for i in commandlist:
try:
sftp = pysftp.Connection("localhost",username="root",password="123456",port=222)
out = sftp.execute(i)[0]
ans.append(out)
except (paramiko.ssh_exception.AuthenticationException,paramiko.ssh_exception.SSHException), e:
ans.append(e)
return ans
now here its creating connection for every command to execute,
please help me, let me know if this can be done by other liberary like paramiko or fabric and how.
Thanks.

Checking user's network environment - Python application

My Python application connects to a MSSQL database to verify some matter numbers, but this step is only necessary to assist with data integrity and does not bring the user any additional functionality.
The database is only accessible when on my office's local network. How do I check a user's environment during startup to see if this connection can be made?
I'm using pyodbc, but I also need this program to work on OS X, so I'm not importing that module until this check returns a positive result. Any ideas?

You could try something like this:
#!/usr/bin/python
import socket
mssql_server = 'foobar.example.not' # set this to your own value
s = socket.socket()
s.settimeout(3)
try:
server = [x for x in socket.getaddrinfo(mssql_server,1433) if x[0]==2 ][0][-1]
except socket.gaierror:
server = None
if server:
try:
s.connect(server)
except (socket.timeout, socket.error):
server = None
s.close()
... this should attempt to find an IPv4 address for your server (using the first one returned by getaddrinfo()) and then attempt to connect to the MS SQL TCP port (1433 by default) on that system. (Yes, change 1433 if your server is on a different port). If there's a timeout or any other socket error reported on the attempt, then server is set to None. Otherwise you probably have an MS SQL server that you could access.

Verify the host is available using ping before import:
import subprocess
host = "<hostname>"
# add sample text for ping resolution errors here
errors = ("could not find", "unreachable")
ping = "ping {0}".format(host)
(stdout, stderr) = subprocess.Popen(ping, stdout=subprocess.PIPE).communicate()
# stdout is a byte string and must be decoded before compare
if not any(error in stdout.decode("utf-8") for error in errors):
import pyodbc
.....