I have an application where Python acts as a client talking to another shell (tcl). I would like to have Python connect automatically - I have it all working now but I have to manually do a sock.connect in python. The problem is the tcl shell is started from the Python script in an os.system call:
def start_tcl():
exit_tcl() #Close any existing connections
server.reset(2540) #Create new socket
os.system("vivado -mode tcl -source server.tcl") #Open tcl server and connect to socket
So if I put connect right after that, the server isn't actually opened yet, and I get endpoint not terminated. Is there a way to have Python wait to connect to the socket once the server is connected? I'm sure I don't fully understand sockets, am I going about this all wrong?
I ended up using a while loop in case anyone is interested:
while ( server.connect() is "None" ):
time.sleep(1)
– Kaleb Droskiewicz
Related
I have just started using ngrok, and while using the standard procedure, I can start the tunnel using ./ngrok tcp 22 and see that tunnel open in my dashboard,
But I would like to use pyngrok, and here when I use:
from pyngrok.conf import PyngrokConfig
from pyngrok import ngrok
ngrok.set_auth_token("<NGROK_AUTH_TOKEN>")
pyngrok_config = PyngrokConfig(config_path="/opt/ngrok/ngrok.yml")
ngrok.get_tunnels(pyngrok_config=pyngrok_config)
ssh_url = ngrok.connect()
It connects and generates a tunnel, but I can't see anything open in the dashboard, why?
Maybe because the python script executes and generates URL and then stops and comes out of it, but then how to make it keep running, or how to even start a tunnel using python or even API ? Please suggest the correct script, using python or API?
The thread with the ngrok tunnel will terminate as soon as the Python process terminates. So you are correct, the reason this is happening is because your script is not long lived. The easiest way to accomplish this is by following the example in the documentation.
Another issue is how you're setting the authtoken. Since you're not using the default config_path, you need to set this before setting the authtoken so it gets updated in the correct file (you'd also need to pass it to connect()). There are a couple ways to do this, but the easiest way from the docs is to just update the default config (since that's what will be used if you don't pass a pyngrok_config to any future method calls).
I also see that you're response variable is ssh_url, so you probably want to start a TCP tunnel to a port other than 80 (the default)—perhaps you've configured this in your ngrok.yml, but if not, I've updated the call to connect() to ensure this is the type of tunnel started for you and in case others try to use this same code snippet.
Full disclosure, I am the developer of pyngrok. Here is your code snippet updated with my changes.
import os, time
from pyngrok.conf import PyngrokConfig
from pyngrok import ngrok, conf
conf.get_default().config_path = "/opt/ngrok/ngrok.yml"
ngrok.set_auth_token(os.environ.get("NGROK_AUTH_TOKEN"))
ssh_tunnel = ngrok.connect(22, "tcp")
ngrok_process = ngrok.get_ngrok_process()
try:
# Block until CTRL-C or some other terminating event
ngrok_process.proc.wait()
except KeyboardInterrupt:
print(" Shutting down server.")
ngrok.kill()
I'm trying to develop a server script using python 3.4 that runs perpetually and responds to client requests on up to 5 separate ports. My preferred platform is Debian 8.0. The platform currently runs on a virtual machine in the cloud. My script works fine when I run it off the command line - I need to now (1) keep it running once I log off the server and (2) keep several ports open through the script so that a windows client can connect to them.
For (1),
After trying several options [I tried using upstart, added the script to rc.local, used nohup with & to run it off the terminal, etc] that didn't seem to work, I eventually found something that does seem to keep the script running, even if it's not very elegant - I wrote an hourly cron script that checks to see if the script is running in the process list, and if not, to execute it.
Whenever I login to the VM now, I see the following output when I type 'ps -ef':
root 22007 21992 98 Nov10 14-12:52:59 /usr/bin/python3.4 /home/userxyz/cronserver.py
I assume that the script is running based on the fact that there is an active process in the system. I mention this part because I suspect that there could be a correlation with part (2) of my issue.
For (2),
The script is supposed to open ports 49100 - 49105 and listen for connection requests, etc. When I run the script from the terminal, zenmap from my client machine verifies that these ports are open. However, when the cron job initiates the script, these ports don't seem to stay open. My windows client program can't connect to the script either.
The python code I use for listening to a port:
f = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
f.bind((serviceIP, 49101))
f.listen(5)
while True:
scName, address = f.accept()
[code to handle request]
scName.shutdown(socket.SHUT_WR)
scName.close()
Any insight or assistance would be greatly appreciated!
What you ask is not easy because it depends on a variety of factors:
What is the frequency of the data received?
How many clients are expected to connect to this server?
Is there a chance two clients try to connect at the same time?
How long it takes to handle some received data?
What do you need to do with your data?
Write to a database?
Write to a file?
Calculate something?
Etc.
Depending on your answer you'll have some design decisions to make for your solution.
But since you need an answer, here's a hack that represent a way to do things:
import socketserver
import threading
import datetime
class SleepyGaryReceptionHandler(socketserver.BaseRequestHandler):
log_file_name = "/tmp/sleepygaryserver.log"
def handle(self):
# self.request is defined in BaseRequestHandler
data_received = self.request.recv(1024)
# self.client_address is also defined in BaseRequestHandler
sender_address = self.client_address[0]
# This is where you are supposed to do something with your data
# This is an example
self.write_to_log('Someone from {} sent us "{}"'.format(sender_address,
data_received))
# A way to stop the server from going on forever
# But you could do this other ways but it depends what condition
# should cause the shutdown
if data_received.startswith(b"QUIT"):
finishing_thread = threading.Thread(target=self.finish_in_another_thread)
finishing_thread.start()
# This will be called in another thread to terminate the server
# self.server is also defined in BaseRequestHandler
def finish_in_another_thread(self):
self.write_to_log("Shutting down the server")
self.server.shutdown()
# Write something (with a timestamp) to a text file so that we
# know something is happenning
def write_to_log(self, message):
timestamp = datetime.datetime.now()
timestamp_text = timestamp.isoformat(sep=' ', timespec='seconds')
with open(self.log_file_name, mode='a') as log_file:
log_file.write("{}: {}\n".format(timestamp_text, message))
service_address = "localhost"
port_number = 49101
server = socketserver.TCPServer((service_address, port_number),
SleepyGaryReceptionHandler)
server.serve_forever()
I'm using here the socketserver module instead of listening directly at a socket. This standard library module has been written to simplify writing a server. so use it!
All I do here is write to a text file what has been received. You would have to adapt it to your use.
But to have it running continuously use a cron job but to start it at the startup of the computer. Since this script will block until the server is stopped, we have to run it in the background. It would look something like that:
#reboot /usr/bin/python3 /home/sleepygary/sleppys_server.py &
I have tested it and after 5 hours it still does his thing.
Now like I said, it is a hack. If you want to go all the way and do things like any other services on your computer you have to program it in a certain way. You can find more information on this page: https://www.freedesktop.org/software/systemd/man/daemon.html
I'm really tired so there may be some errors here and there.
I'm working on an application in web.py which can send commands to a device through a website with buttons.
I know which buttons are pushed on my website and I get some parameters back in my python serverprogram. Python-program -> gets the basic commands out of an SQLite-database -> adds the received parameters. But I need to simultaneously set up a connection with the remote device through Ethernet (simple socket connection) to send these commands to the device. That's where I got stuck.
So I have the website working correctly, I also have a small seperate terminal program written to just make connection with the device and with a simple terminal interface to send commands. So basically I have the 2 major parts of the program working, but not simultaneously and I can't figure out how to fit them together.
I have been reading through some information to let the webserver run in a separate thread or maybe I have to connect and close the socket connection with the device each time I get information (command/parameters) from the website? Can somebody push me somewhat in the right direction?
NB: the server is running on a Raspberry Pi
Yes, your problem appears to be caused by socket connection not being thread-safe.
Each request to web.py server runs in its own thread, and if you want to access socket connection to your device, then you have to use locks or manage connection pool, if your device supports multiple connections.
To force web.py running in single thread mode please see the following answer:
Forcing single threaded request handling with web.py
Note that you don't have to lock all requests (as in that answer) and may put lock only on the part of the code where connection is used.
We are attempting to use the paramiko module for creating SSH tunnels on demand to arbitrary servers for purposes of querying remote databases. We attempted to use the forward.py demo that ships with paramiko but the big limitation is there does not seem to be an easy way to close an SSH tunnel and the SSH connection once the socket server is started up.
The limitation we have is that we cannot activate this from a shell and then kill the shell manually to stop the listner. We need to open the SSH connection, tunnel, perform some actions through the tunnel, close the tunnel, and close the SSH connection within python.
I've seen references to a server.shutdown() method but it isn't clear how to implement it correctly.
I'm not sure what you mean by "implement it correctly" -- you just need to keep track of the server object and call shutdown on it when you want. In forward.py, the server isn't kept track of, because the last line of forward_tunnel is
ForwardServer(('', local_port), SubHander).serve_forever()
so the server object is not easily reachable any more. But you can just change that to, e.g.:
global theserver
theserver = ForwardServer(('', local_port), SubHander)
theserver.serve_forever()
and run the forward_tunnel function in a separate thread, so that the main function gets control back (while the serve_forever is running in said separate thread) and can call theserver.shutdown() whenever that's appropriate and needed.
I have written a simple twisted application that connects to a server that listens on 1 or more ports. The twisted app connects to this server and usually connects to a few of the open ports at a time. This server is a serial logger that connects to serial devices and provides the serial line information through a raw TCP Socket and I need to log all this data to disk.
My current app logs any received information to disk without issue.
What I now need to do but am unable to make progress on is add the ability to interact with my application through stdin. I need to be able to issue commands to the local application but also send text commands through the connected sockets.
I have a basic prompt using basic.LineReceiver and adding this to my reactor but can't figure out how to send the data to the server or even if this is the correct way of doing this.
A simplified example would be helpful to show what I need to do.
Thanks
J
To add an interactive console to your Twisted app, see this article -- it explains how to use twisted.internet.stdio for the purpose.