The log output of my python program (using the builtin logging module, but occurs even when using simple prints) is partially messed up, as you can see in the following image. Note the first line, first word still being correct and then it gets mixed up:
I tried to visualize the situation where this happens:
Basically in my main thread/program I start a simple socketserver.TCPServer to listen for incoming messages. That server runs on its own thread (QtCore.QThread) so my program is not blocked. If some other application sends a message the request handler of the TCPServer will simply forward the message to the main thread using a QtCore.SIGNAL like:
self.emit(QtCore.SIGNAL('received(const QString)'), receivedMessage)
The program then does some parsing and computation with that message and logs those, thereby producing the gibberish seen above. At some point the logging returns back to working normally.
I am not sure if this is related to sockets or threading or both, but I guess it may be a common issue and therefore I am thankful for any hints why this occurs.
I think I have located the problem:
When the external application wants to send a message it will always create a new client socket, connect to the server, send the message and then close the client socket.
The sock.close() does not seem to close immediately, the docs say I should call sock.shutdown(how) first, but unfortunately this did not help as well. I can use a small time.sleep(0.5) after the close to fix the logging issue, but instead I did something like this:
def ensure_closed(self):
while True:
try:
self.sock.recv(1024)
except:
break
def close_connection(self):
self.sock.close()
self.ensure_closed()
# Continue with other stuff.
# Now the logging behaves normally.
There might be better ways to do it.
Related
I'm starting to learn more about TCP protocols in Python and I've been having some trouble with blocking threads inside clients.
Ideally, my application would work like this: I have different clients with thread functions, each one of them containing an input function in order to receive a specific command to send to the server (for example 'X'). When the 'X' is tapped in ONE client, the server receives it and sends a message to all the other clients informing that the program will continue and releasing them from their input functions - almost like cancelling them.
The problem lies on the fact that the input functions are blocking the clients from leaving the loop. I've tried setting the input thread functions as daemon but it blocks until you tap something anyway - which is unfortunately the only workaround that I've found so far.
I would like to use socket and the select module for connection, without being attached to any particular OS (so no msvcrt that works on Windows or the select module to monitor the stdin, which is only available in UNIX based OS).
Any help would be greatly appreciated!
I tried to make a simple chat system with the socket module in Python. everything works, except, that i need to kill the process everytime when i want to shutdown the server. And i don't want to do this everytime.
So my question is:
How can i make a function, that when i type shutdown in the server terminal, it shutdowns the whole server?
I already tried to do this:
def close(self):
server.close(self)
server.shutdown(self)
But it doesn't work. When i type close(), nothing happens. Nothing.
Heres the full code of the server.py:
https://pastebin.com/gA4QYmQe
Every help is useful. Thanks.
Well... there are many problems with the code (your "MY IP" and "SERVERIP" are probably not what you want, but this is beside the point.
Your close() function has a "self" parameter, which is pointless as this is not a class. You also need to move the close function to the beginning of your code if you want to call it from your try-except -structure. You need to call shutdown() first and then close(), and shutdown takes an argument. I modified your close() to do this and it works.
def close():
server.shutdown(socket.SHUT_RDWR)
server.close()
print ("closed")
When you open your socket, you should also set SO_REUSEADDR to make the address reusable (meaning you can start the server again if you shut it down, instead of waiting for a minute for TIME_WAIT status to finish with your server port):
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
But how exactly are you calling close() when you type "shutdown" somewhere? You are not doing this. Your program is in the socket loop and is not reading keyboard input.
I see no point whatsoever adding keyboard input to this program. First, it adds complexity as you are operating with two possibly blocking inputs (socket input and keyboard input) and you would need to manage this. It is possible but definitely complicated. Second, it is faster to press Control + C instead of typing "shutdown" and hitting enter.
You currently do not call close after a keyboard interrupt. I added this to the inner KeyboardInterrupt (the outer you can remove - it is not doing anything and is never reached) and it now shuts down your program neatly, closing all connections. Remember to move close() function from the bottom of your code to the front before the try: statement:
except KeyboardInterrupt:
print("[!] Keyboard Interrupted!")
close()
break
If you want a remote shutdown (server shuts down if you type "shutdown" to the socket), you can add this to your server loop:
if message == "shutdown":
close()
exit(0)
There are other problems as well. For example, if you start your server, connect to it and shut down the connection, your server exits as it does not return to listen().
This is also (in my opinion), somewhat bad programming, as you use "server" as a global variable. I would rather create a class and put all socket operations in it, but if style is not important, this should work.
Hannu
One other approach would be having a default admin-client that can control the server. Admin-client will be created when the server starts and from that client admin can shutdown and do any of the admin tasks on the server.
I am rather a noob here, but trying to setup a script where I can poll a socket, and when no socket data has been sent, a loop continues to run and do other things. I have been playing with several examples I found using select(), but no matter how I organize the code, it seems to stop on or near the server.recv() line and wait for a response. I want to skip out of this if no data has been sent by a client, or if no client connection exists.
Note that this application does not require the server script to send any reply data, if it makes any difference.
The actual application is to run a loop and animate some LEDs (which needs root access to the I/O on a Raspberry Pi). I am going to send this script data from another separate script via sockets that will pass in control parameters for the animations. This way the external script does not require root access.
So far the sending and receiving of data works great, I just can't get loop to keep spinning in the absence of incoming data. It is my understanding that this is what select() was intended to allow, but the examples I've found don't seem to be working that way.
I have attempted adding server.setblocking(0) a few different places to no avail. (If I understand correctly a non-blocking instance should allow the code to skip over the recv() if no data has been sent, but I may be off on this).
I have based my code on an example here:
http://ilab.cs.byu.edu/python/select/echoserver.html
Here is the server side script followed by the client side script.
Server Code: sockselectserver.py
#!/usr/bin/env python
import select
import socket
import sys
server = socket.socket()
host = socket.gethostname()
port = 20568
size = 1024
server.bind((host,port))
server.listen(5)
input = [server,sys.stdin]
running = 1
while running:
inputready,outputready,exceptready = select.select(input,[],[])
for s in inputready:
if s == server:
# handle the server socket
client, address = server.accept()
input.append(client)
elif s == sys.stdin:
# handle standard input
junk = sys.stdin.readline()
running = 0
else:
# handle all other sockets
data = s.recv(size)
if data:
s.send(data)
else:
s.close()
input.remove(s)
print "looping"
server.close()
Client Code: skclient.py
#!/usr/bin/python # This is client.py file
import socket # Import socket module
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 20568 # Reserve a port for your service.
s.connect((host, port))
data = "123:120:230:51:210:120:55:12:35:24"
s.send(data)
print s.recv(1024)
s.close # Close the socket when done
What I would like to achieve by this example is to see "looping" repeated forever, then when the client script sends data, see that data print, then see the "looping" resume printing over and over. That would tell me it's doing what is intended I can take it from there.
Interesting enough, when I test this as is, whenever I run the client, I see "looping" printed 3 times on the screen, then no more. I don't fully understand what is happening inside the select, but I'd assume it would only print 1 time.
I tried moving the inputready.. select.select() around to different places but found it appears to need to be called each time, otherwise the server stops responding (for example if it is called once prior to the endless while: loop).
I'm hoping this can be made simple enough that it can be taught to other hacker types in a maker class, so I'm hopeful I don't need to get too crazy with multi-threading and more elaborate solutions. As a last resort I'm considering logging all my parameters to mySQL from the external script then using this script to query them back out of tables. I've got experience there and would probably work, but it seems this socket angle would be a more direct solution.
Any help very much appreciated.
Great news. This was an easy fix, wanted to post in case anyone else needed it. The suggestion from acw1668 above got me going.
Simply added a timeout of "0" to the select.select() like this:
inputready,outputready,exceptready = select.select(input,[],[],0)
This is in the python docs but somehow I missed it. Link here: https://docs.python.org/2/library/select.html
Per the docs:
The optional timeout argument specifies a time-out as a floating point number in seconds. When the timeout argument is omitted the function blocks until at least one file descriptor is ready. A time-out value of zero specifies a poll and never blocks.
I tested the same code as above, adding a delay of 5 seconds using time.sleep(5) right after the print "looping" line. With the delay, if no data or client is present the code just loops every 5 seconds and prints "looping" to the screen. If I kick off the client script during the 5 second delay, it pauses and the message is processed the next time the 5 second delay ends. Occasionally it doesn't respond the very next loop, but rather the loop following. I assume this is because the first time through the server.accept is running and the next time through the s.recv() is running which actually exchanges the data.
I'm making a cmd IRC client in Python. I want to receive data at the same time I can write message, in the previous code I did I could only write 2 messages and then it bugs and I can't write until it receives some kind of data.
The question is, can I have one cmd window running the received data and other one with a constant input waiting for me to write something to send?, maybe with threads?
I've looked through the subprocess library but I don't really know how to code it.
CMD1:
while Connected:
print socket.recv(1024)
CMD2:
while Connected:
text = raw_input("Text to send>> ")
socket.send(text)
(This is a pseudocode not a real one)
This approach you are proposing could be done by making a server like application, and 2 client applications that connect via localhost to send and receive events. So that way you could have 2 terminals open , connected to the same session of the server.
On the other side you should consider a different design approach that include ncurses which allow you to make a terminal ui with input and output at the same terminal (two sections up and down). You can reference: http://gnosis.cx/publish/programming/charming_python_6.html
I have a python script (running on Mac OS X) that needs to be restarted when the internet goes down. If the internet is down, I would like to kill the current script, wait for the internet to go back up, and then restart it. Or, if possible, restart the function from within.
The problematic section of the Python code is as follows:
import tweetstream
# ...
with tweetstream.FilterStream(username, password, track = words) as stream:
for tweet in stream:
db.tweets.save(tweet)
Currently, if the internet goes down, the stream stops and doesn't reconnect.
It depends from os. There are few os specific methods.
First cross platform method will be own ping which will be send some packets to the Internet server. If you can not receive info that means Internet is goes down.
Try using this python implementation of ping as a subprocess. Thus, if too many timeouts occur, then you'll know the network's down and you can re-initiate the tweet process (however, to do this, you should probably put the entire tweeting process in a function of its own)
Perhaps you could try something like this:
import urllib2
def internet_on():
try:
response=urllib2.urlopen('http://74.125.131.94/',timeout=1)
return True
except urllib2.URLError as err: pass
return False
74.125.131.94 is the ip address of google.co.in . You can use whatever site you think will respond more quickly. Using a numerical IP-address avoids a DNS lookup, which may block the urllib2.urlopen call.The timeout=1 parameter makes sure that the call to urlopen will not take longer than 1 second even if the internet is not available.
Now you just need to call the internet_on() function. It will return true if the connection is on, else return false. Then you might want to wrap all the tweeting code inside a function and call it. (As #inspectorG4dget suggested).
EDIT:
For continuous checking you could do something like
def check():
while not internet_on():
pass
print "internet connection is on"
// call the tweet stuff function here.
Then when your stream stops just call the check() function and when the internet connection is back , it will call your tweet function to restart it.