Develop Reference

Socket python doesn't send data if program is alive

I'm trying to run a client/server script, where the client sends a file to the server and waits for responses until the server sends a stop message.
The problem is: once the connection is established the client starts sending data but until I press CTRL-C the server cannot recreate the file. Only after CTRL-C print "file is fully created" and the file becomes visible, instead, before it's seems to be waiting for something. idk where the problem is. Also tried changing condition on send loop using len(), but doesn't work. Anyone know how to fix it ?
client.py :
import socket # Import socket module
# from threading import Thread
s = socket.socket() # Create a socket object
HOST = "101.xx.x.xxx" # public IP address
PORT = 4243 # Reserve a port for your service.
PDF_PATH = "exam.pdf"
s.connect((HOST, PORT))
def send():
f = open(PDF_PATH, "rb")
while data := f.read(4096):
s.send(data)
f.close()
return
def receive():
while 1:
exercise = s.recv(4096)
if exercise == "stop!":
s.close()
break
f = open(f"{exercise}.txt", "wb")
while data := f.read(4096):
f.write(data)
return
def main():
send()
receive()
if __name__ == "__main__":
main()
server.py :
import socket
from threading import Thread
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
HOST = socket.gethostname()
IP = socket.gethostbyname(HOST)
PORT = 4243
s.bind(('', PORT))
s.listen(5)
def receive_file(conn, i):
f = open(f"exam.pdf", "wb")
while received := conn.recv(4096):
f.write(received)
print("File is fully copied\n")
f.close()
def send_result(conn,i):
while 1:
nbr = str(input("which exercise? "))
if nbr == "stop!":
break
f = open(f"exercise{nbr}.txt", "rb")
conn.send(bytes(f"exercise{nbr}.txt", encoding="utf-8"))
while data := f.read(4096):
conn.send(data)
f.close()
def main():
try:
while 1:
i = 0
conn, addr = s.accept()
print("Got connection from", addr)
# c.send(b"Thank you for connecting")
t = Thread(target=receive_file, args=(conn, i))
t.start()
t.join()
t = Thread(target=send_result, args=(conn, i))
t.start()
t.join()
except KeyboardInterrupt:
print("interrupting \n")
conn.close()
s.close()
if _name_ == '_main_':
main()

conn.recv() in the server won't return '' (no more data) unless the client closes the connection or calls shutdown(SHUT_WR) to indicate sends are complete:
def send():
with open(PDF_PATH, "rb") as f:
while data := f.read(4096):
s.sendall(data)
s.shutdown(socket.SHUT_WR)
An alternative is to design a protocol that sends the length of data before the data so you know when you've received the complete transmission. This would be required if you need to send more than one thing without closing the socket or shutting down sends. You're going to need this to make the receive portion of the server work if you want to send more than one exercise file.
Refer to this answer for an example of sending multiple files over a socket.

multiple console windows for one Python script

I've seen similar questions such as this one: keep multiple console windows open from batch.
However, I have a different situation. I do not want to run a different script in a different console window. My idea is to have socket running as a server and accepting all connections. When a connection is accepted, a new console window is created, and all in-coming and out-going data is shown there. Is that even possible?

A process can only be attached to one console (i.e. instance of conhost.exe) at a time, and a console with no attached processes automatically closes. You would need to spawn a child process with creationflags=CREATE_NEW_CONSOLE.
The following demo script requires Windows Python 3.3+. It spawns two worker processes and duplicates each socket connection into the worker via socket.share and socket.fromshare. The marshaled socket information is sent to the child's stdin over a pipe. After loading the socket connection, the pipe is closed and CONIN$ is opened as sys.stdin to read standard input from the console.
import sys
import time
import socket
import atexit
import threading
import subprocess
HOST = 'localhost'
PORT = 12345
def worker():
conn = socket.fromshare(sys.stdin.buffer.read())
sys.stdin = open('CONIN$', buffering=1)
while True:
msg = conn.recv(1024).decode('utf-8')
if not msg:
break
print(msg)
conn.sendall(b'ok')
input('press enter to quit')
return 0
def client(messages):
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
for msg in messages:
s.sendall(msg.encode('utf-8'))
response = s.recv(1024)
if response != b'ok':
break
time.sleep(1)
procs = []
def server():
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind((HOST, PORT))
s.listen(1)
while True:
conn, addr = s.accept()
with conn:
p = subprocess.Popen(
['python', sys.argv[0], '-worker'],
stdin=subprocess.PIPE, bufsize=0,
creationflags=subprocess.CREATE_NEW_CONSOLE)
p.stdin.write(conn.share(p.pid))
p.stdin.close()
procs.append(p)
def cleanup():
for p in procs:
if p.poll() is None:
p.terminate()
if __name__ == '__main__':
if '-worker' in sys.argv[1:]:
sys.exit(worker())
atexit.register(cleanup)
threading.Thread(target=server, daemon=True).start()
tcli = []
for msgs in (['spam', 'eggs'], ['foo', 'bar']):
t = threading.Thread(target=client, args=(msgs,))
t.start()
tcli.append(t)
for t in tcli:
t.join()
input('press enter to quit')

Will TCP Socket Server client connection fd cause memory leak?

I don't if i need to close the client socket handle( conn ) such as "conn.close()" ?
If I run multithread to handler the client socket fd ( conn ). Does it cause memory leak if the server runs too long time?
Will the server not close the client socket fd if client no invokes conn.close()?
Following is my tcp-socket server code:
# coding: utf-8
import socket
import os, os.path
import time
sockfile = "./communicate.sock"
if os.path.exists( sockfile ):
os.remove( sockfile )
print "Opening socket..."
server = socket.socket( socket.AF_UNIX, socket.SOCK_STREAM )
server.bind(sockfile)
server.listen(5)
print "Listening..."
while True:
conn, addr = server.accept()
print 'accepted connection'
while True:
data = conn.recv(1024)
if not data:
break
else:
print "-" * 20
print data
print "DONE" == data
if "DONE" == data:
# If I need to invoke conn.close() here?
break
print "-" * 20
print "Shutting down..."
server.close()
os.remove( sockfile )
print "Done"

According to the document, close is called when the socket is garbage collected. So if you didn't close it for whatever reason, your program would probably be fine. Provided your socket objects do get GCed.
However, as a standard practice, you must close the socket, or release whatever resource, when your code is done with it.
For managing socket objects in Python, check out
How to use socket in Python as a context manager?

One way to find out is to test it and see! Here is a little Python script that I ran on my Mac (OS X 10.11.5). If I un-comment the holdSockets.append() line, this script errors out ("socket.error: Too many open files") after creating 253 sockets. However, if I leave the holdSockets.append() line commented out, so that the sockets can be garbage collected, the script runs indefinitely without any errors.
#!/bin/python
import socket
import time
count = 0
holdSockets = []
while True:
count += 1
nextSock = socket.socket( socket.AF_UNIX, socket.SOCK_STREAM )
#holdSockets.append(nextSock)
time.sleep(0.1)
print "Created %i sockets" % count

Python select for blocking

I'm writting a server code to listen information from a socket, but when I press the letter P I have to send back to the client a "Message". Here is what I have done so far:
import select
import Queue
import socket
import time
import sys
import msvcrt
from msvcrt import getch
n = 0
HOST = ''
PORT = 2323
servsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
servsock.bind((HOST, PORT))
servsock.listen(5)
servsock.setblocking(0)
inputs = [servsock]
outputs = []
message_queues = {}
while 1:
(sread, swrite, serr) = select.select(inputs, outputs, []);
for sock in sread:
if sock is servsock:
(newsock, address) = servsock.accept()
newsock.setblocking(0)
print "New connection: ", address
inputs.append(newsock)
newsock.send("You are connected")
outputs.append(newsock)
else:
recv_msg = sock.recv(100)
if recv_msg == "":
(host, port) = sock.getpeername()
print "Client %s:%s disconnected" % (host,port)
sock.close()
inputs.remove(sock)
outputs.remove(sock)
else:
(host, port) = sock.getpeername()
print "Client %s:%s sent: %s "% (host,port,recv_msg)
for sock in swrite:
while msvcrt.kbhit():
if msvcrt.getch() == "p":
sock.send("Message")
But looks like the select reaches the swrite, enters there, but then it does not iterates anymore between sread and swrite. If I press "1" anytime, indeed it will send to the socket a "Message" but it never returns to the sread!
What I just need is to read for incomming messages all the time, and when I press the key P to send a "Message", without blocking the rest of the program.
Can you please help me?
Thanks

Why do you expect it to go into the read segment? What is it supposed to read that it isn't? In other words, how do you know it's not getting there when it's supposed to? If there are no new connections or data being sent to this, the select will never trigger for reading because there's no reading to do.
This code isn't good, though, as you will be spamming the swrite section because your sockets are (almost) always writeable. You should only be selecting for write when you want to write something. If you check your CPU, you will be almost guaranteed 100% utilization.
Ideally, you would find a library which would allow you to add keyboard input as a selectable object so you could look for keyboard hits in your select. However, I don't know if there is a way to do that in python.

You forgot the timeout on the select.select, so it is blocking there. Here's an example that waits for upto 100ms (0.1seconds):
(sread, swrite, serr) = select.select(inputs, outputs, [], 0.1);

Python terminate multiprocessing and gui process gracefully

I am using glade as my gui and creating a process to run my gui in. This app will open a socket when 'on' is clicked. When i press 'send', it will send whatever is in an textfield to the socket. The socket receives this data and sends it back. The problem is after i send data to the socket the thread doesn't terminate. Also after i close my gui it calls a sys.exit() but also leaves a process and doesn't terminate. I believe the error is in how i am implementing my processes or all my processing in general. Can anyone shine some light on this? It also relates to my last post as well. Thanks
main.py
// Main thread that create a new process for my gui and displays it
import socket, thread, gtk, Handler, sys, os, multiprocessing
sys.setrecursionlimit(10000)
if __name__ == '__main__':
builder = gtk.Builder()
#32bit template.glade 64bit template-2.22
# #todo add switching between architectures
#
builder.add_from_file("template/template-2.22.glade")
builder.connect_signals(Handler.Handler(builder))
window = builder.get_object("window1")
window.show_all()
try:
p = multiprocessing.Process(target=gtk.main())
p.start()
except:
print "Error Starting new Thread"
handler.py
// Handler for gtk glade signals, creates new threads and handles button and stuff
import thread, threading, os, server, client,multiprocessing, time
import sys, gtk
class Handler(object):
'''
classdocs
'''
myobject = ''
def __init__(self,object1):
#Getting glade builder
self.myobject = object1
'''
Constructor
'''
def clickme(self,value):
myserver = server.Server()
try:
p = multiprocessing.Process(target=myserver.run)
p.start()
except:
pass
def sendmessage(self,value):
text = self.myobject.get_object('entry1').get_text()
print text
msg = client.MyClass()
p = multiprocessing.Process(target=msg.run,args=([text]))
p.start()
server.py
// Opens a socket and listens for incoming data and sends it back
import socket,multiprocessing, gtk, sys
class Server:
'''
classdocs
'''
def __init__(self):
'''
Constructor
'''
def run(self):
try:
while 1:
HOST = 'localhost' # Symbolic name meaning the local host
PORT = 50006 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((HOST, PORT))
s.listen(5)
conn, addr = s.accept()
print 'Connected by', addr
while True:
data = conn.recv(1024)
if not data:
conn.close()
sys.exit()
break
elif data != '':
conn.sendall(data)
break
print "Closing"
#conn.close()
finally:
print "End"
pass
client.py
// Sends whatever is inside text area to socket
import time
class MyClass:
'''
classdocs
'''
def __init__(self):
'''
Constructor
'''
def run(self,text):
try:
import socket
HOST = 'localhost' # The localhost
PORT = 50006 # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.send(text)
data = s.recv(1024)
while 1:
if data != '':
print 'Received', repr(data)
break
finally:
pass

This is just wrong:
p = multiprocessing.Process(target=gtk.main())
p.start()
First, you can't start the gtk main loop in a subprocess, even if you did it rigth. Fortunately the process never really tries to start main as you call gtk.main(), which will block until the main loop exits and then return None. So what you're actually doing is:
gtk.main()
p = multiprocessing.Process(target=None)
p.start()
Througout the rest of your code you keep creating new processes and then forgetting about them. If you would keep a reference to them, you could at least try to send the TERM signal to them to shut them down (using Process.terminate, or set the daemon flag). If you want to shut down the subprocess cleanly, you either need to handle that signal in the subprocess, or use other IPC mechanisms to get it to shut down cleanly (like mutliprocessing.Event, ...).
Then there is this:
while True:
data = conn.recv(1024)
if not data:
conn.close()
sys.exit()
break
elif data != '':
conn.sendall(data)
break
This while loop will never loop (unless recv magically returns something else then a string). The first execution path ends with sys.exit() (taking the whole server down - the break is unreachable), the second ends with break, so the loop is useless.
A few lines below you have the exact opposite:
data = s.recv(1024)
while 1:
if data != '':
print 'Received', repr(data)
break
Unless data was '' in the first line, this will be an endless loop, as data's value won't change anymore.
Generally you don't really need multiprocessing for most of this. Starting a server in a different process may be ok if if has to do a lot of work, but spawing a subprocess just to send some data is overkill. Sending and receiving using sockets are IO bound, using threading here would be more reasonable.
You have two classes (Server and Handler) which have only two methods, one of which is __init__, and the other one is only used as target for a subprocess:
myserver = server.Server()
try:
p = multiprocessing.Process(target=myserver.run)
and:
msg = client.MyClass()
p = multiprocessing.Process(target=msg.run,args=([text]))
That's a sign that these shouldn't be classes but functions.