I don't if i need to close the client socket handle( conn ) such as "conn.close()" ?
If I run multithread to handler the client socket fd ( conn ). Does it cause memory leak if the server runs too long time?
Will the server not close the client socket fd if client no invokes conn.close()?
Following is my tcp-socket server code:
# coding: utf-8
import socket
import os, os.path
import time
sockfile = "./communicate.sock"
if os.path.exists( sockfile ):
os.remove( sockfile )
print "Opening socket..."
server = socket.socket( socket.AF_UNIX, socket.SOCK_STREAM )
server.bind(sockfile)
server.listen(5)
print "Listening..."
while True:
conn, addr = server.accept()
print 'accepted connection'
while True:
data = conn.recv(1024)
if not data:
break
else:
print "-" * 20
print data
print "DONE" == data
if "DONE" == data:
# If I need to invoke conn.close() here?
break
print "-" * 20
print "Shutting down..."
server.close()
os.remove( sockfile )
print "Done"
According to the document, close is called when the socket is garbage collected. So if you didn't close it for whatever reason, your program would probably be fine. Provided your socket objects do get GCed.
However, as a standard practice, you must close the socket, or release whatever resource, when your code is done with it.
For managing socket objects in Python, check out
How to use socket in Python as a context manager?
One way to find out is to test it and see! Here is a little Python script that I ran on my Mac (OS X 10.11.5). If I un-comment the holdSockets.append() line, this script errors out ("socket.error: Too many open files") after creating 253 sockets. However, if I leave the holdSockets.append() line commented out, so that the sockets can be garbage collected, the script runs indefinitely without any errors.
#!/bin/python
import socket
import time
count = 0
holdSockets = []
while True:
count += 1
nextSock = socket.socket( socket.AF_UNIX, socket.SOCK_STREAM )
#holdSockets.append(nextSock)
time.sleep(0.1)
print "Created %i sockets" % count
Related
I am new to networking programming and python.
I am trying to figure out how to run different jobs at the server side.
For example, I want one function to create connections for incoming clients but in the same time I can still do some administration work from the terminal.
My code is as below but it doesn't work:
Edited: it doesn't work means it will get stuck in the init_conn() function
Like:
starting up on localhost port 8887
Thread: 0 Connected with 127.0.0.1:48080
# waiting
I am looking into SocketServer framework but don't know how that works.
from thread import *
import socket
def init_conn():
thread_count =0
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 8887)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(10)
#now keep talking with the client
while 1:
#wait to accept a connection - blocking call
conn, addr = sock.accept()
print 'Thread: '+ str(thread_count) + ' Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread takes 1st argument as a function name to be run, second is the tuple of arguments to the function.
start_new_thread(clientthread ,(conn,))
thread_count +=1
sock.close()
def clientthread(conn):
# receive data from client and send back
def console():
print 'this is console'
option = raw_input('-v view clients')
if option == 'v':
print 'you press v'
def main():
start_new_thread( init_conn(),() )
start_new_thread( console(),() )
if __name__ == "__main__":
main()
Your problem is probably that you start the program, sometimes it prints "this is console" and then it ends.
The first bug is that you call the methods instead of passing the handle to start_new_thread. It must be:
start_new_thread( init_conn, () )
i.e. no () after the function name.
The program doesn't do much because start_new_thread() apparent starts a thread and then waits for it to stop. The documentation is pretty unclear. It's better to use the new threading module; See http://pymotw.com/2/threading/
def main():
t = threading.Thread( target=init_conn )
t.daemon = True
t.start()
console()
so the code will run until console() ends.
I suggest to split the server and the command line tool. Create a client which accepts commands from the command line and sends them to the server. That way, you can start the console from anywhere and you can keep the code for the two separate.
Seeing that you're new to python, have you tried taking a look at the threading module that comes with the standard library?
import threading
... #rest of your code
while conditions==True:
i = threading.Thread(target=init_conn)
c = threading.Thread(target=console)
i.start()
c.start()
Can't say I've done too much with networking programming with python, so I don't really have much to say in that manner, but at least this should get you started with adding multithreading to your project.
Using SocketServer you may implement a client/server system. The documentation gives small examples which may be useful for you. Here is an extended example from there:
server.py :
import SocketServer
import os
import logging
FORMAT = '[%(asctime)-15s] %(message)s'
logging.basicConfig(format=FORMAT, level=logging.DEBUG)
class MyServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
# By setting this we allow the server to re-bind to the address by
# setting SO_REUSEADDR, meaning you don't have to wait for
# timeouts when you kill the server and the sockets don't get
# closed down correctly.
allow_reuse_address = True
request_queue_size = 10
def __init__(self, port):
self.host = os.uname()[1]
self.port = port
SocketServer.TCPServer.__init__(self, (self.host,self.port), MyTCPHandler)
logging.info( "Server has been started on {h}:{p}".format(h=self.host,p=self.port) )
class MyTCPHandler(SocketServer.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
# max length is here 1024 chars
self.data = self.request.recv(1024).strip()
logging.info( "received: {d}".format(d=self.data) )
# here you may execute different functions according to the
# request string
# here: just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
PORT = 8887
if __name__ == "__main__":
# Create the server, binding to localhost on port 8887
#server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
server = MyServer( PORT )
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
server.serve_forever()
client.py
import socket
import sys
import logging
FORMAT = '[%(asctime)-15s] %(message)s'
logging.basicConfig(format=FORMAT, level=logging.DEBUG)
HOST, PORT = "workstation04", 8887
logging.info( "connect to server {h}:{p}".format(h=HOST,p=PORT ) )
# read command line
data = " ".join(sys.argv[1:])
# Create a socket (SOCK_STREAM means a TCP socket)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
# Connect to server and send data
sock.connect((HOST, PORT))
sock.sendall(data + "\n")
# Receive data from the server and shut down
received = sock.recv(1024)
finally:
sock.close()
logging.info( "Sent: {}".format(data) )
logging.info( "Received: {}".format(received) )
The output looks something like:
server side:
> python server.py
[2015-05-28 11:17:49,263] Server has been started on disasterarea:8887
[2015-05-28 11:17:50,972] received: my message
client side:
[2015-05-28 11:17:50,971] connect to server disasterarea:8887
[2015-05-28 11:17:50,972] Sent: my message
[2015-05-28 11:17:50,972] Received: MY MESSAGE
You can run several clients (from different consoles) in parallel. You may implement a request processor on the server side which processes the incoming requests and executes certain functions.
Alternatively, you may use the python module ParallelPython which executes python code locally on a multicore system or on a cluster and clusters. Check the http examples.
I had to force pip to install this module:
pip install --allow-external pp --allow-unverified pp pp
Hello stackoverflow users, so I have this problem where i am trying to code a web server but the script ends before I can even test if it works. So my question is how can I make the script so that it will keep running forever?
#import threading
#import socket
#import signal # Signal support (server shutdown on signal receive)
import multiprocessing
#import queue
def MasterProcessA():
import socket
import multiprocessing
import threading
HOST = '97.107.139.231' # Symbolic name, meaning all available interfaces
PORT = 8080 # Arbitrary non-privileged port
#print(PORT)
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#print 'Socket created'
#Bind socket to local host and port
#try:
socket.bind((HOST, PORT))
#except socket.error as msg:
##print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
#print 'Socket bind complete'
#Start listening on socket
socket.listen(100)
print "starting server"
def ConnProcessA():
print "thread step one"
Conn, Address = socket.accept()
t = threading.Thread(target=ConnectionProcessorA)
print "thread step two"
#t.daemon = True
t.start()
#print("A Got connection from:", Address)
DataRecived = Conn.recv(1024) #receive data from client
DataRecived = bytes.decode(DataRecived) #decode it to string
print DataRecived
Conn.send("HELLO World")
Conn.clouse()
ConnProcessA = threading.Thread(target=ConnProcessA)
#t.daemon = True
ConnProcessA.start()
MasterProcessA = multiprocessing.Process(target=MasterProcessA)
MasterProcessA.start()
There are several issues with your codes.
The thread doesn't run. You need to modify:
ConnProcessA.start()
ConnProcessA.join()
ConnectionProcessorA is not included in the codes you pasted. So I have to comment out these 2 lines:
t = threading.Thread(target=ConnectionProcessorA)
t.start()
Name shadowing. Refer to these lines:
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ConnProcessA = threading.Thread(target=ConnProcessA)
MasterProcessA = multiprocessing.Process(target=MasterProcessA)
You named instances with existing module/function names. Name shadowing is very dangerous. Just try to execute any of the 3 lines twice, you will see the error.
On the other side, it's ok to reuse a variable to hold different things, if straightforward enough. E.g., following two lines are close enough to avoid confusion:
DataRecived = Conn.recv(1024) #receive data from client
DataRecived = bytes.decode(DataRecived) #decode it to string
Seems your socket server is to continuously listen to a same port, then you probably need to add s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1). As said at the most bottom of the doc.
A less severe point, in Python coding convention, CapWords is usually used for class names ref. You used it for both function names and variable names. Better to keep your codes consistent.
I need to make a program that communicates within the same program back and forth between client and server, but after following the instructions on: http://woozle.org/~neale/papers/sockets.html it just keeps listening and I see nothing printed.
How do I enable basic client server functionality within the same file?
#!/usr/bin/python # This is server.py file
import socket # Import socket module
import random
import os
import time as t
#open socket
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = random.randint(0,65535) # Reserve a port for your service.
if os.fork() == 0:
#server
s.listen(1)
print 'about to listen'
while 1:
c = s.accept()
cli_sock, cli_addr = c
cli_sock.send("Hello to you! %s" % cli_addr)
elif os.fork() == 0:
t.sleep(1)
#client
print 'in here2'
s.bind((host, port)) # Bind to the port
s.connect((host,port))
s.send("Hello!\n")
print s.recv(8192)
s.close()
You're never going to hit your client code, as you enter an infinite loop right after starting the listener. For a toy example like this, you'll need to create 2 socket objects, one for the server and one for the client, then pingpong back and forth between them within your code; you can't use a serve forever style loop like you are here unless it runs in a parallel thread/process so you don't block execution of the main thread.
I have a Blender code which takes sets of data from a csv file and uses them to rotate a robot arm and a human model in the Game Engine. This code works fine, but now I want to send data across a wireless connection to Blender.
I have a server code set up in Blender (which runs on Python 3)
# Server Program
# Make sure the client is being run on the data generation computer
SERVER_LOOP = True
import socket
import sys
import json
import bge
cont = bge.logic.getCurrentController()
owner = cont.owner
print ('INFO: Starting up')
# Create a TCP/IP socket to listen on
print ('INFO: Creating TCP/IP Socket')
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Prevent from 'ADDRESS ALREADY IN USE' upon restart
print ('INFO: Housekeeping...')
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# Bind the socket to port 8081 on all interfaces
server_address = ('localhost', 8081)
print ('INFO: Binding and starting up on %s port %s' % server_address)
server.bind(server_address)
print ('INFO: Server bound')
def send_welcome(cont):
cont.send('SERVER: Welcome'.encode('utf8'))
# Listen for connectons for 5 seconds
server.listen(5)
# Connection is the SOCKET OBJECT for the connection
# Client_address is the connected peer(the client)
connection, client_address = server.accept()
print ('INFO: Connection from', connection.getpeername())
print ('INFO: Sending welcome msg')
send_welcome(connection)
print ()
while SERVER_LOOP:
# Receive data
try:
data = connection.recv(10000)
# Unless there's an error
except OSError:
print (connection)
# Decode the data into usable lists
if type(data) != type(''):
data = data.decode()
# If we want to end the client stream but keep the server running
if data=='end' or data=='End' or data=='END':
print ('INFO: Closing connection with ',connection.getpeername())
connection.shutdown(socket.SHUT_RD | socket.SHUT_WR)
print ()
connection.close()
connection, client_address = server.accept()
print ('INFO: Connection from', connection.getpeername())
print ('INFO: Sending welcome msg')
send_welcome(connection)
print ()
# If we want to stop running the server
elif data=='end server' or data=='End server' or data=='End Server':
print ()
print ('SERVER SHUT DOWN')
SERVER_LOOP = False
# Display when data is loaded back on the client side
else:
# gives feedback in server command line
data = json.loads(data)
owner['test'] = data
print ('CLIENT: %s' % data)
message = 'ping'
connection.send(('SERVER: %s' % message).encode('utf-8'))
print ('SERVER: %s' % message)
And the client code to run with it (this one runs on Python 2.7)
# Client Program
# Make sure the server is being run in Blender
import socket
import time
import json
print 'INFO: Creating Socket'
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ip_addr = raw_input('IP: ')
port_addr = raw_input('PORT: ')
# Type 'localhost' in the IP field
# Type '8081' in the PORT field
print 'INFO: Connecting to server'
s.settimeout(5) # Times out if 5 seconds without connecting to client
s.connect((ip_addr, int(port_addr)))
# Listen for welcome
data = s.recv(10000)
print data
print ''
while 1:
message = raw_input('CLIENT: ')
if message=='end' or message=='End' or message=='END':
print ''
print 'SHUTTING DOWN CLIENT, SERVER STILL RUNNING'
s.send(message)
break
elif message=='end server' or message=='End server' or message=='End Server':
print ''
print 'SHUTTING DOWN SERVER'
s.send(message)
break
else:
s.send(message)
data = s.recv(10000)
print data
print 'INFO: Closing socket'
s.close()
print 'INFO: Quitting'
Now, obviously this doesn't do the rotations; it's just a test script to make sure that the data transfer between the two works. And it does - in Blender's system console, the data is displayed just as I want it. However, I have a string debug property in Blender titled "test", which is supposed to display the current number just typed in the client, and it's not until I close the whole program down.
For example:
I run the server script in Blender
I run the client script in IDLE
I type in numbers on the client side
They appear in the system console on the server side, but they do NOT appear in the Game Engine
I close the server from the client side
Now, the last number I typed finally appears on the server side
So the problem is that Blender runs my script and then the Game Engine after it's done, but I want them to run concurrently.
Let me know if my explanation doesn't make sense; I can provide downloads to my stuff if need be.
I don't know if this is still a problem - you posted in February and it's now August, but I was just searching for the answer of a similar problem. Your problem is that Blender doesn't update its frames until a script has finished running. Your game is literally stuck on the first frame it plays because it starts a script as soon as that frame hits, and because of the nature of your script, never ends.
Currently, you use server.listen(5) to mean that it listens to five seconds, but the number 5 in that function refers to the backlog instead of the length of time [source]. socket.listen() will stall your game indefinitely (as far as I understand) just like an infinite loop would.
This may not be the answer you were looking for, but it's definitely an answer.
I'm trying to make a transparent proxy in python using the socket module. but for some reason it hangs on connect()ing the socket. here is the code i'm using:
from __future__ import division
import socket
import struct
#import mcpackets
import sys
import time
#CUSTOM SETTINGS
HOST="192.168.178.28"
PORT=25565
#END CUSTOM SETTINGS
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind(('',25565))
serversocket.listen(1)
print "waiting for client, press multiplayer and use 'localhost' as server"
clientsocket,address=serversocket.accept()
print "client connected from %s:%d"%address
serversocket.close()
print "connecting to '%s:%d'"%(HOST,PORT)
serversocket=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print "socket created."
serversocket.connect((HOST,PORT))#------------------------------ freezes here
print "socket connected."
serversocket.settimeout(0)
clientsocket.settimeout(0)
print "timeouts set."
print "now proxying."
#tdata=[]
try:
while(True):
dat=None
try:
dat=clientsocket.recv(4096)
except socket.timeout:
pass
if(dat!=None):
try:
serversocket.send(dat)
except socket.timeout:
pass
#vice versa
dat=None
try:
dat=serversocket.recv(4096)
except socket.timeout:
pass
if(dat!=None):
try:
clientsocket.send(dat)
except socket.timeout:
pass
except:
clientsocket.close()
#with open("data.log","w") as fid:
# fid.write(''.join(tdata))
raise
the problem doesn't lie in the network as connecting to the server directly works fine. any ideas on what's going wrong?
This is a part of TCP sockets implementation where the operating system refuses to allow a new socket connection after a socket with the same name has been disconnected recently.
In order to force this request, set the REUSEADDR socket option on your socket, before connecting it (for both of your server socket creations):
serversocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
This way after you close your first server socket, when you want to connect the new server socket (with the same host, port), OS would not refuse.
I'm having difficulty reproducing this as it doesn't appear to hang on Mac OS X or Windows 7 with Python 2.7. So without being able to reproduce I'm guessing there's a problem with reusing serversocket so soon after closing it on your OS. Closing a socket puts that socket into the TIME_WAIT state so it's not closed immediately. How long it takes to really close the socket is dependent on the OS and may be what's causing your problem.
Although people seem to recommend that you don't use it, you might look into using the SO_LINGER option to force the socket to close immediately.
For example:
l_onoff, l_linger = 1, 1 # send RST (hard reset the socket) after 1 second
serversocket.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER,
struct.pack('ii', l_onoff, l_linger))
# this should now complete after l_linger timeout
serversocket.close()