i use a Remote Procedure call to communicate between two prozesses. I send objects from the on hand to the other. the object is a object by a django model. The object has different variables, integers and strings.
If i change only integer variables everything works fine. if i change a string variable in the first time it works, too but if i change a string second time my code crashes and i get the following error message
Traceback (most recent call last):
File "/home/manch011/disserver/src/disserver/gui/backends/receiver.py", line 69, in run
name, args, kwargs = cPickle.load(connFile)
cPickle.UnpicklingError: pickle data was truncated
This is my code,
on the server-side:
_exportedMethods = {
'changes': signal_when_changes,
}
class ServerThread(QtCore.QThread):
def __init__(self):
super(ServerThread,self).__init__()
st = self
#threading.Thread.__init__(self)
def run(self):
HOST = '' # local host
PORT = 50000
SERVER_ADDRESS = HOST, PORT
# set up server socket
s = socket.socket()
s.bind(SERVER_ADDRESS)
s.listen(5)
while True:
conn, addr = s.accept()
connFile = conn.makefile()
name, args, kwargs = cPickle.load(connFile)
res = _exportedMethods[name](*args,**kwargs)
cPickle.dump(res,connFile) ; connFile.flush()
conn.close()
And this is the client-side:
class RemoteFunction(object):
def __init__(self,serverAddress,name):
self.serverAddress = serverAddress
self.name = name
def __call__(self,*args,**kwargs):
s = socket.socket()
s.connect(self.serverAddress)
f = s.makefile()
cPickle.dump((self.name,args,kwargs), f)
f.flush()
res = cPickle.load(f)
s.close()
return res
def machine_changed_signal(machine):
HOST = ''
PORT = 50000
SERVER_ADDRESS = HOST, PORT
advise = RemoteFunction(SERVER_ADDRESS,'changes')
advise(machine)
I am not familiar with cPickle and hence cannot figure this one out, can someone explain it to me?
Thanks in advance Chis
I solved my own problem. But first the error message I describe in my question is not meaningful.
I am the problem tackled new and have used the Pyro4 Framework. So i got a new error message which was equivalent to the old but clearly. U cant pickle class objects.
Because I need in my case only the attribute values I pass this in a simple dictionary.
At first download Pyro4 and install it
A simple example similar to the example on the Pyro homepage:
# saved as helloworld.py
import Pyro4
import threading
import os
class HelloWorld(object):
def get_hello_world(self, name):
return "HelloWorld,{0}.".format(name)
#The NameServer had to run in a own thread because he has his own eventloop
class NameServer(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
os.system("python -m Pyro4.naming")
ns = NameServer()
ns.start()
hello_world=HelloWorld()
daemon=Pyro4.Daemon() # make a Pyro daemon
ns=Pyro4.locateNS() # find the name server
uri=daemon.register(hello_world) # register the greeting object as a Pyro object
ns.register("example.helloworld", uri) # register the object with a name in the name server
print "Ready."
daemon.requestLoop() # start the event loop of the server to wait for calls
run this programm and execute the next after
# saved as client.py
import Pyro4
name=raw_input("What is your name? ").strip()
helloworld=Pyro4.Proxy("PYRONAME:example.helloworld") # use name server object lookup uri shortcut
print helloworld.get_hello_world(name)
Important u cant transfer class instances. So "name" could not be a class instance.
Related
As a part of security testing, I am developing a script which check the connected port in the host and return 0 if the status is true (if its connected to the port) or else it return some error code.. so I need to include that condition in the below code under the def getPort_on_host() where the result == should return the connected ports ..
Would be great if anyone can help me out here...
import socket
class A:
s= socket.socket(socket.AF_INET,socket.SOCK_STREAM)
def getAddress():
host= raw_input("Enter the Host name :" )
result = s.gethostbyname(host)
return result
def getPort_on_host():
portlist =[20,23,80,912,135,445,20]
for port in portlist:
result = s.connect_ex((getAddress,port))
if result ==""
print result
return port
s.close()
def getService():
status = s.getserverbyport(getPOrt_on_host)
print status
Couple of notes - watch your indentation. The levels for functions all have to be indented past the function or it won't work.
Same for conditional statements: for, if, else, etc. all need to have one indentation level below them. All these statements need
a colon after them as well.
Next, it seems like you might be trying to pass s as an instance variable (i.e. global to the class) which means you should
denote it with self.. If you continue to use the class structure you should also create an __init__ method.
You will also need to pass that as a parameter in all your methods. Also note that if you use
return in a Python function, it will not execute code below that. Here your s.close() is unreachable.
When passing a method, if you want to pass the method object itself, calling the method name is fine. If you want to
pass the value returned by the function you will need to add parenthesis to the method name, i.e. result = self.s.connect_ex((self.getAddress(),port))
I see that you are calling the gethostbyname on the socket object when you need to call that on the socket interface itself.
You should convert the hostname on a different variable, then call the connect_ex on the socket object after.
Be aware that in that for loop you are passing the function every single time as well. So it will ask for a host name each time.
Here is an updated version that gets you closer to where you need to be. Note that opening the host and testing each socket will take some
time for the host to be printed:
import socket
class A:
def __init__(self):
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
def getAddress(self):
newSocketobj = socket
host = raw_input("Enter the Host name :" )
result = newSocketobj.gethostbyname(host)
print result
return result
def getPort_on_host(self):
portlist =[20,23,80,912,135,445,20]
hostname = self.getAddress()
for port in portlist:
status = self.s.connect_ex((hostname,port))
print status
self.s.close()
#return port
def getService(self):
status = self.s.getserverbyport(self.getPort_on_host)
print status
newTest = A()
newTest.getPort_on_host()
I am working on UDP chat which should be listening and being able to send message any time using only one socket. Example, I will have the chat program done, I will open it first time, then second time and I must be able to communicate over UDP from both programs, simply each program has only one opened socket.
My two threads are for listening, which is deamon thread, because I want it to listen to new messages nonstop, and my other is sending the messages, which is just like a normal thread.
First of all, my problem is that it looks like my threads are blocking each other, because if I run the program, I only get output from the first thread I start.
Second problem is that I am not sure if my sending function or the entire class is written properly, or if there is something missing or incorrect.
Thanks in advance. Btw, I am new into python and I am using python 3, just to make it clear.
import socket
import threading
import logging
import time
from sys import byteorder
class Sending():
def __init__(self, name, tHost, tPort):
self.name = name
self.host = tHost
self.port = tPort
def set_name(self, name):
self.name = name
def send(self, name, tHost, tPort, msgType, dgramSize):
logging.debug('Starting send run')
message = input('Enter message: ')
data = bytearray()
data.extend( (name.encode('utf-8'), message.encode('utf-8'), msgType.to_bytes(1, byteorder = 'little')) )
#data.extend(message.encode(encoding='utf_8'))
self.sock.sendto(bytearray(data), (tHost, tPort))
def run(self):
th2 = threading.Thread(name = 'send', target=self.send('username', 'localhost', 8001, 1, 1400))
th2.start()
class Receiving():
def __init__(self, host, port):
self.host = host
self.port = port
def create_socket(self, host, port):
logging.debug('Starting socket')
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((host, port))
#print ('socket ready')
time.sleep(5)
while True:
data, addr = sock.recvfrom(1500)
print('Prijata:' + data + addr)
def run(self):
th1 = threading.Thread(name = 'rec', target=self.create_socket('localhost', 8000))
th1.setDaemon(True)
th1.start()
if __name__ == '__main__':
#print ('running')
rec = Receiving('localhost', 8000)
send = Sending('username', 'localhost', 8001)
send.run()
rec.run()
Congrats on your introduction to Python! It looks like you're using Python 3, and in future questions it's helpful if you are explicit about which version you're using because there are minor but program-breaking incompatibilities in some code (including this code!).
I found a few errors in your program:
The most major issue - as Trevor Barnwell says, you're not calling threading.Thread quite correctly. The target= argument needs to be a callable object (i.e. function), but in this case it should just be a reference to the function. If you add brackets to the function, self.create_socket(host, port) as you have above, it actually runs the function immediately. As Trevor explained, your Sending.send() method was called early, but additionally there was a similar bug in Receiving. Because Receiving.create_socket() creates an infinite loop, it never returns program execution. While the console output looks correct to the user, the actual program execution has never made it to running the listener in a separate thread.
bytearray.extend() takes an iterable of ints, what you're passing right now is a tuple of byte objects.
In Sending.send() you call self.sock, but you never assign self.sock a value, so it fails.
Sending.run() only runs Sending.send() one time. After completing input for the user, it immediately exits, because the program has finished.
If you're looking for an in-depth, project based introduction to Python appropriate for an experienced programmer (including an exercise very similar to this question on basic sockets, and another on threading), I highly recommend you check out Wesley Chun's "Core Python Applications Programming". The most recent edition (3rd) has a lot of Python 2 code, but it's easily portable to Python 3 with some minor work on the reader's part.
I tried to modify your code as little as possible to get it working, here it is:
import socket
import threading
import logging
import time
class Sending():
def __init__(self, name, tHost, tPort, target):
self.name = name
self.host = tHost
self.port = tPort
self.target_port = target
self.sock = self.create_socket()
def create_socket(self):
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((self.host, self.port))
return sock
def set_name(self, name):
self.name = name
def send_loop(self):
while True:
logging.debug('Starting send run')
message = input('Enter message: ')
data = bytearray()
data.extend(message.encode('utf-8'))
self.sock.sendto(bytearray(data), (self.host, self.target_port))
def run(self):
th2 = threading.Thread(name='send', target=self.send_loop)
th2.start()
class Receiving():
def __init__(self, host, port):
self.host = host
self.port = port
def create_socket(self):
logging.debug('Starting socket')
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((self.host, self.port))
print ('socket ready')
time.sleep(5)
while True:
data, addr = sock.recvfrom(1500)
print('\nPrijata:' + data.decode('utf-8') + str(addr))
def run(self):
th1 = threading.Thread(name='rec', target=self.create_socket)
print("Made it here")
th1.daemon = True
th1.start()
return
if __name__ == '__main__':
print('running')
rec = Receiving('localhost', 8000)
send = Sending('username', 'localhost', 8001, 8000)
rec.run()
send.run()
The threads are not blocking each other. send is called before a thread is even created.
th2 = threading.Thread(name = 'send', target=self.send('username', 'localhost', 8001, 1, 1400))
This line makes a call to send at:
self.send('username', 'localhost', 8001, 1, 1400)
I think you meant to do this:
th2 = threading.Thread(
target=self.send
args=('username', 'localhost', 8001, 1, 1400))
That way a thread will start that calls send on the next line.
Two other things:
You will want to loop in your functions because the thread terminates once the function does.
I think you mean raw_input instead of input
I have a problem with a Python script on my rpi. If I create a process object, it starts automatically and blocks everything else. I want it to run in the background, and to be able to start it by calling the start() method.
network_manager.py:
import socketserver
class NetworkManagerHandler(socketserver.StreamRequestHandler):
def handle(self):
print("Got some Data!")
class NetworkManagerServer(socketserver.ForkingMixIn, socketserver.TCPServer):
pass
core.py:
import multiprocessing
from network_manager import NetworkManagerServer, NetworkManagerHandler
HOST, PORT = "100.0.0.1", 11891
network_manager = NetworkManagerServer((HOST, PORT), NetworkManagerHandler)
network_manager_process =
multiprocessing.Process(target=network_manager.serve_forever())
# !-> Program is blocking here, but the Server is working. <-!
network_manager_process.daemon = True
network_manager_process.start()
print("Networkmanager is running. (%s:%s)" % (HOST, PORT))
# network_manager.shutdown()
Thanks.
This:
network_manager_process =
multiprocessing.Process(target=network_manager.serve_forever())
Should be this:
network_manager_process =
multiprocessing.Process(target=network_manager.serve_forever)
You don't actually want to call serve_forever, you just want to pass the function to the Process object.
Short Question
Using my examples below, is there a Pythonic way to share my_object's actual instance with with the BaseRequestHandler class?
Background
By definition, the BaseRequestHandler class creates a new instance for each request. Because of this, I am struggling to try find a solution on how to get data from the handle() function back to the ProtocolInterface instance. Note that this might be the wrong approach if I am needing to do something in handle() other than print to stdout.
At this point in time, I do not believe that global variables will work because my_object is passed in and is expected to change often (this is why handle() needs to see it. To see an example client (sending bogus data) see my other SO question. I think the biggest issue I am facing is the the socketservers are running in a background thread.
Example of what I would like to do
class ProtocolHandler(SocketServer.BaseRequestHandler):
def handle(self):
while(1):
self.data = self.request.recv(1024)
if self.data == '':
break
self.request.send("Success")
self.my_object.success = True# <--- How can I share my_object's instance?
class ProtocolInterface():
def __init__(self, obj, host='127.0.0.1', port=8000, single_connection=False):
self.my_object = obj # <--- This ideally is the same instance seen in ProtocolHandler
self.host = host
self.port = port
# Create the socket server to process in coming traffic
if(single_connection):
self.server = SocketServer.TCPServer((self.host, self.port), ProtocolHandler)
else:
self.server = SocketServer.ThreadingTCPServer((self.host, self.port), ProtocolHandler)
def start(self):
print "Server Starting on HOST: " + self.host
server_thread = threading.Thread(target=self.server.serve_forever)
server_thread.daemon = True
server_thread.start()
You could pass the object through the server instance:
self.server = SocketServer.TCPServer((self.host, self.port), ProtocolHandler)
self.server.my_object = self.my_object
The documentation indicates that you can have access to the server instance in handle() as self.server.
I've just started working with the basics of python socket networking. As an exercise in understanding, I've been trying to hash out a basic server that will ask it's client for a file type, and upon receiving a string of the extension, ask for the actual file. I've found numerous tutorials online that use the asyncore library, specifically asynchat to setup this kind of call and response functionality.
The most basic one I've been following can be found here (I've copied it)
http://effbot.org/librarybook/asynchat.htm
import asyncore, asynchat
import os, socket, string
PORT = 8000
class HTTPChannel(asynchat.async_chat):
def __init__(self, server, sock, addr):
asynchat.async_chat.__init__(self, sock)
self.set_terminator("\r\n")
self.request = None
self.data = ""
self.shutdown = 0
def collect_incoming_data(self, data):
self.data = self.data + data
def found_terminator(self):
if not self.request:
# got the request line
self.request = string.split(self.data, None, 2)
if len(self.request) != 3:
self.shutdown = 1
else:
self.push("HTTP/1.0 200 OK\r\n")
self.push("Content-type: text/html\r\n")
self.push("\r\n")
self.data = self.data + "\r\n"
self.set_terminator("\r\n\r\n") # look for end of headers
else:
# return payload.
self.push("<html><body><pre>\r\n")
self.push(self.data)
self.push("</pre></body></html>\r\n")
self.close_when_done()
class HTTPServer(asyncore.dispatcher):
def __init__(self, port):
asyncore.dispatcher.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
self.bind(("", port))
self.listen(5)
def handle_accept(self):
conn, addr = self.accept()
HTTPChannel(self, conn, addr)
#
# try it out
s = HTTPServer(PORT)
print "serving at port", PORT, "..."
My question has to do with the handle_accept method of the HTTPServer class. If every time a request comes in, the HTTPChannel object is initialized, wouldn't it be impossible in this kind of setup to create a call and response? I was thinking one could set flags for _hastype and _hasfile in the channel object, but since the accept inits it for each individual connection, the object's state is forgotten with every inidividual request. I realize this setup is supposed to be a basic HTTPServer, but my question is, how could I edit it to setup something like what I've described? Would the server object need to inherit asynchat itself and forego dispatcher completely? The channel object would have to have some state to know that the filetype has already been sent, and then ask for the binary of the file instead. I'm very curious to know what the cleanest possible implementation of this might look like.
Thanks a ton - I'm very new to sockets. Please let me know if I haven't been clear.
Normally the connection would be kept open after it's initially created, so all the parts of the communication from the same client go to the same HTTPChannel object - accept is only called when a new connection is created.