Background: So we were provided this script in class with the objective of opening random unprivileged ports on our Ubuntu VMs. He gave us two TCP examples and asked us to open two more additional TCP ports as well as two UDP ports. We were to accomplish this using the socket library and the Python programming language.
So I initially focused on the problem that he gave us. Using the python terminal this was the final script before I initially executed it knowing that the general concepts would open the ports for a connection on the Linux guest:
#!/usr/bin/env python
import signal
import socket
import sys
import time
import os
s1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s3 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s4 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s5 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s6 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
def tcp_server() :
TCP_IP = '127.0.0.1'
s1.bind((TCP_IP, 6005))
s2.bind((TCP_IP, 1123))
s3.bind((TCP_IP, 6009))
s4.bind((TCP_IP, 1124))
s5.bind((TCP_IP, 6006))
s6.bind((TCP_IP, 6007))
s1.listen(1)
s2.listen(2)
s3.listen(3)
s4.listen(4)
tcp_server()
def SigAlarmHandler(signal, frame) :
print("Received alarm TCP server is shutting down")
sys.exit(0)
signal.signal(signal.SIGALRM, SigAlarmHandler)
signal.alarm(int(sys.argv[1]))
while True :
pass
When I execute the script on the Ubuntu VM I receive the following error message:
Traceback (most recent call last):
signal.alarm(int(sys.argv[1]))
IndexError: list index out of range
So I dug and I found these two little nuggets of information.
signal.alarm Python Docs:
https://docs.python.org/2/library/signal.html
Unix man page alarm(2)
http://unixhelp.ed.ac.uk/CGI/man-cgi?alarm+2
Looking at the man page for the alarm it seems that it is expecting an int type so I am not confident an explicit data conversion is necessary. Although I'm also not confident in the total direction of the script. The professor just gave it to us for bonus. He said he would look at it, but I'm not sure when he will get back to me.
I'm thinking that bit of code is setup that if one of the ports opened is probed the script will terminate. Looking at the man page it seems that if an int greater than 0 is returned the alarm will generate. Triggering the termination of the script. Although with an IndexError and not knowing what index it is referring to I'm unsure of where to narrow in on to resolve the issue.
The most likely explanation is that you are not passing any command line arguments to your script, and that it's sys.argv[1] that is raising the IndexError. Nothing to do with signals or sockets.
Call your script using ./ScriptName.py 5 and it should work, the alarm will fire after 5 seconds, and your server should exit.
References in case you're not familiar with sys.argv:
https://docs.python.org/2/library/sys.html#sys.argv
http://www.pythonforbeginners.com/system/python-sys-argv
Related
i wrote a script in python for serial communication between my M5Stack Stick C (like raduino) and the raspberry pi.
all work fine. i can send "X","Y" or "Z" from raspberry py to the stick and he will reply the value (G-Force) back to the raspi! so far so good
Codes:
Python on raspy:
import serial
import time
import threading
ser = serial.Serial('/dev/rfcomm5') #init serial port
input_line = []#init input char array
def process_data(_data):
#called every time a sream is terminated by \n
#and the command string is ready to use
command = convert(_data)
print(command)
def convert(s): #convert the char list in a string
new = "" #init string to append all chars from char array
for x in s: # traverse in the string
new += str(x)
return new # return string
def processIncomingByte(inByte):#adding incoming chars to input_line
global input_line# globalize the input_line
if(inByte == '\n'):#if \n is incoming, end the chararray and release process data method
process_data(input_line)
input_line = []#reset input_line for next incoming string
elif(inByte == '\r'):
pass
else:#put all incoming chars in input_line
input_line.append(inByte)
while True:
while(ser.in_waiting > 0):#while some data is waiting to read....
processIncomingByte(ser.read())#.... process bytes whit method
ser.write(b'X\n')
time.sleep(0.5)
before the script work, i have to manually bind the m5Stak Stick-C over Blueman
to /dev/Rfcomm5. it work just fine over GUI or Console....
but now i would like to connect the stick via python to rfcomm5 (just by know the MAC adress, will be found in a config file later on...)
i startet to investigate a bit, but the more i research the more confused i am!!
i read some stuff over sockets and server-client aproaches. over a seperated script and so on....
i tested this code:
from bluetooth import *
target_name = "M5-Stick-C"
target_address = None
nearby_devices = discover_devices()
for address in nearby_devices:
if (target_name == lookup_name( address )):
target_address = address
break
if (target_address is not None):
print ("found target bluetooth device with address ", target_address)
else:
print ("could not find target bluetooth device nearby")
and indeed it found the device (just testing)!
but do i realy need to make a second script/process to connect to from my script?
is the the M5stack Stick-C the server? (i think so)
im so confused about all that stuff. i coded a lot, but never whit sockets, server-client stuff.
basically the communication (server/client?) works.
i just need to connect the device i found in the second script via macadress to rfcomm5 (or whatever rfcomm).
do i need a bluetooth socket? like in this example
https://gist.github.com/kevindoran/5428612
isnt the rfcomm the socket or am i wrong?
There are a number of layers that are used in the communication process and depending where you tap into that stack will depend what coding you need to do. The other complication is that BlueZ (the Bluetooth stack on linux) changed how it works over recent times leaving a lot of out of date information on the internet and easy for people to get confused.
With two Bluetooth devices, they need to establish a pairng. This is typically a one off provisioning step. This can be done with tools like Blueman or on the command line with bluetoothctl. Once you have a pairing established between your RPi and the M5Stack Stick, you shouldn't need to discover nearby devices again. Your script should just be able to connect if you tell it which device to connect to.
The M5Stack stick is advertising as having a Serial Port Profile (SPP). This is a layer on top of rfcomm.
There is a blog post about how this type of connection can be done with the standard Python3 installation: http://blog.kevindoran.co/bluetooth-programming-with-python-3/
My expectation is that you will only have to do the client.py on your RPi as the M5Stack Stick is the server. You will need to know its address and which port to connect on. Might be some trial and error on the port number (1 and 3 seem to be common).
Another library that I find helpful for SPP, is bluedot as it abstracts away some of the boilerplate code: https://bluedot.readthedocs.io/en/latest/btcommapi.html#bluetoothclient
So in summary, my recommendation is to use the standard Python Socket library or Bluedot. This will allow you to specify the address of the device you wish to connect to in your code and the underlying libraries will take care of making the connection and setting up the serial port (as long as you have already paired the two devices).
Example of what the above might look like with Bluedot
from bluedot.btcomm import BluetoothClient
from signal import pause
from time import sleep
# Callback to handle data
def data_received(data):
print(data)
sleep(0.5)
c.send("X\n")
# Make connection and establish serial connection
c = BluetoothClient("M5-Stick-C", data_received)
# Send initial requests
c.send("X\n")
# Cause the process to sleep until data received
pause()
Example using the Python socket library:
import socket
from time import sleep
# Device specific information
m5stick_addr = 'xx:xx:xx:xx:xx:xx'
port = 5 # This needs to match M5Stick setting
# Establish connection and setup serial communication
s = socket.socket(socket.AF_BLUETOOTH, socket.SOCK_STREAM, socket.BTPROTO_RFCOMM)
s.connect((m5stick_addr, port))
# Send and receive data
while True:
s.sendall(b'X\n')
data = s.recv(1024)
print(data)
sleep(0.5)
s.close()
I'm a beginner in the field of sockets and lately trying ti create a terminal chat app with that.I still have trouble understanding setblocking and select functions
"This is the code i have taken from a website i'm reading from and in the code if there is nothing in data, how does it mean that the socket has been disconnected and please also do explain what affect the setblocking in the server or the client does.I have read somewhere that setblocking allows to move on if the data has been fully not recieved,i'm not quite satisfied with the explaination.Please explain in simple words "
import select
import socket
import sys
import Queue
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setblocking(0)
server_address = ('localhost', 10000)
server.bind(server_address)
server.listen(5)
inputs = [ server ]
outputs = [ ]
message_queues = {}
while inputs:
readable, writable, exceptional = select.select(inputs, outputs, inputs)
for s in readable:
if s is server:
connection, client_address = s.accept()
connection.setblocking(0)
inputs.append(connection)
message_queues[connection] = Queue.Queue()
else:
data = s.recv(1024)
if data:
message_queues[s].put(data)
if s not in outputs:
outputs.append(s)
else:
if s in outputs:
outputs.remove(s)
inputs.remove(s)
s.close()
if there is nothing in data, how does it mean that the socket has been disconnected
The POSIX specification of recv() says:
Upon successful completion, recv() shall return the length of the message in bytes. If no messages are available to be
received and the peer has performed an orderly shutdown, recv() shall return 0. …
In the Python interface, return value 0 corresponds to a returned buffer of length 0, i. e. nothing in data.
what affect the setblocking in the server or the client does.
The setblocking(0) sets the socket to non-blocking, i. e. if e. g. the accept() or recv() cannot be completed immediately, the operation fails rather than blocks until complete. In the given code, this can hardly happen, since the operations are not tried before they are possible (due to the use of select()). However, the example is bad in that it includes output in the select() arguments, resulting in a busy loop since output is writable most of the time.
I have a simple network of two machines connected directly by cable (no switches, routers, or anything else). One of the machines is a radar, which continously multicasts image data. The other machine is a Windows PC, on which I want to receive that data.
For a first test, I have a simple Python script:
import socket
MULTICAST_GROUP = '239.0.17.8'
PORT = 6108
LOCAL_IF = '192.168.3.42'
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.settimeout(1.0)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((LOCAL_IF, PORT))
sock.setsockopt(
socket.IPPROTO_IP,
socket.IP_ADD_MEMBERSHIP,
socket.inet_aton(MULTICAST_GROUP) + socket.inet_aton(LOCAL_IF)
)
while True:
try:
data, address = sock.recvfrom(64*1024)
except socket.timeout:
print 'timeout'
else:
print address, len(data)
If I run this from IDLE, it works fine. But if I run it stand-alone (from the command prompt, or double-click in Explorer), it doesn't receive any data; it only prints 'timeout' once a second.
I've been looking at Wireshark output to try to find the difference, but I've found none. Same data arrives, same membership request is sent (the membership is sent twice actually; is that normal?).
The datagrams are quite large (29504 bytes); could that be a problem?
What could be the big difference between running the script within or without IDLE? How can I make it always work ?
As Michele d'Amico suspected, the problem was a mis-configured firewall. Shame on me for not discovering that myself.
User parses number of remote machines as a argument(python scriptName 3). During the execution script needs to connect and send data to this machines(in this case to 3 machines) in random order.
My code right now:
def createSocket(ip):
s = socket.socket()
s.connect((ip, 55555))
def sendData(data):
s.send(data)
def closeSocket():
s.close()
createSocket(ip)
sendData(data)
closeSocket()
So I'm using one socket and re-connect each time I need to connect to other machine. Because of that script transmits data really slowly.
Can I somehow assign required number of sockets during execution and use them? Or maybe there is a better way of keeping connection to all machines?
Don't create a single global socket, instead keep a list of sockets. Don't close your sockets after each use, and then reopen them: just keep them open.
eg.
def createSocket(ip): # return the new socket object
s = socket.socket()
s.connect((ip, 55555))
return s
addresses=[ip, ip2, ip3, ...]
sockets=[createSocket(addr) for addr in addresses]
sock = chooseSocket(sockets) # pick one (somehow)
sock.send(data) # use the selected socket
I have client-server architecture build in python, unfortunately the original design was made that each request to server is represented by one TCP connection and I have to send requests in large groups (20 000+) and sometimes there occurs socket error #10055.
I've already found out how to handle it in python:
>>> errno.errorcode[10055]
'WSAENOBUFS'
>>> errno.WSAENOBUFS
10055
And build a code that is able to handle that error and reconnect (of course with little time delay to give server time to do whatever it has to do):
class MyConnect:
# __init__ and send are not important here
def __enter__(self):
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Try several reconnects
for i in range(0,100):
try:
self.sock.connect((self.address, self.port))
break
except socket.error as e:
if e.errno == errno.WSAENOBUFS:
time.sleep(1)
else:
raise
return self
def __exit__(self, type, value, traceback):
self.sock.close()
# Pseudocode
for i in range(0,20000):
with MyConnect(ip,port) as c:
c.send(i)
My questions are:
is there any "good practice" way to do this?
is e.errno == errno.WSAENOBUFS multi-platform? If not so, how to make it multi-platform?
Note: I've tested in just on Windows yet, I need it to work on Linux too.
You are clogging your TCP stack with outgoing data and all the connection establishment and termination packets.
If you have to stick to this design, then force each connection to linger until its data has been successfully sent. That is to say, that by default, close() on the socket returns immediately and further delivery attempts and connection tear-down happen "in the background". You can see that doing so over 20000+ times in a tight loop can easily overwhelm the OS network stack.
The following will force your socket close() to hang on for up to 10 seconds trying to deliver the data:
import struct
s.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack('ii', 1, 10))
Note that this is not the same as Python socket.sendall() - that one just passes all the bytes to the kernel .
Hope this helps.