I want to connect to a detector and receive image data. After working with Wireshark software, I realized that I should use protocol UDP. I wrote the following code in Python and got the data.
The received data sample is as follows:
import socket
UDP_IP = "1.1.1.1" #IP
UDP_PORT = 123 #port
sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
sock.bind((UDP_IP,UDP_PORT))
while True:
data, addr = sock.recvfrom(2048)
print("recived message: %s" % data)
Output:
data1 =b'\x83\xa4A\xd2\x01\x00\x00\x00\x12\x05\xf3\x04.\x04\xb7\x04#\x05Y\x04\x9f\x05\xf4\x046\x05\x89\x04\x1b\x08\x9b\x05^\x05\xb7\x04G\x05\x04\x04\xc7\x05\\\x05\xc8\x04$\x06:\x05\xf1\x05\x90\x06\x1f\x05w\x05\x8d\x05\x15\x05P\x06\x19\x05\x8a\x06e\x06\xfd\x05w\x06f\x05\x1f\x05!\x06\xed\x04\xb8\x06\xeb\x05\xd9\x05\x13\x06{\x05\x01\x07i\x07\xed\x07f\x08\x9c\x08\x80\x08\\\x08A\x08w\x08y\x089\x08<\x08\x11\x08m\x08\x1f\x08M\x07$\x08\xc5\x07\xe5\x08\xd9\x08q\x07\x7f\x08^\x08i\x08S\x08\xa7\x07.\t\x15\x08\x1d\x08\xae\x08\x8e\x07E\x08\xdc\x07\xf3\x061\t\xcd\x07\xe8\x08Y\x08\x00\x08\xb5\x08\xa3\x08\t\tt\x08q\x08w\t\r\t\xbd\x08p\x08\xfb\x08\xc5\x08_\x08\xe4\x07\n\x08/\x08\xf4\x07\xdc\x07#\x08\x99\x08\xf3\x07\xc5\x07Y\t\r\x08\xdd\x08\x91\x08\x91\x07\x91\x07?\x08\xd8\x08J\x08_\x08E\x08\xf9\x07\x1f\x08L\x08\xff\x07\xd7\x07M\x08<\x08\xdf\x08\x89\x075\tw\x08\x89\x08\x15\t......................
But I don't know how to decode the data. Wireshark itself does this. But I don't know how to decode with Python.
7c 10 c9 20 e3 31 da 55 aa 55 aa 01 08 00 45 00 05 dc 00 00 00 00 80 11 a9 97 c0 a8 05 15 c0 a8 05 14 15 be 15 be 05 c8 00 00
I use the following code, but it returns meaningless data.
data.decode("utf-16")
My first question is, how can I get from the primary data to the data that Wireshark decodes with Python?
And my second question is: How should I get from this data to the data I want, which are approximately numbers between 1000 and 10000? A number for each pixel, which is the sequence of received data, in the row and column of forming an image.
Related
As I've posted about before, I am writing a MIDI parser in Python. I am encountering an error where my parser is getting stuck because it's trying to read an event called 2a, but such an event does not exist. below is an excerpt from the MIDI file in question:
5d7f 00b5 5d7f 00b6 5d7f 00b1 5d00 00b9
5d00 8356 9923 7f00 2a44 0192 367f 0091
237f 0099 4640 0092 2f7c 0099 3f53 0b3f
I have parsed the file by hand, and I am getting stuck in the same spot as my parser! The MIDI file plays, so I know it's valid, but I'm certain that I am reading the events wrong.
The Standard MIDI Files 1.0 specification says:
Running status is used: status bytes of MIDI channel messages may be omitted if the preceding event is a MIDI channel message with the same status. The first event in each MTrk chunk must specify status. Delta-time is not considered an event itself: it is an integral part of the syntax for an MTrk event. Notice that running status occurs across delta-times.
Your excerpt would be decoded as follows:
delta <- event ------->
time status parameters
----- ------ ----------
... 5d 7f
00 b5 5d 7f
00 b6 5d 7f
00 b1 5d 00
00 b9 5d 00
83 56 99 23 7f
00 2a 44
01 92 36 7f
00 91 23 7f
00 99 46 40
00 92 2f 7c
00 99 3f 53
0b 3f ...
I have written a socket packet sniffer in Python using this code.
import socket, struct
# Setup socket object
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_UDP)
s.bind((socket.gethostbyname(socket.gethostname()), 0))
s.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
Id = 0
while (True):
data = s.recvfrom(65565)
packet = data[0]
address = data[1]
header = struct.unpack("!BBHHHBBHBBBBBBBB", packet[:20])
if (header[6] == 6):
protocol = "TCP"
elif (header[6] == 17):
protocol = "UDP"
print("Number: ", Id, end="\t")
print("Protocol: ", protocol, end="\t")
print("Address: ", address, end="")
print("Header: ", header)
#print("Data: ", data)
print()
Id += 1
I know I can find the receivers IP address at index 0 of the data variable (data[0]), but where in the packet would i find the senders IP address and port number?
What do all the integers in the header tuple mean? I know header[6] is the protocol TCP/UDP, but what about the rest?
Header: (69, 0, 61, 1541, 0, 128, 17, 0, 192, 168, 56, 1, 255, 255, 255, 255)
The IP header has a well-defined structure and an RFC. You decided to unpack it with bytes and shorts. Generally speaking, you want to match the size of each field with the correct data type. Taking a look at the proper header size, you can see the individual breakdown:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| IHL |Type of Service| Total Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identification |Flags| Fragment Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Time to Live | Protocol | Header Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Destination Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Options | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
This can be done programmatically as:
struct.unpack('!BBHHHBBHII')
# alternatively:
struct.unpack('!BBHHHBBH4s4s')
I prefer the latter because you can fairly easily convert it to an IP address:
Example Ethernet header:
0000 3c 4a 92 1f 04 00 74 c6 3b 8d 82 69 08 00 45 00 <J....t.;..i..E.
0010 00 3c 0a 24 40 00 40 06 23 76 0a 14 01 0d 01 01 .<.$#.#.#v......
0020 01 01 e7 7a 04 d2 a2 5e 0c d2 00 00 00 00 a0 02 ...z...^........
0030 72 10 0d 51 00 00 02 04 05 b4 04 02 08 0a 9c d4 r..Q............
0040 c0 c0 00 00 00 00 01 03 03 07 ..........
the 20-byte IPv4 header exists at packet[14:34]. Unpacking it with the above format yields this:
>>> header = struct.unpack('!BBHHHBBH4s4s', packet[14:34])
>>> header
(69, 0, 60, 2596, 16384, 64, 6, 9078, b'\n\x14\x01\r', b'\x01\x01\x01\x01')
Indices 8 and 9 are the source and destination IP's respectively. Because they are bytes, we can convert them to int and then str to get the IP in string format:
# Note: Python 3 only
>>> ip_src = '.'.join(map(str, header[8])) # 10.20.1.13
>>> ip_dst = '.'.join(map(str, header[9])) # 1.1.1.1
So I'm working with incoming audio from Watson Text to Speech. I want to play the sound immediately when data arrives to Python with a websocket from nodeJS.
This is a example of data I'm sending with the websocket:
<Buffer e3 f8 28 f9 fa f9 5d fb 6c fc a6 fd 12 ff b3 00 b8 02 93 04 42 06 5b 07 e4 07 af 08 18 0a 95 0b 01 0d a2 0e a4 10 d7 12 f4 12 84 12 39 13 b0 12 3b 13 ... >
So the data arrives as a hex bytestream and I try to convert it to something that Sounddevice can read/play. (See documentation: The types 'float32', 'int32', 'int16', 'int8' and 'uint8' can be used for all streams and functions.) But how can I convert this?
I already tried something, but when I run my code I only hear some noise, nothing recognizable.
Here you can read some parts of my code:
def onMessage(self, payload, isBinary):
a = payload.encode('hex')
queue.put(a)
After I receive the bytesstream and convert to hex, I try to send the incoming bytestream to Sounddevice:
def stream_audio():
with sd.OutputStream(channels=1, samplerate=24000, dtype='int16', callback=callback):
sd.sleep(int(20 * 1000))
def callback(outdata, frames, time, status):
global reststuff, i, string
LENGTH = frames
while len(reststuff) < LENGTH:
a = queue.get()
reststuff += a
returnstring = reststuff[:LENGTH]
reststuff = reststuff[LENGTH:]
for char in returnstring:
i += 1
string += char
if i % 2 == 0:
print string
outdata[:] = int(string, 16)
string = ""
look at your stream of data:
e3 f8 28 f9 fa f9 5d fb 6c fc a6 fd 12 ff b3 00
b8 02 93 04 42 06 5b 07 e4 07 af 08 18 0a 95 0b
01 0d a2 0e a4 10 d7 12 f4 12 84 12 39 13 b0 12
3b 13
you see here that every two bytes the second one is starting with e/f/0/1 which means near zero (in two's complement).
So that's your most significant bytes, so your stream is little-endian!
you should consider that in your conversion.
If I have more data I would have tested but this is worth some miliseconds!
Consider the below string which will be given as the input to a function.
01 02 01 0D A1 D6 72 02 00 01 00 00 00 00 53 73 F2
The highlighted part is the address I need.
If the preceding byte is 1 then I have to take only 6 octet and assign it to a variable.
If it is more than 1 the I should read 6 * Num(preceding value) and assign 6 octets for each variable.
Currently I am assigning it statically.
def main(line_input):
Device = ' '.join(line_input[9:3:-1])
Length = line_input[2]
var1 = line_input[3]
main("01 02 02 0D A1 D6 72 02 00 01 00 00 00 00 53 73 F2")
Can this be done?
Here I think this does it, let me know if there is anything that needs changing:
import string
def address_extract(line_input):
line_input = string.split(line_input, ' ')
length = 6 * int(line_input[2])
device_list = []
for x in range(3, 3+length, 6):
if x+6 > len(line_input):
print "Length multiplier too long for input string"
else:
device_list.append(' '.join(line_input[x:x+6]))
return device_list
print address_extract("01 02 02 0D A1 D6 72 02 00 01 00 00 00 00 53 73 F2")
#output = ['0D A1 D6 72 02 00', '01 00 00 00 00 53']
Here is some code that I hope will help you. I tried to add many comments to explain what is happening
import binascii
import struct
#note python 3 behaves differently and won't work with this code (personnaly I find it easyer for strings convertion to bytes)
def main(line_input):
formated_line = line_input.split(" ") #I start by cutting the input on each space character
print formated_line #the output is a list. Each element is composed of 2 chars
formated_line = [binascii.unhexlify(xx) for xx in formated_line] #create a list composed of unhelified bytes of each elements of the original list
print formated_line #the output is a list of bytes char
#can be done in one step but I try to be clearer as you are nee to python (moereover this is easyer in python-3.x)
formated_line = map(ord, formated_line) #convert to a list of int (this is not needed in python 3)
print formated_line
Length = formated_line[2] #this is an int
unformated_var1 = formated_line[3:3+(6*length)] #keep only interesting data
#now you can format your address
main("01 02 02 0D A1 D6 72 02 00 01 00 00 00 00 53 73 F2")
#if the input comes from a machine and not a human, they could exchange 17bytes instead of (17x3)characters
#main("\x01\x02\x02\x0D\xA1\xD6\x72\x02\x00\x01\x00\x00\x00\x00\x53\x73\xF2")
#then the parsing could be done with struct.unpack
In this question: https://serverfault.com/questions/173187/what-does-a-dns-request-look-like there is an example UDP packet that can be used to query a DNS server for the address of www.google.com.
I wrote a basic Python snippet:
from socket import *
s = socket(AF_INET, SOCK_DGRAM)
raw = "24 1a 01 00 00 01 00 00 00 00 00 00 03 77 77 77 06 67 6f 6f 67 6c 65 03 63 6f 6d 00 00 01 00 01"
hex = raw.split()
ints = [int(i, 16) for i in hex]
bytes = [chr(i) for i in ints]
packet = "".join(bytes)
s.sendto(packet, ("8.8.8.8", 53))
s.recvfrom(1024)
But nothing happens. I get no response. What am I doing wrong? Am I missing something? Shouldn't I receive an UDP packet on the same port with the answer to my query?