Develop Reference

Generating a UDP message in python with a header and payload in python3

I am new to Networking and trying to implement a network calculator using python3 where the client's responsibility is to send operands and operators and the server will calculate the result and send it back to the client. Communication is through UDP messages and I am working on client side. Each message is comprised of a header and a payload and they are described as shown in the below figures.
UDP header:
I am familiar with sending string messages using sockets but having a hard-time with how to make a message with both header and payload and how to assign the bits for various attributes or how to generate message/client id's in the header and If there is any way to automatically generate the Id's. Any help or suggestions will be highly appreciated.
Thanks in advance

I will only do a portion of your homework.
I hope it will help you to find energy to work on missing parts.
import struct
import socket
CPROTO_ECODE_REQUEST, CPROTO_ECODE_SUCCESS, CPROTO_ECODE_FAIL = (0,1,2)
ver = 1 # version of protocol
mid = 0 # initial value
cid = 99 # client Id (arbitrary)
sock = socket.socket( ...) # to be customized
def sendRecv( num1, op, num2):
global mid
ocs = ("+", "-", "*", "/").index( op)
byte0 = ver + (ocs << 3) + (CPROTO_ECODE_REQUEST << 6)
hdr = struct.pack( "!BBH", byte0, mid, cid)
parts1 = (b'0000' + num1.encode() + b'0000').split(b'.')
parts2 = (b'0000' + num2.encode() + b'0000').split(b'.')
msg = hdr + parts1[0][-4:] + parts1[1][:4] + parts2[0][-4:] + parts2[1][:4]
socket.send( msg) # send request
bufr = socket.recv( 512) # get answer
# to do:
# complete socket_send and socket.recv
# unpack bufr into: verr,ecr,opr,value_i, value_f
# verify that verr, ecr, opr, are appropriate
# combine value_i and value_f into answer
mid += 1
return answer
result = sendRecv( '2.47', '+', '46.234')
There are many elements that haven't be specified by your teacher:
what should be the byte-ordering on the network (bigEndian or littleEndian)? The above example suppose it's bigEndian but you can easily modify the 'pack' statement to use littleEndian.
What should the program do if the received packet header is invalid?
What should the program do if there's no answer from server?
Payload: how should we interpret "4 most significant digits of fraction"? Does that mean that the value is in ASCII? That's not specified.
Payload: assuming the fraction is in ASCII, should it be right-justified or left-justified in the packet?
Payload: same question for integer portion.
Payload: if the values are in binary, are they signed or unsigned. It will have an affect on the unpacking statement.
In the program above, I assumed that:
values are positive and in ASCII (without sign)
integer portion is right-justified
fractional portion is left justified
Have fun!

Adding padding to packet using scapy

how to inject zeros to the end of the UDP segment’s header to make it equal to 20 bytes.
Can anyone help me?
this my code :
if UDP in packet:
"""get layers after udp"""
layer_after = packet[UDP].payload.copy()
"""build a padding layer"""
pad = Padding()
pad.load = '\x00' * 12
layer_before = packet.copy()
layer_before[UDP].remove_payload()
packet = layer_before / pad / layer_after
and this is a output for a udp packet :
before : b'\xb8\xaco6\x1c\xa2\xe8\xe72<eP\x08\x00E\x00\x00.\x0bT#\x00\xec\x11\x18~MH\xa9\x82\x83\xca\xf0W+iC8\x00\x1a\x85+H\x00Y\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
--------------------------------------------------------------
after : b'\xb8\xaco6\x1c\xa2\xe8\xe72<eP\x08\x00E\x00\x00.\x0bT#\x00\xec\x11\x18~MH\xa9\x82\x83\xca\xf0W+iC8\x00\x1a\x85+H\x00Y\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
but it adds zeros to the end of packets!!!!!

I have the same question with yours.Maybe you can try this:
if UDP in packet:
layer_after = packet[UDP].payload.copy()
pad = Padding()
pad.load = '\x00' * 12
layer_before = packet.copy()
layer_before[UDP].remove_payload()
packet = layer_before / raw(pad) / layer_after
The padding will follow the end of the UDP segment’s header.I hope this would help you.

How to get the size of struct in python if the data inside struct has variable length binary string?

This code will read data from block device and pack data using struct.pack() and again unpack data using struct.unpack(). This is part of my main socket program. But I am facing an issue in calculating struct size. So I am putting a small code here to demonstrate my problem.
import sys,os
import struct as st
dev_read_data = os.open("/dev/sdc1",os.O_RDONLY)
buffer_size = 230400
offset = 0
while True:
data = os.pread(dev_read_data,buffer_size,offset)
packed_data = st.pack("%dsi" %(len(data)),data,offset) # This statement packs data and assign it to packed_data. Till here code works fine.
print ('Packed_Data {]'.format(packed_data))
unpacked_data = st.unpack("%dsi" %(len(data)),packed_data) # This unpacks data successfully.
offset += buffer_size
if buffer_size == 10036977152:
break
Now I want to calculate the size of struct using function:
struct.calcsize(format)
But in this case only one parameter can be passed. So how to get struct size in case of variable length binary string?
I will be very thankful if experts can find some time to answer my query.

like this :
import os
import binascii
import zlib
path = "/dev/sdc1"
stat = os.stat(path)
block_SIZE = stat.st_blksize
block_COUNT = os.statvfs(path).f_blocks
image = file(path,"rb")
indicator = 0
while True :
try :
if indicator > 2 : break #if indicator > block_Count : break
image.seek(indicator*block_SIZE)
data = image.read(block_SIZE)
HEX_CRC32 = binascii.unhexlify(str(hex(zlib.crc32(data) & 0xffffffff))[2:])
header = binascii.unhexlify(("%010X" % indicator)+("%04x"%block_SIZE))
""" NOW SEND TO SOCKET
My_socket.write(header+data+HEX_CRC32)
check Frame number : first 5 byte
Data Length : 2 Byte (block_SIZE)
Data : Data content
CRC32 : 2 Byte (32-bit value)
Client side : Check CRC32 , get `data[7:-2]` for current block
"""
indicator += 1
except Exception,e :
"""
very important (if failed,re send packet )
"""
print e
if indicator > block_Count : break
Not only brake ! Tell to client all packet has ben succesfully send, please close to socket on client side.

What is the difference between these two sections of code

Hi guys I have two sections of code, what is confusing me, is although they should be equivalent I am not receiving the same results when running both of these code, what is the difference between them ?
Here is the first and working section :
packet = struct.pack(">BHHLH", relayCmd, 0, streamId, 0, len(payload)) + payload
and second non working section :
# packet = struct.pack(">B", relayCmd)
# packet += struct.pack("H", 0)
# packet += struct.pack("H", streamId)
# packet += struct.pack("L", 0)
# packet += struct.pack("H", len(payload))
# packet += payload

In the first version you specify the format to Big Endian by ">" and then all of the formats parameters are encoded this way. In the second example you specify the Big Endian only in the first line and then all of the other parameters are encoded using native encoding of the system ("#" is used as default).

From the struct documentation:
Note: By default, the result of packing a given C struct includes pad bytes in order to maintain proper alignment for the C types involved; similarly, alignment is taken into account when unpacking. This behavior is chosen so that the bytes of a packed struct correspond exactly to the layout in memory of the corresponding C struct. To handle platform-independent data formats or omit implicit pad bytes, use standard size and alignment instead of native size and alignment: see Byte Order, Size, and Alignment for details.
You used the default # alignment (using native alignment) when you didn't specify the alignment for the 4 additional lines. You only used > standard alignment for the first relayCmd codepoint.
As a result, the sizes produced are different:
>>> import struct
>>> struct.calcsize('>BHHLH')
11
>>> struct.calcsize('>B')
1
>>> struct.calcsize('H')
2
>>> struct.calcsize('L')
8
>>> 1 + 3 * 2 + 8
15
The difference is in the padded L; if you use the > big endian marker for all pack() calls it only takes four bytes:
>>> struct.calcsize('>L')
4
So this works:
packet = struct.pack(">B", relayCmd)
packet += struct.pack(">H", 0)
packet += struct.pack(">H", streamId)
packet += struct.pack(">L", 0)
packet += struct.pack(">H", len(payload))
packet += payload

You have to prepend > to each letter so everything is big-endian.
#!/usr/bin/env python2
import struct
relayCmd = 170
streamId = 10000
payload = "A"
packet = struct.pack(">BHHLH", relayCmd, 0, streamId, 0, len(payload)) + payload
print(''.join("{:02x} ".format(ord(i)) for i in packet))
packet = struct.pack(">B", relayCmd)
packet += struct.pack(">H", 0)
packet += struct.pack(">H", streamId)
packet += struct.pack(">L", 0)
packet += struct.pack(">H", len(payload))
packet += payload
print(''.join("{:02x} ".format(ord(i)) for i in packet))

python script to read tcp, apply math, format string, and write file

absolute newbie to python after years of C/C++. I want to write a Python script to tell my Rasberry Pi to read a smart relay board, calculate a temperature, and write a formatted string to a file. After much googling and newsgroup searching, I may have most of it:
import socket
// open TCP client socket:
IPADDR = '192.168.99.9'
PORTNUM = 2101
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((IPADDR, PORTNUM))
// send command to read A/D value: two hex bytes
tdata = bytes.fromhex('FEA4')
s.send(tdata)
// receive 2-byte (hex) reply
rdata=s.recv(256)
// close socket
s.close()
// convert reply to a voltage reference (unsigned short)
vRef = (rdata[0] * 256)+(rdata[1])
// convert vref to float as degrees Farenheit
degF = vRef * 4930 / 1024
degF = degF / 10
degF = degF - 273.15
degF = degF * 9 / 5 + 32
// open text file
fo = open("\mnt\stuff\temp.txt", "w")
// write formatted string as number only e.g., 32.6
fo.write("{:+.1f}.format(degF)\n")
// Close file
fo.close()
I'm not sure about accessing the received data and creating the unsigned short value. I will receive something like /x02/x55, which is (2*256)+85 = 597.
The floating-point math, not sure here either, but that's how I convert a reading of 597 to a degrees-F value of 57.6
Finally, I need to write the string "57.6" to a file.
Lastly, and not in this code, I will need a way to have the RasPi run this code once per minute to update the file. I have a web server that reads the file and creates HTML text from that.
thanks ANYONE for any help...

I'll assume your socket code is right, and I'll fix the comments:
import socket
# open TCP client socket:
IPADDR = '192.168.99.9'
PORTNUM = 2101
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((IPADDR, PORTNUM))
# send command to read A/D value: two hex bytes
tdata = bytes.fromhex('FEA4')
s.send(tdata)
# receive 2-byte (hex) reply
rdata=s.recv(256)
# close socket
s.close()
I'll assume your math is mostly correct, but if you're using Python 2, you might be doing floor division if rdata[0] is an integer (class int).
# convert reply to a voltage reference (unsigned short)
vRef = (rdata[0] * 256)+(rdata[1])
To be safe, first convert vRef to float before the other calculations:
vRef = float(rdata[0] * 256)+(rdata[1])
And then procede:
# convert vref to float as degrees Farenheit
degF = vRef * 4930 / 1024
degF = degF / 10
degF = degF - 273.15
degF = degF * 9 / 5 + 32
This:
# open text file
fo = open("\mnt\stuff\temp.txt", "w")
# write formatted string as number only e.g., 32.6
fo.write("{:+.1f}.format(degF)\n")
# Close file
fo.close()
Can be replaced with this:
with open("\mnt\stuff\temp.txt", "w") as file:
file.write("{:+.1f}.format(degF)\n")
The with keyword opens the file as a context manager, and automatically closes the file for you, even if your code raises an exception. Sockets don't work as context managers in Python 2, unless you create a wrapper for it yourself.