I have IPv4 address and want to convert it to 32 bit integer.
i am able to convert IPv4 address into string using socket.inet_ntop and then convert that string to 32 bit integer
but is there a direct way?
An IPv4 address in its basic form is a 32-bit integer in network byte order.
I'm assuming you have it as a sequence of bytes (because that is what you would normally hand off to inet_ntop).
What you will need to convert it into a python integer is the struct module and its unpack method along with the "!I" format specification (which means network byte order, unsigned 32-bit integer). See this code:
from socket import inet_ntop, inet_pton, AF_INET
from struct import unpack
ip = inet_pton(AF_INET, "192.168.1.42")
ip_as_integer = unpack("!I", ip)[0]
print("As string[{}] => As bytes[{}] => As integer[{}]".format(
inet_ntop(AF_INET, ip), ip, ip_as_integer))
You could of course also reconstruct the integer bytewise:
ip_as_integer = (ip[0] << 24) | (ip[1] << 16) | (ip[2] << 8) | ip[3]
Related
I am receiving data via socket interface from an application (server) written in C. The data being posted has the following structure. I am receiving data with a client written in Python.
struct hdr
{
int Id;
char PktType;
int SeqNo;
int Pktlength;
};
struct trl
{
char Message[16];
long long info;
};
struct data
{
char value[10];
double result;
long long count;
short int valueid;
};
typedef struct
{
struct hdr hdr_buf;
struct data data_buf[100];
struct trl trl_buf;
} trx_unit;
How do I unpack the received data to access my inner data buffer?
Using the struct library is the way to go. However, you will have to know a bit more about the C program that is serializing the data. Consider the hdr structure. If the C program is sending it using the naive approach:
struct hdr header;
send(sd, &hdr, sizeof(header), 0);
Then your client cannot safely interpret the bytes that are sent to it because there is an indeterminate amount of padding inserted between the struct members. In particular, I would expect three bytes of padding following the PktType member.
The safest way to approach sending around binary data is to have the server and client serialize the bytes directly to ensure that there is no additional padding and to make the byte ordering of multibyte integers explicit. For example:
/*
* Send a header over a socket.
*
* The header is sent as a stream of packed bytes with
* integers in "network" byte order. For example, a
* header value of:
* Id: 0x11223344
* PktType: 0xff
* SeqNo: 0x55667788
* PktLength: 0x99aabbcc
*
* is sent as the following byte stream:
* 11 22 33 44 ff 55 66 77 88 99 aa bb cc
*/
void
send_header(int sd, struct hdr const* header)
{ /* NO ERROR HANDLING */
uint32_t num = htonl((uint32_t)header->Id);
send(sd, &num, sizeof(num), 0);
send(sd, &header->PktType, sizeof(header->PktType), 0);
num = htonl((uint32_t)header->SeqNo);
send(sd, &num, sizeof(num), 0);
num = htonl((uint32_t)header->PktLength);
send(sd, &num, sizeof(num), 0);
}
This will ensure that your client can safely decode it using the struct module:
buf = s.recv(13) # packed data is 13 bytes long
id_, pkt_type, seq_no, pkt_length = struct.unpack('>IBII', buf)
If you cannot modify the C code to fix the serialization indeterminacy, then you will have to read the data from the stream and figure out where the C compiler is inserting padding and manually build struct format strings to match using the padding byte format character to ignore padding values.
I usually write a decoder class in Python that reads a complete value from the socket. In your case it would look something like:
class PacketReader(object):
def __init__(self, sd):
self._socket = sd
def read_packet(self):
id_, pkt_type, seq_no, pkt_length = self._read_header()
data_bufs = [self._read_data_buf() for _ in range(0, 100)]
message, info = self._read_trl()
return {'id': id_, 'pkt_type': pkt_type, 'seq_no': seq_no,
'data_bufs': data_bufs, 'message': message,
'info': info}
def _read_header(self):
"""
Read and unpack a ``hdr`` structure.
:returns: a :class:`tuple` of the header data values
in order - *Id*, *PktType*, *SeqNo*, and *PktLength*
The header is assumed to be packed as 13 bytes with
integers in network byte order.
"""
buf = self._socket.read(13)
# > Multibyte values in network order
# I Id as 32-bit unsigned integer value
# B PktType as 8-bit unsigned integer value
# I SeqNo as 32-bit unsigned integer value
# I PktLength as 32-bit unsigned integer value
return struct.unpack('>IBII', buf)
def _read_data_buf(self):
"""
Read and unpack a single ``data`` structure.
:returns: a :class:`tuple` of data values in order -
*value*, *result*, *count*, and *value*
The data structure is assumed to be packed as 28 bytes
with integers in network byte order and doubles encoded
as IEEE 754 binary64 in network byte order.
"""
buf = self._socket.read(28) # assumes double is binary64
# > Multibyte values in network order
# 10s value bytes
# d result encoded as IEEE 754 binary64 value
# q count encoded as a 64-bit signed integer
# H valueid as a 16-bit unsigned integer value
return struct.unpack('>10sdqH', buf)
def _read_trl(self):
"""
Read and unpack a ``trl`` structure.
:returns: a :class:`tuple` of trl values in order -
*Message* as byte string, *info*
The structure is assumed to be packed as 24 bytes with
integers in network byte order.
"""
buf = self.socket.read(24)
# > Multibyte values in network order
# 16s message bytes
# q info encoded as a 64-bit signed value
return struct.unpack('>16sq', buf)
Mind you that this is untested and probably contains syntax errors but that is how I would approach the problem.
The struct library has all you need to do this.
I'm working with pySerial library to read some byte from Serial Port.
I need to store these bytes into a list, but I would store data in Hex format because the serial peripheral send to me specific command code like 0x10, 0x30, ...
The code:
readByte = serialCOM.read(1)
print 'Read Payload byte:' + str(readByte)
packetPayload.append(readByte)
creates a list of char.
Which is the way to create a list of hex values?
Thanks!
In Python, hex values are just an alternate view of integers:
>>> 1 == 0x01
True
>>> 16 == 0x10
True
So, here is a list:
>>> print([0x10, 0x20])
[16, 32]
It's also possible to use the int function to get integer value from a string:
>>> int('0x10', 16) # here, the 16 is for the base, and 0x is optionnal
16
I am building a p2p application where it is necessary for me to unpack an udp announce response from the tracker.
announce response:
Offset Size Name
0 32-bit integer action
4 32-bit integer transaction_id
8 32-bit integer interval
12 32-bit integer leechers
16 32-bit integer seeders
20 + 6 * n 32-bit integer IP address
24 + 6 * n 16-bit integer TCP port
20 + 6 * N
I need to read all the data listed in name field from the above table, but the values of seeders, ip address and port are variable.
how do i unpack all the data from the response properly using python struct ?
The struct module is pretty usefull for unpacking binary data.
action, transaction_id, interval, leechers, seeders = struct.unpack('>iiiii', announse_response)
You then have to loop over the rest of the data to get all ip/port data:
ip_port_list = []
while True:
try: ip_port_list.append(struct.unpack('>ih', announse_response))
except: break
Without struct you will have to read byte by byte and then convert the big endian problem.
I experienced the same struggle, so wrote a simple package for this called binread.
from binread import formatclass, U32, U16, Array
#formatclass
class Seeder:
ip_addr = U32
tcp_port = U16
#formatclass
class AnnounceResponse:
action = U32
transaction_id = U32
interval = U32
leechers = U32
seeders = U32
# `length` refers to the value of the previous `seeders` field
seeder_data = Array(Seeder, length="seeders")
# `data` is a bytes object
resp = AnnounceResponse.read(data)
print(resp.transaction_id)
print(len(resp.seeder_data))
print(resp.seeder_data[0].tcp_port)
You could optionally specify the endianness for all values with #formatclass(byteorder='little') (or 'big').
Full disclosure, I am the author of binread.
Suppose there is a user-defined protocol as below:
The protocol:
------------- ------------- ---------------- -----------------
| Seqno. | ip | port | user name |
| int, 4 bytes| int, 4 bytes| short, 2 bytes | string, 50 bytes|
the [user name] field stores a string ending with zero,
if the string length is less than 50 bytes, padding with zeros.
Usually I will pack these fields in C language like this:
//Pseudo code
buffer = new char[60];
memset(buffer, 0, 60);
memcpy(buffer, &htonl(Seqno), 4);
memcpy(buffer+4, &htonl(ip), 4);
memcpy(buffer+4, &htons(port), 2);
memcpy(buffer+2, Usrname.c_str(), Usrname.length() + 1);
But how can we pack the protocol data in python? I am new to python.
Use the struct module:
import struct
binary_value = struct.pack('!2IH50s', seqno, ip, port, usrname)
This packs 2 4-byte unsigned integers, a 2-byte unsigned short and a 50-byte string into 60 bytes with network (big-endian) byte ordering. The string will be padded out with nulls to make up the length:
>>> import struct
>>> seqno = 42
>>> ip = 0xc6fcce10
>>> port = 80
>>> usrname = 'Martijn Pieters'
>>> struct.pack('!2IH50s', seqno, ip, port, usrname)
'\x00\x00\x00*\xc6\xfc\xce\x10\x00PMartijn Pieters\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
Python's string representation uses ASCII characters for any bytes in the ASCII printable range, \xhh for most other byte points, so the 42 became \x00\x00\x00*.
I'm trying to transfer int data into ip address, for IPV6 and IPv4
def int2ip(i):
if i > sys.maxint * 2 - 1: # ipv6
socket.inet_ntop(socket.AF_INET6, struct.pack('!L', i))
socket.inet_ntop(socket.AF_INET, struct.pack('!I', i)) # ipv4
But for a IPv6 address 240e:800::, it's int represent is 47925062263553288224699872696711774208, but while passing the int value to int2ip, throw the following error:
struct.error: integer out of range for 'L' format code
but the int value is really a long type:
>>> print(type(47925062263553288224699872696711774208)
<type 'long'>
Why?
You are trying to put a 128bit number into 32bit ('L' format code).
Python 3 has a library to manage ip addresses (4 & 6): https://docs.python.org/3/library/ipaddress.html.
For python2 there is a backport: https://pypi.python.org/pypi/ipaddress