I want to convert an 32-byte (although I might need other lengths) integer to a bytes object in python. Is there a clean and simple way to do this?
to_bytes(length, byteorder[, signed]) is all you need starting from 3.2. In this case, someidentifier.to_bytes(4,'big') should give you the bytes string you need.
I'm guessing you need a 32-bit integer, and big-endian to boot:
>>> from ctypes import c_uint32
>>> l = c_uint32(0x12345678)
>>> bytes(l)
b'xV4\x12'
There is c_uint8, c_uint16 and c_uint64 as well. For longer ints you need to make it manually, using divmod(x, 256).
>>> def bytify(v):
... v, r = divmod(v, 256)
... yield r
... if v == 0:
... raise StopIteration
... for r in bytify(v):
... yield r
...
>>> [x for x in bytify(0x12345678)]
[120, 86, 52, 18]
>>> bytes(bytify(0x12345678))
b'xV4\x12
>>> bytes(bytify(0x123456789098765432101234567890987654321))
b'!Ce\x87\t\x89gE#\x01!Ce\x87\t\x89gE#\x01'
You can use bytes("iterable") directly. Where every value in iterable will be specific byte in bytes(). Example for little endian encoding:
>>> var=0x12345678
>>> var_tuple=((var)&0xff, (var>>8)&0xff, (var>>16)&0xff, (var>>24)&0xff)
>>> bytes(var_tuple)
b'xV4\x12'
Suppose you have
var = 'і' # var is ukrainian і
We want to get binary from it.
Flow is this. value/which is string => bytes => int => binary
binary_var = '{:b}'.format(int.from_bytes(var.encode('utf-8'), byteorder='big'))
Now binary_var is '1101000110010110'. It type is string.
Now go back, you want get unicode value from binary:
int_var = int(binary_var, 2) # here we get int value, int_var = 53654
Now we need convert integer to bytes. Ukrainian 'і' is not gonna fit into 1 byte but in 2. We convert to actual bytes bytes_var = b'\xd1\x96'
bytes_var = int_var.to_bytes(2, byteorder='big')
Finally we decode our bytes.
ukr_i = bytes_var.decode('utf-8') # urk_i = 'і'
Related
I am trying to multiply two data of bytes data type using python by trying to converting them into hexadecimal values. but it seems that the hex() function is not working. Can you please help me on that. Below is my code (x and y are our data of type bytes)
data=bytes.hex(x) * bytes.hex(y)
I am getting the error: TypeError:'str' object cannot be interpreted as an integer
by trying
data = hex(x) * hex(y)
the error become: TypeError:'bytes' object cannot be interpreted as an integer
Can anyone help please?
hex(my_var) returns a string. To do math you need to convert your variables to int:
x_int = int(x, base=16)
y_int = int(y, base=16)
decimal_value = x_int * y_int # integer
hex_value = hex(decimal_value) # string
data = hex(x) * hex(y)
This doesn't work because the hex() function expects an int as an argument and outputs a string.
In Python 3, you can do the following (interprets bytes as base 10)
data = hex(int(x)*int(y))
I see in the comments below your question that you have examples of your x and y values - you have some very large numbers to process if you interpret the bytes objects as arbitrary length integers. Not sure this is what you want, but this will do conversion of bytes to/from integer values.
>>> int_x = int.from_bytes(x, 'big') # From bytes to int.
>>> int_y = int.from_bytes(y, 'big')
>>> int_z = int_x * int_y # int_z as an integer.
>>>
>>> # Converting int_z to a bytes object:
>>>
>>> bytes_length = (int_z.bit_length() + 7) // 8
>>>
>>> z = int_z.to_bytes(bytes_length, 'big')
If you want them interpreted as little endian, replace 'big' with 'little'.
If your y value is already an int type, which it looks like in your comment, then don't convert it before multiplying:
>>> int_x = int.from_bytes(x, 'big')
>>> int_y = y
>>> int_z = int_x * int_y
>>> z = int_z.to_bytes(bytes_length, 'big')
The first parameter to int.to_bytes() is the length of the bytes object to create. There are other ways to calculate this value, but the way I included is the fastest method out of 3 that I timed.
If all you wanted to do was convert a bytes object to its hex string representation (z below is treated as a bytes object):
>>> hex_z = '0x' + ''.join(hex(b)[2:] for b in z)
Or to convert an int to a hex string:
>>> hex_z = hex(int_z)
hex() can handle very large integers.
I have a bytestring it multiple 4 bytes. Each 4 bytes is a float but for each 4 bytes first two bytes and next two bytes need to be swapped.
example:
input = b'\x00\x00#\xb9a\xa8\xbdf'
struct.unpack('f', b'\x00\x00#\xb9') #-0.00018310546875 is incorrect
struct.unpack('>f', b'\x00\x00#\xb9') #2.3218114255397894e-41 is incorrect
struct.unpack('>f', b'#\xb9\x00\x00') # 5.78125 is correct
How i can easy swap two hi bytes and two low bytes?
Next code is correct
h = b''
for i in range(0, len(input), 4):
h += struct.pack('BBBB', *(d[i+2],d[i+3],d[i],d[i+1]))
struct.unpack('>ff', h) # is correct (5.78125, -0.05624547600746155)
but may be there are another more easiest way.
This works for me:
>>> a = b'\x12\x34\x56\x78\x9a\xbc\xde\xf0'
>>> ''.join( [chr(a[i^2]) for i in range(len(a))] )
'Vx\x124Þð\x9a¼'
>>>
You don't even need to use a struct for that.
I'm currently working on an encryption/decryption program and I need to be able to convert bytes to an integer. I know that:
bytes([3]) = b'\x03'
Yet I cannot find out how to do the inverse. What am I doing terribly wrong?
Assuming you're on at least 3.2, there's a built in for this:
int.from_bytes( bytes, byteorder, *, signed=False )
...
The argument bytes must either be a bytes-like object or an iterable
producing bytes.
The byteorder argument determines the byte order used to represent the
integer. If byteorder is "big", the most significant byte is at the
beginning of the byte array. If byteorder is "little", the most
significant byte is at the end of the byte array. To request the
native byte order of the host system, use sys.byteorder as the byte
order value.
The signed argument indicates whether two’s complement is used to
represent the integer.
## Examples:
int.from_bytes(b'\x00\x01', "big") # 1
int.from_bytes(b'\x00\x01', "little") # 256
int.from_bytes(b'\x00\x10', byteorder='little') # 4096
int.from_bytes(b'\xfc\x00', byteorder='big', signed=True) #-1024
Lists of bytes are subscriptable (at least in Python 3.6). This way you can retrieve the decimal value of each byte individually.
>>> intlist = [64, 4, 26, 163, 255]
>>> bytelist = bytes(intlist) # b'#\x04\x1a\xa3\xff'
>>> for b in bytelist:
... print(b) # 64 4 26 163 255
>>> [b for b in bytelist] # [64, 4, 26, 163, 255]
>>> bytelist[2] # 26
list() can be used to convert bytes to int (works in Python 3.7):
list(b'\x03\x04\x05')
[3, 4, 5]
int.from_bytes( bytes, byteorder, *, signed=False )
doesn't work with me
I used function from this website, it works well
https://coderwall.com/p/x6xtxq/convert-bytes-to-int-or-int-to-bytes-in-python
def bytes_to_int(bytes):
result = 0
for b in bytes:
result = result * 256 + int(b)
return result
def int_to_bytes(value, length):
result = []
for i in range(0, length):
result.append(value >> (i * 8) & 0xff)
result.reverse()
return result
In case of working with buffered data I found this useful:
int.from_bytes([buf[0],buf[1],buf[2],buf[3]], "big")
Assuming that all elements in buf are 8-bit long.
An old question that I stumbled upon while looking for an existing solution. Rolled my own and thought I'd share because it allows you to create a 32-bit integer from a list of bytes, specifying an offset.
def bytes_to_int(bList, offset):
r = 0
for i in range(4):
d = 32 - ((i + 1) * 8)
r += bList[offset + i] << d
return r
#convert bytes to int
def bytes_to_int(value):
return int.from_bytes(bytearray(value), 'little')
bytes_to_int(b'\xa231')
I need to split a 16-bit unsigned integer into an array of bytes (i.e. array.array('B')) in python.
For example:
>>> reg_val = 0xABCD
[insert python magic here]
>>> print("0x%X" % myarray[0])
0xCD
>>> print("0x%X" % myarray[1])
0xAB
The way I'm currently doing it seems very complicated for something so simple:
>>> import struct
>>> import array
>>> reg_val = 0xABCD
>>> reg_val_msb, reg_val_lsb = struct.unpack("<BB", struct.pack("<H", (0xFFFF & reg_val)))
>>> myarray = array.array('B')
>>> myarray.append(reg_val_msb)
>>> myarray.append(reg_val_lsb)
Is there a better/more efficient/more pythonic way of accomplishing the same thing?
(using python 3 here, there are some nomenclature differences in 2)
Well first, you could just leave everything as bytes. This is perfectly valid:
reg_val_msb, reg_val_lsb = struct.pack('<H', 0xABCD)
bytes allows for "tuple unpacking" (not related to struct.unpack, tuple unpacking is used all over python). And bytes is an array of bytes, which can be accessed via index as you wanted.
b = struct.pack('<H',0xABCD)
b[0],b[1]
Out[52]: (205, 171)
If you truly wanted to get it into an array.array('B'), it's still rather easy:
ary = array('B',struct.pack('<H',0xABCD))
# ary = array('B', [205, 171])
print("0x%X" % ary[0])
# 0xCD
For non-complex numbers you can use divmod(a, b), which returns a tuple of the quotient and remainder of arguments.
The following example uses map() for demonstration purposes. In both examples we're simply telling divmod to return a tuple (a/b, a%b), where a=0xABCD and b=256.
>>> map(hex, divmod(0xABCD, 1<<8)) # Add a list() call here if your working with python 3.x
['0xab', '0xcd']
# Or if the bit shift notation is distrubing:
>>> map(hex, divmod(0xABCD, 256))
Or you can just place them in the array:
>>> arr = array.array('B')
>>> arr.extend(divmod(0xABCD, 256))
>>> arr
array('B', [171, 205])
You can write your own function like this.
def binarray(i):
while i:
yield i & 0xff
i = i >> 8
print list(binarray(0xABCD))
#[205, 171]
I have a non-negative int and I would like to efficiently convert it to a big-endian string containing the same data. For example, the int 1245427 (which is 0x1300F3) should result in a string of length 3 containing three characters whose byte values are 0x13, 0x00, and 0xf3.
My ints are on the scale of 35 (base-10) digits.
How do I do this?
In Python 3.2+, you can use int.to_bytes:
If you don't want to specify the size
>>> n = 1245427
>>> n.to_bytes((n.bit_length() + 7) // 8, 'big') or b'\0'
b'\x13\x00\xf3'
If you don't mind specifying the size
>>> (1245427).to_bytes(3, byteorder='big')
b'\x13\x00\xf3'
You can use the struct module:
import struct
print(struct.pack('>I', your_int))
'>I' is a format string. > means big endian and I means unsigned int. Check the documentation for more format chars.
This is fast and works for small and (arbitrary) large ints:
def Dump(n):
s = '%x' % n
if len(s) & 1:
s = '0' + s
return s.decode('hex')
print repr(Dump(1245427)) #: '\x13\x00\xf3'
Probably the best way is via the built-in struct module:
>>> import struct
>>> x = 1245427
>>> struct.pack('>BH', x >> 16, x & 0xFFFF)
'\x13\x00\xf3'
>>> struct.pack('>L', x)[1:] # could do it this way too
'\x13\x00\xf3'
Alternatively -- and I wouldn't usually recommend this, because it's mistake-prone -- you can do it "manually" by shifting and the chr() function:
>>> x = 1245427
>>> chr((x >> 16) & 0xFF) + chr((x >> 8) & 0xFF) + chr(x & 0xFF)
'\x13\x00\xf3'
Out of curiosity, why do you only want three bytes? Usually you'd pack such an integer into a full 32 bits (a C unsigned long), and use struct.pack('>L', 1245427) but skip the [1:] step?
def tost(i):
result = []
while i:
result.append(chr(i&0xFF))
i >>= 8
result.reverse()
return ''.join(result)
Single-source Python 2/3 compatible version based on #pts' answer:
#!/usr/bin/env python
import binascii
def int2bytes(i):
hex_string = '%x' % i
n = len(hex_string)
return binascii.unhexlify(hex_string.zfill(n + (n & 1)))
print(int2bytes(1245427))
# -> b'\x13\x00\xf3'
The shortest way, I think, is the following:
import struct
val = 0x11223344
val = struct.unpack("<I", struct.pack(">I", val))[0]
print "%08x" % val
This converts an integer to a byte-swapped integer.
Using the bitstring module:
>>> bitstring.BitArray(uint=1245427, length=24).bytes
'\x13\x00\xf3'
Note though that for this method you need to specify the length in bits of the bitstring you are creating.
Internally this is pretty much the same as Alex's answer, but the module has a lot of extra functionality available if you want to do more with your data.
Very easy with pwntools , the tools created for software hacking
(Un-ironically, I stumbled across this thread and tried solutions here, until I realised there exists conversion functionality in pwntools)
import pwntools
x2 = p32(x1)