I have a string of bytearray which i want to convert to bytearray in python 3
For example,
x = "\x01\x02\x03\x04"
I get the x variable from server, it is a string but content is byte array, how to convert it to byte array. Really stuck at it. Thanks
You can encode a string to a bytes object and convert that to a bytearray, or convert it directly given some encoding.
x = "\x01\x02\x03\x04" # type: str
y = x.encode() # type: bytes
a = bytearray(x.encode()) # type: bytearray
b = bytearray(x, 'utf-8') # type: bytearray
Note that bytearray(:str, ...) is specified to use str.encode, so the later two practically do the same. The major difference is that you have to specify the encoding explicitly.
You can use ord to convert each byte in the string to an integer. bytearray takes an iterable of integers as an argument, so
x = "\x01\x02\x03\x04"
b = bytearray(ord(c) for c in x) # bytearray(b'\x01\x02\x03\x04')
Try this :
x = bytes(x, 'utf-8')
Now type(x) is bytes.
Related
I have a bytes list and want to get the last item, while preserving its bytes type. Using [-1] it gives out an int type, so this is not a direct solution.
Example code:
x = b'\x41\x42\x43'
y = x[-1]
print (y, type(y))
# outputs:
67 <class 'int'>
For an arbitrary index I know how to do it:
x = b'\x41\x42\x43'
i = 2 # assume here a valid index in reference to list length
y = x[i:i+1]
print (y, type(y))
# outputs:
b'C' <class 'bytes'>
Probably I can calculate the list length and then point an absolute length-1 number, rather than relative to list end.
However, is there a more elegant way to do this ? (i.e. similar to the simple [-1]) I also cannot imagine how to adapt the [i:i+1] principle in reverse from list end.
There are a number of ways I think the one you're interested in is:
someBytes[-1:]
Edit: I just randomly decided to elaborate a bit on what is going on and why this works. A bytes object is an immutable arbitrary memory buffer so a single element of a bytes object is a byte, which is best represented by an int. This is why someBytes[-1] will be the last int in the buffer. It can be counterintuitive when you're using a bytes object like a string for whatever reason (pattern matching, handling ascii data and not bothering to convert to a string,) because a string in python (or a str to be pedantic,) represents the idea of textual data and isn't tied to any particular binary encoding (though it defaults to UTF-8). So the last element of "hello" is "o" which is a string since python has no single char type, just strings of length 1. So if you're treating a bytes object like a memory buffer you likely want an int but if you're treating it like a string you want a bytes object of length 1. So this line tells python to return a slice of the bytes object from the last element to the end of the bytes object which results in a slice length one containing only the last value in the bytes object and a slice of a bytes object is a bytes object.
You can trivially cast that int back to a bytes object:
>>> z = bytes([y])
>>> z == b'C'
True
...in the event that you can't easily get around fetching the values as ints, say because another function you don't have control of returns them that way.
If you have:
x = b'\x41\x42\x43'
Then you will get:
>>> x
b'ABC'
As you said, x[-1] will give you Ord() value.
>>> x[-1]
67
However, if you want to get the value of this, you can give:
>>> x.decode()[-1]
'C'
If you do want to get the value 43, then you can give it as follows:
>>> "{0:x}".format(x[-1])
'43'
Example above
>>> z = bytes([y])
>>> z == b'C'
True
Same you can get with
x.strip()[-1:]
Output
b'C'
So,
bytes(b'\x41\x42\x43')
Give
b'ABC'
I have encoded a string to integer in the following way in python:
b = bytearray()
b.extend(input_number_or_text.encode('ascii'))
input_number_or_text = int.from_bytes(b,byteorder='big', signed=False)
I am encrypting this integer to get a new value and subsequently decrypting to get back the original integer.
Now how do I get back the string from the integer
I have tried the following method for decryption:
decrypted_data.to_bytes(1,byteorder='big').decode('ascii')
but I get int too big to convert error.
How to fix this problem?
You told it the int should be convertable to a length 1 byte string. If it's longer, that won't work. You can remember the length, or you can guess at it:
num_bytes = (decrypted_data.bit_length() + 7) // 8
decrypted_data.to_bytes(num_bytes, byteorder='big').decode('ascii')
Adding 7 and floor-dividing by 8 ensures enough bytes for the data. -(-decrypted_data.bit_length() // 8) also works (and is trivially faster on Python), but is a bit more magical looking.
The byte representation of an integer is different than a string.
For example - 1 , '1', 1.0 all look different when looking at the byte representation.
From the code you supply -
b.extend(input_number_or_text.encode('ascii'))
and int.from_bytes(b,byteorder='big', signed=False)
Seems like your encoding a string of a number, and trying to decode it as a int.
See the next example:
In [3]: b = bytearray()
In [4]: a = '1'
In [5]: b.extend(a.encode('ascii'))
In [6]: int.from_bytes(b,byteorder='big',signed=False)
Out[6]: 49
If you are encoding a string, you should first decode a string, and then convert to int.
In [1]: b = bytearray()
In [2]: a = '123'
In [3]: b.extend(a.encode('ascii'))
In [4]: decoded = int(b.decode('ascii'))
In [5]: decoded
Out[5]: 123
I have a bitstring which I encoded with the function below when I try to decode with my function it doesn't work. what can I do?
def bitstring_to_bytes(self,s):
return int(s, 2).to_bytes(len(s) // 8, byteorder='big')
def bytes_to_string(self,xbytes):
return xbytes.from_bytes(xbytes, 'big')
(More or less) analogous to how the int type knows how to convert an integer instance to bytes, it is int again who knows how to convert bytes back to an integer.
Your xbytes variable is a bytes object, so it doesn't know how to convert itself to an integer.
Instead you do it like this:
intermediate_result = int.from_bytes(xbytes, byteorder='big')
(And afterwards you will want to convert it into a string of 0s and 1s.)
The reason it's not completely orthogonal (Converting forward you use value.method() and converting back you use type.method(value)) is that in the forward case your value is already of the type which knows how to convert itself, but in the backward case, your value is of the other type, who doesn't know how to convert back.
You can also think about it in English terms, if you like:
value.to_bytes(byteorder='big')
would be Convert this integer "value" to a bytes object.
int.from_bytes(value, byteorder='big')
would be Create a fresh integer object from that bytes "value".
I have an array of integer, and I need to transform it into string.
[1,2,3,4] => '\x01\x02\x03\x04'
What function can I use for it? I tried with str(), but it returns '1234'.
string = ""
for val in [1,2,3,4]:
string += str(val) # '1234'
''.join([chr(x) for x in [1, 2, 3, 4]])
You can convert a list of small numbers directly to a bytearray:
If it is an iterable, it must be an iterable of integers in the range 0 <= x < 256, which are used as the initial contents of the array.
And you can convert a bytearray directly to a str (2.x) or bytes (3.x, or 2.6+).
In fact, in 3.x, you can even convert the list straight to bytes without going through bytearray:
constructor arguments are interpreted as for bytearray().
So:
str(bytearray([1,2,3,4])) # 2.6-2.7 only
bytes(bytearray([1,2,3,4])) # 2.6-2.7, 3.0+
bytes([1,2,3,4]) # 3.0+ only
If you really want a string in 3.x, as opposed to a byte string, you need to decode it:
bytes(bytearray([1,2,3,4])).decode('ascii')
See Binary Sequence Types in the docs for more details.
Simple solution
digits = [1,2,3,4]
print(''.join(map(str,digits)))
I have a long Hex string that represents a series of values of different types. I need to convert this Hex String into bytes or bytearray so that I can extract each value from the raw data. How can I do this?
For example, the string "ab" should convert to the bytes b"\xab" or equivalent byte array. Longer example:
>>> # what to use in place of `convert` here?
>>> convert("8e71c61de6a2321336184f813379ec6bf4a3fb79e63cd12b")
b'\x8eq\xc6\x1d\xe6\xa22\x136\x18O\x813y\xeck\xf4\xa3\xfby\xe6<\xd1+'
Suppose your hex string is something like
>>> hex_string = "deadbeef"
Convert it to a bytearray (Python 3 and 2.7):
>>> bytearray.fromhex(hex_string)
bytearray(b'\xde\xad\xbe\xef')
Convert it to a bytes object (Python 3):
>>> bytes.fromhex(hex_string)
b'\xde\xad\xbe\xef'
Note that bytes is an immutable version of bytearray.
Convert it to a string (Python ≤ 2.7):
>>> hex_data = hex_string.decode("hex")
>>> hex_data
"\xde\xad\xbe\xef"
There is a built-in function in bytearray that does what you intend.
bytearray.fromhex("de ad be ef 00")
It returns a bytearray and it reads hex strings with or without space separator.
provided I understood correctly, you should look for binascii.unhexlify
import binascii
a='45222e'
s=binascii.unhexlify(a)
b=[ord(x) for x in s]
Assuming you have a byte string like so
"\x12\x45\x00\xAB"
and you know the amount of bytes and their type you can also use this approach
import struct
bytes = '\x12\x45\x00\xAB'
val = struct.unpack('<BBH', bytes)
#val = (18, 69, 43776)
As I specified little endian (using the '<' char) at the start of the format string the function returned the decimal equivalent.
0x12 = 18
0x45 = 69
0xAB00 = 43776
B is equal to one byte (8 bit) unsigned
H is equal to two bytes (16 bit) unsigned
More available characters and byte sizes can be found here
The advantages are..
You can specify more than one byte and the endian of the values
Disadvantages..
You really need to know the type and length of data your dealing with
You can use the Codecs module in the Python Standard Library, i.e.
import codecs
codecs.decode(hexstring, 'hex_codec')
You should be able to build a string holding the binary data using something like:
data = "fef0babe"
bits = ""
for x in xrange(0, len(data), 2)
bits += chr(int(data[x:x+2], 16))
This is probably not the fastest way (many string appends), but quite simple using only core Python.
A good one liner is:
byte_list = map(ord, hex_string)
This will iterate over each char in the string and run it through the ord() function. Only tested on python 2.6, not too sure about 3.0+.
-Josh