Develop Reference

Reverse int as hex

I have int in python that I want to reverse
x = int(1234567899)
I want to result will be 3674379849
explain : = 1234567899 = 0x499602DB and 3674379849 = 0xDB029649
How to do that in python ?

>>> import struct
>>> struct.unpack('>I', struct.pack('<I', 1234567899))[0]
3674379849
>>>
This converts the integer to a 4-byte array (I), then decodes it in reverse order (> vs <).
Documentation: struct

If you just want the result, use sabiks approach - if you want the intermediate steps for bragging rights, you would need to
create the hex of the number (#1) and maybe add a leading 0 for correctness
reverse it 2-byte-wise (#2)
create an integer again (#3)
f.e. like so
n = 1234567899
# 1
h = hex(n)
if len(h) % 2: # fix for uneven lengthy inputs (f.e. n = int("234",16))
h = '0x0'+h[2:]
# 2 (skips 0x and prepends 0x for looks only)
bh = '0x'+''.join([h[i: i+2] for i in range(2, len(h), 2)][::-1])
# 3
b = int(bh, 16)
print(n, h, bh, b)
to get
1234567899 0x499602db 0xdb029649 3674379849

Python code to convert from objectSid to SID representation

I want to retrieve base64 encoded objectSid from an LDAP query to an Active Directory database and convert them to the standard SID representation. Can you please give me a Python snippet that does that?

This should do the trick:
import struct
def convert(binary):
version = struct.unpack('B', binary[0])[0]
# I do not know how to treat version != 1 (it does not exist yet)
assert version == 1, version
length = struct.unpack('B', binary[1])[0]
authority = struct.unpack('>Q', '\x00\x00' + binary[2:8])[0]
string = 'S-%d-%d' % (version, authority)
binary = binary[8:]
assert len(binary) == 4 * length
for i in xrange(length):
value = struct.unpack('<L', binary[4*i:4*(i+1)])[0]
string += '-%d' % value
return string
References: http://blogs.msdn.com/b/oldnewthing/archive/2004/03/15/89753.aspx and http://codeimpossible.com/2008/04/07/Converting-a-Security-Identifier-from-binary-to-string/.

This is #Giovanni Mascellani answer, adapted for Python 3.x:
import struct
def convert(binary):
version = struct.unpack('B', binary[0:1])[0]
# I do not know how to treat version != 1 (it does not exist yet)
assert version == 1, version
length = struct.unpack('B', binary[1:2])[0]
authority = struct.unpack(b'>Q', b'\x00\x00' + binary[2:8])[0]
string = 'S-%d-%d' % (version, authority)
binary = binary[8:]
assert len(binary) == 4 * length
for i in range(length):
value = struct.unpack('<L', binary[4*i:4*(i+1)])[0]
string += '-%d' % value
return string

use or see implementation in ldap3
ldap-doc
source
ldap3.protocol.formatters.formatters.format_sid

A SID in format S-1-5-21-2562418665-3218585558-1813906818-1576 has the following hex raw format: 010500000000000515000000e967bb98d6b7d7bf82051e6c28060000 and can be break down as follows:
S : it means simply this is an objectSid
01 : (1) is the revision ans is always 1 as far as I know
05 : count of sub authorities or you can simply say dash count minus two
000000000005 : (5) big endian
15000000 : (21) small endian
e967bb98 : (2562418665) small endian
d6b7d7bf : (3218585558) small endian
82051e6c : (1813906818) small endian
28060000 : (1576) small endian
Small endian should be read in reverse order. That is how the binary data is represented, where is the least significant byte is stored first.
Accordingly Big endian is the order where the most significant byte is stored first. Here is a good article explaining the byte order.
This is a brief introduction about the structure of the objecSid, and this is a nice blog post.
According to these information let's try to read a SID, which is returned as a binary from the LDAP query. The binascii library can be used for Python 2 as well as for Python 3:
from binascii import b2a_hex
def sid_to_str(sid):
try:
# Python 3
if str is not bytes:
# revision
revision = int(sid[0])
# count of sub authorities
sub_authorities = int(sid[1])
# big endian
identifier_authority = int.from_bytes(sid[2:8], byteorder='big')
# If true then it is represented in hex
if identifier_authority >= 2 ** 32:
identifier_authority = hex(identifier_authority)
# loop over the count of small endians
sub_authority = '-' + '-'.join([str(int.from_bytes(sid[8 + (i * 4): 12 + (i * 4)], byteorder='little')) for i in range(sub_authorities)])
# Python 2
else:
revision = int(b2a_hex(sid[0]))
sub_authorities = int(b2a_hex(sid[1]))
identifier_authority = int(b2a_hex(sid[2:8]), 16)
if identifier_authority >= 2 ** 32:
identifier_authority = hex(identifier_authority)
sub_authority = '-' + '-'.join([str(int(b2a_hex(sid[11 + (i * 4): 7 + (i * 4): -1]), 16)) for i in range(sub_authorities)])
objectSid = 'S-' + str(revision) + '-' + str(identifier_authority) + sub_authority
return objectSid
except Exception:
pass
return sid
sid = b'\x01\x05\x00\x00\x00\x00\x00\x05\x15\x00\x00\x00\xe9\x67\xbb\x98\xd6\xb7\xd7\xbf\x82\x05\x1e\x6c\x28\x06\x00\x00'
print(sid_to_str(sid)) # S-1-5-21-2562418665-3218585558-1813906818-1576

If you're using Linux and have Samba installed:
from samba.dcerpc import security
from samba.ndr import ndr_unpack
def convert(binary_sid):
return str(ndr_unpack(security.dom_sid, binary_sid))
Where binary_sid is the binary representation of the sid.

Write boolean string to binary file?

I have a string of booleans and I want to create a binary file using these booleans as bits. This is what I am doing:
# first append the string with 0s to make its length a multiple of 8
while len(boolString) % 8 != 0:
boolString += '0'
# write the string to the file byte by byte
i = 0
while i < len(boolString) / 8:
byte = int(boolString[i*8 : (i+1)*8], 2)
outputFile.write('%c' % byte)
i += 1
But this generates the output 1 byte at a time and is slow. What would be a more efficient way to do it?

It should be quicker if you calculate all your bytes first and then write them all together. For example
b = bytearray([int(boolString[x:x+8], 2) for x in range(0, len(boolString), 8)])
outputFile.write(b)
I'm also using a bytearray which is a natural container to use, and can also be written directly to your file.
You can of course use libraries if that's appropriate such as bitarray and bitstring. Using the latter you could just say
bitstring.Bits(bin=boolString).tofile(outputFile)

Here's another answer, this time using an industrial-strength utility function from the PyCrypto - The Python Cryptography Toolkit where, in version 2.6 (the current latest stable release), it's defined inpycrypto-2.6/lib/Crypto/Util/number.py.
The comments preceeding it say:
Improved conversion functions contributed by Barry Warsaw, after careful benchmarking
import struct
def long_to_bytes(n, blocksize=0):
"""long_to_bytes(n:long, blocksize:int) : string
Convert a long integer to a byte string.
If optional blocksize is given and greater than zero, pad the front of the
byte string with binary zeros so that the length is a multiple of
blocksize.
"""
# after much testing, this algorithm was deemed to be the fastest
s = b('')
n = long(n)
pack = struct.pack
while n > 0:
s = pack('>I', n & 0xffffffffL) + s
n = n >> 32
# strip off leading zeros
for i in range(len(s)):
if s[i] != b('\000')[0]:
break
else:
# only happens when n == 0
s = b('\000')
i = 0
s = s[i:]
# add back some pad bytes. this could be done more efficiently w.r.t. the
# de-padding being done above, but sigh...
if blocksize > 0 and len(s) % blocksize:
s = (blocksize - len(s) % blocksize) * b('\000') + s
return s

You can convert a boolean string to a long using data = long(boolString,2). Then to write this long to disk you can use:
while data > 0:
data, byte = divmod(data, 0xff)
file.write('%c' % byte)
However, there is no need to make a boolean string. It is much easier to use a long. The long type can contain an infinite number of bits. Using bit manipulation you can set or clear the bits as needed. You can then write the long to disk as a whole in a single write operation.

You can try this code using the array class:
import array
buffer = array.array('B')
i = 0
while i < len(boolString) / 8:
byte = int(boolString[i*8 : (i+1)*8], 2)
buffer.append(byte)
i += 1
f = file(filename, 'wb')
buffer.tofile(f)
f.close()

A helper class (shown below) makes it easy:
class BitWriter:
def __init__(self, f):
self.acc = 0
self.bcount = 0
self.out = f
def __del__(self):
self.flush()
def writebit(self, bit):
if self.bcount == 8 :
self.flush()
if bit > 0:
self.acc |= (1 << (7-self.bcount))
self.bcount += 1
def writebits(self, bits, n):
while n > 0:
self.writebit( bits & (1 << (n-1)) )
n -= 1
def flush(self):
self.out.write(chr(self.acc))
self.acc = 0
self.bcount = 0
with open('outputFile', 'wb') as f:
bw = BitWriter(f)
bw.writebits(int(boolString,2), len(boolString))
bw.flush()

Use the struct package.
This can be used in handling binary data stored in files or from network connections, among other sources.
Edit:
An example using ? as the format character for a bool.
import struct
p = struct.pack('????', True, False, True, False)
assert p == '\x01\x00\x01\x00'
with open("out", "wb") as o:
o.write(p)
Let's take a look at the file:
$ ls -l out
-rw-r--r-- 1 lutz lutz 4 Okt 1 13:26 out
$ od out
0000000 000001 000001
000000
Read it in again:
with open("out", "rb") as i:
q = struct.unpack('????', i.read())
assert q == (True, False, True, False)

Find maximum signed short integer in python

How do I get the maximum signed short integer in Python (i.e. SHRT_MAX in C's limits.h)?
I want to normalize samples from a single channel of a *.wav file, so instead of a bunch of 16-bit signed integers, I want a bunch of floats between 1 and -1. Here's what I've got (the pertinent code is in the normalized_samples() function):
def samples(clip, chan_no = 0):
# *.wav files generally come in 8-bit unsigned ints or 16-bit signed ints
# python's wave module gives sample width in bytes, so STRUCT_FMT
# basically converts the wave.samplewidth into a struct fmt string
STRUCT_FMT = { 1 : 'B',
2 : 'h' }
for i in range(clip.getnframes()):
yield struct.unpack(STRUCT_FMT[clip.getsampwidth()] * clip.getnchannels(),
clip.readframes(1))[chan_no]
def normalized_samples(clip, chan_no = 0):
for sample in samples(clip, chan_no):
yield float(sample) / float(32767) ### THIS IS WHERE I NEED HELP

GregS is right, this is not the right way to solve the problem. If your samples are known 8 or 16 bit, you don't want to be dividing them by a number that varies by platform.
You may be running into trouble because a signed 16-bit int actually ranges from -32768 to 32767. Dividing by 32767 is going to give you < -1 in the extreme negative case.
Try this:
yield float(sample + 2**15) / 2**15 - 1.0

Here is a way using cython
getlimit.py
import pyximport; pyximport.install()
import limits
print limits.shrt_max
limits.pyx
import cython
cdef extern from "limits.h":
cdef int SHRT_MAX
shrt_max = SHRT_MAX

in module sys, sys.maxint. Though I'm not sure that is the correct way to solve your problem.

I can't imagine circumstances on a modern computer (i.e. one that uses 2's complement integers) where this would fail:
assert -32768 <= signed_16_bit_integer <= 32767
To do exactly what you asked for:
if signed_16_bit_integer >= 0:
afloat = signed_16_bit_integer / 32767.0
else:
afloat = signed_16_bit_integer / -32768.0
Having read your code a bit more closely: you have sample_width_in_bytes so just divide by 255 or 256 if it's B and by 32768 if it's h

#!/usr/bin/env python2
# maximums.py
####################################333#########################
B16_MAX = (1 << 16) - 1
B15_MAX = (1 << 15) - 1
B08_MAX = (1 << 8) - 1
B07_MAX = (1 << 7) - 1
print
print "hex(B16_MAX) =",hex(B16_MAX) # 0xffff
print "hex(B15_MAX) =",hex(B15_MAX) # 0x7fff
print "hex(B08_MAX) =",hex(B08_MAX) # 0xff
print "hex(B07_MAX) =",hex(B07_MAX) # 0x7f
print
####################################333#########################
UBYTE2_MAX = B16_MAX
SBYTE2_MAX = B15_MAX
UBYTE1_MAX = B08_MAX
SBYTE1_MAX = B07_MAX
print
print "UBYTE2_MAX =",UBYTE2_MAX # 65535
print "SBYTE2_MAX =",SBYTE2_MAX # 32767
print "UBYTE1_MAX =",UBYTE1_MAX # 255
print "SBYTE1_MAX =",SBYTE1_MAX # 127
print
####################################333#########################
USHRT_MAX = UBYTE2_MAX
SHRT_MAX = SBYTE2_MAX
CHAR_MAX = UBYTE1_MAX
BYTE_MAX = SBYTE1_MAX
print
print "USHRT_MAX =",USHRT_MAX # 65535
print " SHRT_MAX =", SHRT_MAX # 32767
print " CHAR_MAX =", CHAR_MAX # 255
print " BYTE_MAX =", BYTE_MAX # 127
print
####################################333#########################

Convert a Python int into a big-endian string of bytes

I have a non-negative int and I would like to efficiently convert it to a big-endian string containing the same data. For example, the int 1245427 (which is 0x1300F3) should result in a string of length 3 containing three characters whose byte values are 0x13, 0x00, and 0xf3.
My ints are on the scale of 35 (base-10) digits.
How do I do this?

In Python 3.2+, you can use int.to_bytes:
If you don't want to specify the size
>>> n = 1245427
>>> n.to_bytes((n.bit_length() + 7) // 8, 'big') or b'\0'
b'\x13\x00\xf3'
If you don't mind specifying the size
>>> (1245427).to_bytes(3, byteorder='big')
b'\x13\x00\xf3'

You can use the struct module:
import struct
print(struct.pack('>I', your_int))
'>I' is a format string. > means big endian and I means unsigned int. Check the documentation for more format chars.

This is fast and works for small and (arbitrary) large ints:
def Dump(n):
s = '%x' % n
if len(s) & 1:
s = '0' + s
return s.decode('hex')
print repr(Dump(1245427)) #: '\x13\x00\xf3'

Probably the best way is via the built-in struct module:
>>> import struct
>>> x = 1245427
>>> struct.pack('>BH', x >> 16, x & 0xFFFF)
'\x13\x00\xf3'
>>> struct.pack('>L', x)[1:] # could do it this way too
'\x13\x00\xf3'
Alternatively -- and I wouldn't usually recommend this, because it's mistake-prone -- you can do it "manually" by shifting and the chr() function:
>>> x = 1245427
>>> chr((x >> 16) & 0xFF) + chr((x >> 8) & 0xFF) + chr(x & 0xFF)
'\x13\x00\xf3'
Out of curiosity, why do you only want three bytes? Usually you'd pack such an integer into a full 32 bits (a C unsigned long), and use struct.pack('>L', 1245427) but skip the [1:] step?

def tost(i):
result = []
while i:
result.append(chr(i&0xFF))
i >>= 8
result.reverse()
return ''.join(result)

Single-source Python 2/3 compatible version based on #pts' answer:
#!/usr/bin/env python
import binascii
def int2bytes(i):
hex_string = '%x' % i
n = len(hex_string)
return binascii.unhexlify(hex_string.zfill(n + (n & 1)))
print(int2bytes(1245427))
# -> b'\x13\x00\xf3'

The shortest way, I think, is the following:
import struct
val = 0x11223344
val = struct.unpack("<I", struct.pack(">I", val))[0]
print "%08x" % val
This converts an integer to a byte-swapped integer.

Using the bitstring module:
>>> bitstring.BitArray(uint=1245427, length=24).bytes
'\x13\x00\xf3'
Note though that for this method you need to specify the length in bits of the bitstring you are creating.
Internally this is pretty much the same as Alex's answer, but the module has a lot of extra functionality available if you want to do more with your data.

Very easy with pwntools , the tools created for software hacking
(Un-ironically, I stumbled across this thread and tried solutions here, until I realised there exists conversion functionality in pwntools)
import pwntools
x2 = p32(x1)