Python Bit Shifting Negative vs positive values - python

I was recently going over bit shifting and was wondering why in the below iPython output, shifting the value -1 vs shifting the value 4294967295 yielded different results?
In [30]: val = -1
In [31]: print "hex x %d 0x%08X" % (val, val & 0xffffffff)
hex x -1 0xFFFFFFFF
In [32]: val = val >>22
In [33]: print "hex x %d 0x%08X" % (val, val & 0xffffffff)
hex x -1 0xFFFFFFFF
In [34]: val = 4294967295
In [35]: print "hex x %d 0x%08X" % (val, val & 0xffffffff)
hex x 4294967295 0xFFFFFFFF
In [36]: val = val >>22
In [37]: print "hex x %d 0x%08X" % (val, val & 0xffffffff)
hex x 1023 0x000003FF
I would really appreciate any clarification, thanks.

4294967295 is 0x0...0FFFFFFFF. -1 is 0xF...FFFFFFFF. Forever.

Short version: Because the operation is defined that way.
Long version: per the documentation, "The rules for integer representation are intended to give the most meaningful interpretation of shift and mask operations involving negative integers." In a system where numbers can be arbitrarily sized but have 2s complement representation, it follows that - at least conceptually - the sign extension carries arbitrarily far as well. So this is really the only interpretation that makes sense.
This preserves the invariant that x >> n <=> int(x / (2 ** n)) across the negative numbers as well.

When a negative number is used in an expression, by the rules of two's complement arithmetic the sign bit must be extended all the way to the left bit of the expression. A right shift is preserving those left-most 1 bits. A negative number shifted right by a sufficient number of bits will always equal -1, just as a positive number shifted right by a sufficient number of bits will always equal 0.

Related

Reverse bits program logic not clear

I was asked to write a program of reversing bits of a given 32 bits unsigned integer.
For example, if the input is 43261596, the output should be 964176192 (binary of input is 00000010100101000001111010011100, and after reversing output becomes 00111001011110000010100101000000 whose decimal equivalent is 964176192).
I wrote the following program:
class Solution:
def reverseBits(self, n: int) -> int:
binary = ""
for i in range(32):
binary = binary + str(n%2)
n = n//2
return int(binary,2)
Upon submission, this answer was received with an alternative solution that was 'claimed to be more time ans space efficient'. Here's the alternative one:
class Solution:
def reverseBits(self, n: int) -> int:
if (n == 0):
return 0
result = 0
for i in range(32):
result <<= 1
if ((n & 1) == 1):
result +=1
n >>= 1
return result
The logic they provided was left/right shifting of bits.
I am totally clueless as to how does bit shifting to the left/right help in reversing the bits of a decimal integer. To further clarify, I myself wrote this snippet:
a = 9
print(a>>1,a<<1)
The outputs were 4 and 18 respectively(integer halved and doubled).
But how does this concept contribute to revrsing bits of an integer? And also why is this considered more time and space efficient than my solution? Please help.
The bit shifting is similar to your string concatenation, binary = binary + str(n%2) effectively shifts the already existing characters to the left by one position (similar to result <<= 1). Since for the bit shift approach, this will add a 0 at the end, they need to optionally increase the value by 1 if the last binary digit of the current n is 1 (result += 1). Then n >>= 1 does the same as n //= 2.
The second approach basically requires an additional 32 bit for the result (at least theoretically), while the string approach will end up with a 32-character string which uses about 32 bytes. Also integer operations will likely be faster than string operations.

Negative numbers in binary (ones' complement and two’s complement)

How do I create 1's complement and 2's complement code for negative numbers in Python?
I had some problems with inversion and leading zero (one) in process.
I tried to use res = bin(~int(bin(numb), 2)), but it doesn't work.
I need result like:
Input: -123
Direct code: 11111011
Ones' complement code: 10000100
Two’s complement code: 10000101
The operator - is two's complement and ~ is one's complement. The representation of both are negative numbers, because python integers has infinite digits. Positive means infinite 0s, negative infinite 1s. For display, you need to truncate the numbers to a finite amount of digits.
>>> '{:08b}'.format(-123 & 0xff)
'10000101'
>>> '{:08b}'.format(~123 & 0xff)
'10000100'
>
You can do something like this to compute the 2's compliement:
def extra(x, bits = 64):
mask = (1 << bits) - 1
if x < 0:
x = ((abs(x) ^ mask) + 1)
return bin(x)[2:]
and like this for 1's complement:
def reverse(y, bits = 64):
mask = (1 << bits) - 1
if y < 0:
y = (abs(y) ^ mask)
return bin(y)[2:]

Two's complement of Hex number in Python

Below a and b (hex), representing two's complement signed binary numbers.
For example:
a = 0x17c7cc6e
b = 0xc158a854
Now I want to know the signed representation of a & b in base 10. Sorry I'm a low level programmer and new to python; feel very stupid for asking this. I don't care about additional library's but the answer should be simple and straight forward. Background: a & b are extracted data from a UDP packet. I have no control over the format. So please don't give me an answer that would assume I can change the format of those varibles before hand.
I have converted a & b into the following with this:
aBinary = bin(int(a, 16))[2:].zfill(32) => 00010111110001111100110001101110 => 398969966
bBinary = bin(int(b, 16))[2:].zfill(32) => 11000001010110001010100001010100 => -1051154348
I was trying to do something like this (doesn't work):
if aBinary[1:2] == 1:
aBinary = ~aBinary + int(1, 2)
What is the proper way to do this in python?
why not using ctypes ?
>>> import ctypes
>>> a = 0x17c7cc6e
>>> ctypes.c_int32(a).value
398969966
>>> b = 0xc158a854
>>> ctypes.c_int32(b).value
-1051154348
A nice way to do this in Python is using bitwise operations. For example, for 32-bit values:
def s32(value):
return -(value & 0x80000000) | (value & 0x7fffffff)
Applying this to your values:
>>> s32(a)
398969966
>>> s32(b)
-1051154348
What this function does is sign-extend the value so it's correctly interpreted with the right sign and value.
Python is a bit tricky in that it uses arbitrary precision integers, so negative numbers are treated as if there were an infinite series of leading 1 bits. For example:
>>> bin(-42 & 0xff)
'0b11010110'
>>> bin(-42 & 0xffff)
'0b1111111111010110'
>>> bin(-42 & 0xffffffff)
'0b11111111111111111111111111010110'
>>> import numpy
>>> numpy.int32(0xc158a854)
-1051154348
You'll have to know at least the width of your data. For instance, 0xc158a854 has 8 hexadecimal digits so it must be at least 32 bits wide; it appears to be an unsigned 32 bit value. We can process it using some bitwise operations:
In [232]: b = 0xc158a854
In [233]: if b >= 1<<31: b -= 1<<32
In [234]: b
Out[234]: -1051154348L
The L here marks that Python 2 has switched to processing the value as a long; it's usually not important, but in this case indicates that I've been working with values outside the common int range for this installation. The tool to extract data from binary structures such as UDP packets is struct.unpack; if you just tell it that your value is signed in the first place, it will produce the correct value:
In [240]: s = '\xc1\x58\xa8\x54'
In [241]: import struct
In [242]: struct.unpack('>i', s)
Out[242]: (-1051154348,)
That assumes two's complement representation; one's complement (such as the checksum used in UDP), sign and magnitude, or IEEE 754 floating point are some less common encodings for numbers.
Another modern solution:
>>> b = 0xc158a854
>>> int.from_bytes(bytes.fromhex(hex(b)[2:]), byteorder='big', signed=True)
-1051154348
2^31 = 0x80000000 (sign bit, which in two's compliment represents -2^31)
2^31-1 = 0x7fffffff (all the positive bits)
and hence (n & 0x7fffffff) - (n & 0x80000000) will apply the sign correctly
you could even do, n - ((n & 0x80000000)<<1) to subtract the msb value twice
or finally there's, (n & 0x7fffffff) | -(n & 0x80000000) to just merge the negative bit, not subtract
def signedHex(n): return (n & 0x7fffffff) | -(n & 0x80000000)
signedHex = lambda n: (n & 0x7fffffff) | -(n & 0x80000000)
value=input("enter hexa decimal value for getting compliment values:=")
highest_value="F"*len(value)
resulting_decimal=int(highest_value,16)-int(value,16)
ones_compliment=hex(resulting_decimal)
twos_compliment=hex(r+1)
print(f'ones compliment={ones_compliment}\n twos complimet={twos_compliment}')

two's complement of numbers in python

I am writing code that will have negative and positive numbers all 16 bits long with the MSB being the sign aka two's complement. This means the smallest number I can have is -32768 which is 1000 0000 0000 0000 in two's complement form. The largest number I can have is 32767 which is 0111 1111 1111 1111.
The issue I am having is python is representing the negative numbers with the same binary notation as positive numbers just putting a minus sign out the front i.e. -16384 is displayed as -0100 0000 0000 0000 what I want to be displayed for a number like -16384 is 1100 0000 0000 0000.
I am not quite sure how this can be coded. This is the code i have. Essentially if the number is between 180 and 359 its going to be negative. I need to display this as a twos compliment value. I dont have any code on how to display it because i really have no idea how to do it.
def calculatebearingActive(i):
numTracks = trackQty_Active
bearing = (((i)*360.0)/numTracks)
if 0< bearing <=179:
FC = (bearing/360.0)
FC_scaled = FC/(2**(-16))
return int(FC_scaled)
elif 180<= bearing <=359:
FC = -1*(360-bearing)/(360.0)
FC_scaled = FC/(2**(-16))
return int(FC_scaled)
elif bearing ==360:
FC = 0
return FC
If you're doing something like
format(num, '016b')
to convert your numbers to a two's complement string representation, you'll want to actually take the two's complement of a negative number before stringifying it:
format(num if num >= 0 else (1 << 16) + num, '016b')
or take it mod 65536:
format(num % (1 << 16), '016b')
The two's complement of a value is the one's complement plus one.
You can write your own conversion function based on that:
def to_binary(value):
result = ''
if value < 0:
result = '-'
value = ~value + 1
result += bin(value)
return result
The result looks like this:
>>> to_binary(10)
'0b1010'
>>> to_binary(-10)
'-0b1010'
Edit: To display the bits without the minus in front you can use this function:
def to_twoscomplement(bits, value):
if value < 0:
value = ( 1<<bits ) + value
formatstring = '{:0%ib}' % bits
return formatstring.format(value)
>>> to_twoscomplement(16, 3)
'0000000000000011'
>>> to_twoscomplement(16, -3)
'1111111111111101'
If you actually want to store the numbers using 16 bits, you can use struct.
import struct
>>> struct.pack('h', 32767)
'\xff\x7f'
>>> struct.pack('h', -32767)
'\x01\x80'
You can unpack using unpack
>>> a = struct.pack('h', 32767)
>>> struct.unpack('H', a)
32767
Python can hold unlimited integer values, the bit representation will adopt to hold any number you put. So such technical details as two complement does not make sense in this context. In C 'b1111111111111111' means -1 for int16 and 65535 for uint16 or int32. In python it is always is 65535 as the int will adopt to hold such values.
I think this is why they opted to add - in front of negative numbers regardless of string representation (binary, oct, hex, decimal ...).
If you wish to replicate the C behaviour and get the negative represented in two complement form you have the following options:
1 int > uint > bin using numpy
The most straight forward way is to dump the value as signed limited int and read it as unsigned.
If you have access to numpy the code is pretty easy to read:
>>> bin(np.int16(-30).astype(np.uint16))
'0b1111111111100010'
>>> bin(np.int16(-1).astype(np.uint16))
'0b1111111111111111'
>>> bin(np.int16(-2).astype(np.uint16))
'0b1111111111111110'
>>> bin(np.int16(-16).astype(np.uint16))
'0b1111111111110000'
2 int > uint > bin using struct
You can do the similar think with struct but it is slightly harder to understand
>>> bin(struct.unpack('>H', struct.pack('>h', 30))[0])
'0b1111111111100010'
>>> bin(struct.unpack('>H', struct.pack('>h', -1))[0])
'0b1111111111111111'
>>> bin(struct.unpack('>H', struct.pack('>h', -2))[0])
'0b1111111111111110'
>>> bin(struct.unpack('>H', struct.pack('>h', -16))[0])
'0b1111111111110000'
Note: h is signed and H is unsigned int 16, and '>' stands for bigendian it comes handy if you want to read bytes directly without converting them back to int
3. int using struct, then read byte by byte and convert to bin
>>> ''.join(f'{byte:08b}' for byte in struct.pack('>h', -1<<15))
'1000000000000000'
>>> ''.join(f'{byte:08b}' for byte in struct.pack('>h', -1))
'1111111111111111'
>>> ''.join(f'{byte:08b}' for byte in struct.pack('>h', -2))
'1111111111111110'
>>> ''.join(f'{byte:08b}' for byte in struct.pack('>h', -16))
'1111111111110000'
Note this has some quirks, as you need to remember to enforce the byte binary representation to be 8 digits long with '08'.
3 Directly from math,
Lastly you can go check what wikipedia says about 2 complement representation and get it implemented directly from the math formula Two complement > Subtraction from 2N
>>> bin(2**16 -16)
'0b1111111111110000'
>>> bin(2**16 -3)
'0b1111111111111101'
This looks super simple but it is hard to understand if you are not versed in the way 2 complement representation works.
Since you haven't given any code examples, I can't be sure what's going on. Based on the numbers in your example, I don't think you're using bin(yourint) because you're output doesn't contain 0b. Maybe you're already slicing that off in your examples.
If you are storing your binary data as strings, you could do something like:
def handle_negatives(binary_string):
If binary_string < 0:
binary_string = '1' + str(binary_string)[1:]
Return binary_string
Here are some caveats in printing complements(1s and 2s) in binary form, in python:
UNSIGNED RANGE: 0 to (2^k)-1 for k bit number
ex: 0 to (2^32)-1 numbers
ex: 0 to 7 for 3 bit unsigned numbers (count = 8)
SIGNED RANGE: -2^(k-1) to +2^(k-1)-1 for 1+k bit number (k-1 is for dividing current range k into two equal half)
ex: -2^31 to +(2^31)-1 numbers
ex -8 to +7 for 1+3 bit signed numbers (count = 8)
bin(int)->str converts an integer to binary string
CAVEAT: 1. Since in python there is no limit to length of integer
for ~x or !x (1s_complement/not/negate) we can't determine how many bits after MSB needs to be 1
so python just prints out unsigned value of target negative number in binary format with a
'-' sign in the beginning
ex: for x = b'01010'(10) we get ~x = -0b'1011' (-11)
but we should be getting -16+5 = -11
(-b'10000'+b'00101') = -b'10101' (-11 signed) or (21 unsigned)
to get real binary value after negation(or 1s complement) one could simulate it
NOTE: 2^0 is always 1, so (2**0 == 1) in python
NOTE: (1 << k) is always 2^k (left shift is 2 raised to the power k)
ex: bin((1 << k)-1 - x) which is ((2^k)-1)-x (1s complement)
ex: bin((1 << k)-1 - x) + 1 which is (2^k)-x (2s complement)
2. Same goes for reverse parsing of signed binary string to int:
ex: int("-0b'0101'", 2) gives -5 but instead it actually is -11 assuming -0b represents all bits
from MSB till current to be like 1111......0101 which is actually -16+5 = -11
BUT due to PYTHON's limitation of representing signed binary values we need to adhere to
current way of parsing considering unsigned binary strings with sign in front for -ve numbers
# NOTE: how the significant prefix zeros doesn't matter in both +ve and -ve cases
# Byte type inputs
x = b'+1010' # valid +ve number byte string
x = b'1010' # valid +ve number byte string
x = b'-1010' # valid -ve number byte string
x = b'+01010' # valid +ve number byte string
x = b'01010' # valid +ve number byte string
x = b'-01010' # valid -ve number byte string
int(b'101') # interprets as base 10 for each digit
int(b'101', 2) # interprets as base 2 for each digit
int(b'101', 8) # interprets as base 8 for each digit
int(b'101', 10) # interprets as base 10 for each digit
int(b'101', 16) # interprets as base 16 for each digit
# String type inputs
x = '+1010' # valid +ve number string
x = '1010' # valid +ve number string
x = '-1010' # valid -ve number string
x = '+01010' # valid +ve number string
x = '01010' # valid +ve number string
x = '-01010' # valid -ve number string
int('101') # interprets as base 10 for each digit
int('101', 2) # interprets as base 2 for each digit
int('101', 8) # interprets as base 8 for each digit
int('101', 10) # interprets as base 10 for each digit
int('101', 16) # interprets as base 16 for each digit
# print(bin(int(x, 2)), int(x,2), ~int(x, 2), bin(~int(x,2)), "-"+bin((1<<k)-1 - int(x,2)))
k = 5 # no of bits
assert 2**0 == 1 # (2^0 is always 1)
_2k = (1 << k) # (2^k == left shift (2^0 or 1) by k times == multiply 2 by k times)
x = '01010' # valid +ve number string
x = int(x,2)
print("input:", x) # supposed to be 1s complement of binStr but due to python's limitation,
# we consider it -(unsigned binStr)
_1s = '-'+bin((_2k-1)-x)
print("1s complement(negate/not): ", _1s, ~x)
_2s = '-'+bin(_2k-x)
print("2s complement(1s +1): ", _2s, ~x+1)
output:
k = 5 (5 bit representation)
input: 10
1s complement(negate/not): -0b10101 -11
2s complement(1s +1): -0b10110 -10
k=32 (32 bit representation)
input: 10
1s complement(negate/not): -0b11111111111111111111111111110101 -11
2s complement(1s +1): -0b11111111111111111111111111110110 -10

Python: Creating a Single Integer From Two Integers?

I'm searching for a simple algorithm that 'combines' two 2bytes integers into one unique 4bytes integer.
The two 2bytes integers are both whole positive numbers in the range 0..65535.
I want to create one 4bytes integer that is the exact combination of both, in a way that will make it easy to:
(1) given the two 2bytes integers --> calculate the value of that 4bytes integer.
(2) given the 4bytes integer --> parse the contents of the two 2bytes integers.
Any idea how to achieve this in python?
How about:
def combine(n, m):
return (n << 16) | m
def extract(c):
return (c >> 16), c & 0xffff
This solution places one of the 2-byte integers in the upper half of a 32-bit word, and the other into the lower half. In order to extract the values, simply take the upper half of the word (c >> 16), and the lower half (c & 0xffff).
>>> i1, i2 = 345, 12
>>> i1 * 0x10000 + i2
22609932
>>> divmod(22609932, 0x10000)
(345, 12)

Categories

Resources