Understanding bit-manipulation function in Python - python

I'd really need some help understanding this piece of code: as far as I can see it is inserting a value in an array of bytes in the middle of a byte, but, as I need to reimplement this in Javascript, I'd like to fully understand it.
Array = [0] * 256
Bit_Offset = 0
[...]
def AddBits(byte_offset, bit_offset, bits, value):
global Bit_Offset
global Array
for i in range(bits):
if(value & 0x01):
Array[(byte_offset + ((bit_offset + Bit_Offset + i)/ 8))] |= (0x01 << ((bit_offset + Bit_Offset + i) % 8));
value /=2
Bit_Offset += bits
And it's used like this:
AddBits(3, 0, 8, 0xaa)
AddBits(3, 0, 3, 0xbb)
EDIT: Ok, it does insert values at custom offsets in a byte, but it looks like a very poor algorithm to me (a cycle on every bit to insert, really?). There has to be a better way to do it!
I'm not so good at binary math, but I came up with the idea of shifting the value of something like 8 - bit_offset (so that the first bit it's at the correct offset in a byte), then splitting it in parts if it spans multiple bytes and finally ORing it with the byte in the array. Is this correct? How can I implement the splitting part?

I will just go over the bit type operators.
if(value & 0x01):
This checks to see if the least significant bit is a 1 or 0. If it is a zero we don't do anything.
Array[(byte_offset + ((bit_offset + Bit_Offset + i)/ 8))]
Since all values in the array are a single byte, byte_offset is the index of where we are going to OR the bit. Added to byte_offset is an additional offset of bits from the local bit counter and the global bit counter that are then converted to bytes with the division by 8. Finally we get the index of the byte where we are going to put this bit.
((bit_offset + Bit_Offset + i) % 8)
This is calculating the bit position we are going to OR.
|= (0x01 <<
This takes our bit position and shifts a 1 to that position then ORs it with whatever is already there. We know we can use a 1 here, because the if statement will filter any 0's.
value /=2
This effectively does a bit shift to the right putting the next bit into the LSB. (Could be replaced with value >>=1 )
Bit_Offset += bits
At the end we add the bits that we added to the global offset.
What it does
This function takes Array (an array of bytes) and bit wise ORs in a value at the bit position defined by the global bit offset, argument bit offset, and the argument byte offset. The reason it does it one bit at a time is because you can potentially need to set more than one index of Array (try AddBits(3, 4, 8, 0xaa) and see what I mean).
One improvement would be to change the for loop to:
for i in range(bits):
Array[(byte_offset + ((bit_offset + Bit_Offset + i)/ 8))] |= ((value & 1) << ((bit_offset + Bit_Offset + i) % 8));
value >>=1
Edit: You could also do one byte at a time like this
def AddBits2(byte_offset, bit_offset, bits, value):
global Bit_Offset
global Array
# precompute offsets
bit_offset += Bit_Offset
byte_offset += bit_offset / 8
Bit_Offset += bits
# Mask out any extra bits in value and adjust value to align to byte
value = (((1 << bits) - 1) & value) << (bit_offset % 8)
# go through bytes and OR them in one byte at a time
for i in range((bits + 7) / 8) :
Array[byte_offset + i] |= (value & 0xff)
value >>=8

Related

Inserting a '0' bit into the middle of a bitsequence in python

This should be fairly simple, but I've yet to see a viable solution.
If I have a bit sequence (represented as an integer), how would I insert a 0 at index n?
For example:
insert(0b100101010101,4) -> 0b1001001010101
insert(0b101,3) -> 0b1010
insert(0b10001,2) -> 0b100001
EDIT: To clarify, I would like to do this without using vectors or strings (only bitwise operators)
You would need to isolate the bits to the left and to the right of the insertion point, then shift the left part one position, and combine both parts again:
def insert(n, bit):
length = n.bit_length()
if bit > length:
raise ValueError("argument out of range")
right = n & ((1 << length - bit) - 1)
return ((n - right) << 1) + right

Left shift but replace the shifted bits with ones

In Python the << operator does the following:
Returns x with the bits shifted to the left by y places (and new bits on the right-hand-side are zeros). This is the same as multiplying x by 2**y.
I want to have another version where it fills 1 on the new bits. Is there a built in function for this, if not how do I write it?
Is x = (x << y) | ((1 << y) - 1) what you're looking for?
First, we shift x left y bits:
x = 21
y = 2
bin(x) == 10101 (in binary)
x << y == 1010100
Then, the number ((1 << y) - 1) gives us a number with only the y lowest bits set to 1,
e.g.
1 << 2 == 100
(1 << 2) - 1 == 011
Finally, we or them to get 1010100 | 11 == 1010111.
There's no such function built in, or, for that matter, available in any extension library I know of.
It's pretty easy to do yourself, though. Say the int is n and you want to shift it left by s bits.
First flip all the bits of n (change all 0 bits to 1, and all 1 bits to 0). Then shift left by s. That adds s 0 bits on the right. Then flip the bits again. The new trailing 0 bits are changed to 1 bits, and the original bits of n are restored.
>>> n = 5
>>> bin(n)
'0b101'
>>> ~(~n << 6)
383
>>> bin(_)
'0b101111111'

Reverse bits program logic not clear

I was asked to write a program of reversing bits of a given 32 bits unsigned integer.
For example, if the input is 43261596, the output should be 964176192 (binary of input is 00000010100101000001111010011100, and after reversing output becomes 00111001011110000010100101000000 whose decimal equivalent is 964176192).
I wrote the following program:
class Solution:
def reverseBits(self, n: int) -> int:
binary = ""
for i in range(32):
binary = binary + str(n%2)
n = n//2
return int(binary,2)
Upon submission, this answer was received with an alternative solution that was 'claimed to be more time ans space efficient'. Here's the alternative one:
class Solution:
def reverseBits(self, n: int) -> int:
if (n == 0):
return 0
result = 0
for i in range(32):
result <<= 1
if ((n & 1) == 1):
result +=1
n >>= 1
return result
The logic they provided was left/right shifting of bits.
I am totally clueless as to how does bit shifting to the left/right help in reversing the bits of a decimal integer. To further clarify, I myself wrote this snippet:
a = 9
print(a>>1,a<<1)
The outputs were 4 and 18 respectively(integer halved and doubled).
But how does this concept contribute to revrsing bits of an integer? And also why is this considered more time and space efficient than my solution? Please help.
The bit shifting is similar to your string concatenation, binary = binary + str(n%2) effectively shifts the already existing characters to the left by one position (similar to result <<= 1). Since for the bit shift approach, this will add a 0 at the end, they need to optionally increase the value by 1 if the last binary digit of the current n is 1 (result += 1). Then n >>= 1 does the same as n //= 2.
The second approach basically requires an additional 32 bit for the result (at least theoretically), while the string approach will end up with a 32-character string which uses about 32 bytes. Also integer operations will likely be faster than string operations.

Python Bit Shifting Negative vs positive values

I was recently going over bit shifting and was wondering why in the below iPython output, shifting the value -1 vs shifting the value 4294967295 yielded different results?
In [30]: val = -1
In [31]: print "hex x %d 0x%08X" % (val, val & 0xffffffff)
hex x -1 0xFFFFFFFF
In [32]: val = val >>22
In [33]: print "hex x %d 0x%08X" % (val, val & 0xffffffff)
hex x -1 0xFFFFFFFF
In [34]: val = 4294967295
In [35]: print "hex x %d 0x%08X" % (val, val & 0xffffffff)
hex x 4294967295 0xFFFFFFFF
In [36]: val = val >>22
In [37]: print "hex x %d 0x%08X" % (val, val & 0xffffffff)
hex x 1023 0x000003FF
I would really appreciate any clarification, thanks.
4294967295 is 0x0...0FFFFFFFF. -1 is 0xF...FFFFFFFF. Forever.
Short version: Because the operation is defined that way.
Long version: per the documentation, "The rules for integer representation are intended to give the most meaningful interpretation of shift and mask operations involving negative integers." In a system where numbers can be arbitrarily sized but have 2s complement representation, it follows that - at least conceptually - the sign extension carries arbitrarily far as well. So this is really the only interpretation that makes sense.
This preserves the invariant that x >> n <=> int(x / (2 ** n)) across the negative numbers as well.
When a negative number is used in an expression, by the rules of two's complement arithmetic the sign bit must be extended all the way to the left bit of the expression. A right shift is preserving those left-most 1 bits. A negative number shifted right by a sufficient number of bits will always equal -1, just as a positive number shifted right by a sufficient number of bits will always equal 0.

Convert pseudo code to Python

Just to be fully up front, this is regarding a homework, but this in itself is not the assignment. The assignment is using noise to create cool graphics. I have a bit of experience in Python but not enough to figure this simple thing out.
I'm having trouble generating a seeded-random [-1,1]. The pseudocode my teacher gave me is from Hugo Elias.
Pseudocode:
function Noise1(integer x, integer y)
n = x + y * 57
n = (n<<13) ^ n;
return ( 1.0 - ( (n * (n * n * 15731 + 789221) + 1376312589) & 7fffffff) / 1073741824.0);
end function
My attempt in Python:
def noise(x, y):
n = x + y * 57
n = (n<<5) ^ n;
return ( 1.0 - ( (n * (n * n * 15731 + 789221) + 1376312589) & 7fffffff) / 1073741824.0)
The problem is the & 7fffffff bit in return statement. First, I'm not sure what that operation is. Maybe a bit-shift? Second, I'm not sure how to do that operation in Python. I had just removed that part, but I am getting huge negative numbers, nowhere near a [-1,1]
The & symbol stands for bitwise AND
The ^ symbol stands for bitwise XOR
and the 7FFFFFFF is a HEX number.
In programming you can represent a hex number using 0x where 7FFFFFFF is 0x7FFFFFFF
Some further reading on hex numbers.
To do a binary AND in python you just use & 0x7FFFFFFF
see more here about bitwise operations in python
I replaced 7FFFFFFF with 0x7FFFFFFF in your existing code and tried plugging in some random values and all the answers which I got were in [-1, 1].
The problem is the & 7fffffff bit in return statement. First, I'm not sure what that operation is. Maybe a bit-shift?
A bit mask. << is used for shifting. & is bitwise-AND. 7fffffff, interpreted as a hexadecimal number, is a quantity that, if re-written in binary, has 31 bits, all of which are 1. The effect is to select the low-order 31 bits of the value.
To tell Python that 7fffffff should be interpreted as a hexadecimal number, you must prefix it with 0x, hence 0x7fffffff.
Second, I'm not sure how to do that operation in Python.
The same way as in the pseudocode (i.e. with &).

Categories

Resources