I notice that I can do things like 2 << 5 to get 64 and 1000 >> 2 to get 250.
Also I can use >> in print:
print >>obj, "Hello world"
What is happening here?
The >> operator in your example is used for two different purposes. In C++ terms, this operator is overloaded. In the first example, it is used as a bitwise operator (right shift),
2 << 5 # shift left by 5 bits
# 0b10 -> 0b1000000
1000 >> 2 # shift right by 2 bits
# 0b1111101000 -> 0b11111010
While in the second scenario it is used for output redirection. You use it with file objects, like this example:
with open('foo.txt', 'w') as f:
print >>f, 'Hello world' # "Hello world" now saved in foo.txt
This second use of >> only worked on Python 2. On Python 3 it is possible to redirect the output of print() using the file= argument:
with open('foo.txt', 'w') as f:
print('Hello world', file=f) # "Hello world" now saved in foo.txt
These are bitwise shift operators.
Quoting from the docs:
x << y
Returns x with the bits shifted to the left by y places (and new bits on the right-hand-side are zeros). This is the same as multiplying x by 2**y.
x >> y
Returns x with the bits shifted to the right by y places. This is the same as dividing x by 2**y.
12 << 2
48
Actual binary value of 12 is "00 1100" when we execute the above statement Left shift ( 2 places shifted left) returns the value 48 its binary value is "11 0000".
48 >> 2
12
The binary value of 48 is "11 0000", after executing above statement Right shift ( 2 places shifted right) returns the value 12 its binary value is "00 1100".
They are bit shift operator which exists in many mainstream programming languages, << is the left shift and >> is the right shift, they can be demonstrated as the following table, assume an integer only take 1 byte in memory.
| operate | bit value | octal value | description |
| ------- | --------- | ----------- | -------------------------------------------------------- |
| | 00000100 | 4 | |
| 4 << 2 | 00010000 | 16 | move all bits to left 2 bits, filled with 0 at the right |
| 16 >> 2 | 00000100 | 4 | move all bits to right 2 bits, filled with 0 at the left |
The other case involving print >>obj, "Hello World" is the "print chevron" syntax for the print statement in Python 2 (removed in Python 3, replaced by the file argument of the print() function). Instead of writing to standard output, the output is passed to the obj.write() method. A typical example would be file objects having a write() method. See the answer to a more recent question: Double greater-than sign in Python.
These are the shift operators
x << y Returns x with the bits shifted to the left by y places (and
new bits on the right-hand-side are zeros). This is the same as
multiplying x by 2**y.
x >> y Returns x with the bits shifted to the
right by y places. This is the same as //'ing x by 2**y.
Are "bitwise" operators.
https://wiki.python.org/moin/BitwiseOperators
>>> help("symbols")
+-------------------------------------------------+---------------------------------------+
| Operator | Description |
|=================================================|=======================================|
| "<<", ">>" | Shifts |
+-------------------------------------------------+---------------------------------------+
| "&" | Bitwise AND |
+-------------------------------------------------+---------------------------------------+
| "|" | Bitwise OR |
+-------------------------------------------------+---------------------------------------+
| "~x" | bitwise NOT |
+-----------------------------------------------------------------------------------------+
| "^" | Bitwise XOR |
+-------------------------------------------------+---------------------------------------+
x << y
Returns x with the bits shifted to the left by y places (and new bits on the right-hand-side are zeros). This is the same as multiplying x by 2**y.
x >> y
Returns x with the bits shifted to the right by y places. This is the same as //'ing x by 2**y.
PD: In python 3.9 the operator " | " Applied to dictionaries merges dictionaries.
https://docs.python.org/3.9/whatsnew/3.9.html
>>> x = {"key1": "value1 from x", "key2": "value2 from x"}
>>> y = {"key2": "value2 from y", "key3": "value3 from y"}
>>> x | y
{'key1': 'value1 from x', 'key2': 'value2 from y', 'key3': 'value3 from y'}
>>> y | x
{'key2': 'value2 from x', 'key3': 'value3 from y', 'key1': 'value1 from x'}
2 << 5 (shift to left)
Shifting 2 (in binary) 5 bits to the left. (inject zero to the right)
bin(16) # '0b10'
shifted = bin(2) + '0' * 5 # '0b1000000'
int(shifted, 2) # 64
2 << 5 # 64
1000 >> 2 (shift to right)
Shifting 1000 (in binary) 2 bits to the right. (inject zero to the left)
bin(1000) # '0b1111101000'
# add 00 to the left and remove last digit from the right
# '0b 00(add these bits) 11111010 00(remove these bits)'
shifted = '0b0011111010'
int(shifted, 2) # 250
1000 >> 2 # 250
I verified the following on both Python 2.7 and Python 3.8
I did print(100<<3)
Converting 100 to Binary gives 1100100.
What I did is I droped the first 3 bits and added 3 bits with the value '0' at the end.
So it should result as 0100000, and I converted this to Decimal and the answer was 32.
For my suprise when I executed print(100<<3) the answer was 800. I was puzzled.
I converted 800 to Binary to check whats going on.
And this is what I got 1100100000.
If you see how 800 was Python answer, they did not shift or drop the first 3 bits but they added value '0' to last 3 bits.
Where as print(100>>3) , worked perfect. I did manual calculation and cheked the print result from python. It worked correctly. Dropped last 3 bits and added value '0' to first 3 bits.
Looks like (100<<3) , left shift operator has a bug on Python.
<< Mean any given number will be multiply by 2the power
for exp:- 2<<2=2*2'1=4
6<<2'4=6*2*2*2*2*2=64
Related
In Python the << operator does the following:
Returns x with the bits shifted to the left by y places (and new bits on the right-hand-side are zeros). This is the same as multiplying x by 2**y.
I want to have another version where it fills 1 on the new bits. Is there a built in function for this, if not how do I write it?
Is x = (x << y) | ((1 << y) - 1) what you're looking for?
First, we shift x left y bits:
x = 21
y = 2
bin(x) == 10101 (in binary)
x << y == 1010100
Then, the number ((1 << y) - 1) gives us a number with only the y lowest bits set to 1,
e.g.
1 << 2 == 100
(1 << 2) - 1 == 011
Finally, we or them to get 1010100 | 11 == 1010111.
There's no such function built in, or, for that matter, available in any extension library I know of.
It's pretty easy to do yourself, though. Say the int is n and you want to shift it left by s bits.
First flip all the bits of n (change all 0 bits to 1, and all 1 bits to 0). Then shift left by s. That adds s 0 bits on the right. Then flip the bits again. The new trailing 0 bits are changed to 1 bits, and the original bits of n are restored.
>>> n = 5
>>> bin(n)
'0b101'
>>> ~(~n << 6)
383
>>> bin(_)
'0b101111111'
I was asked to find out the number from a list of numbers in Python that is present only once inside the list. As usual, I could easily solve it using the normal method that immediately comes out:
class Solution(object):
def singleNumber(self, nums):
"""
:type nums: List[int]
:rtype: int
"""
for i in nums:
if(nums.count(i) == 1):
return i
return nums[0]
Trying to find out another better way, Internet suggested me to use Bitwise XOR operation and provided the following code which was way faster and more efficient. Below is the code:
class Solution(object):
def singleNumber(self, nums):
"""
:type nums: List[int]
:rtype: int
"""
a = 0
for i in nums:
a^=i
print(a)
return a
The logic was if we do a XOR operation on 2 same bits(0,0 or 1,1) the answer will always be 0 and it will be 1 if the bits are different. Using this, it came out with such an approach.
My mind bogs down at trying to realize how is the normal concept of XOR operation coming to be useful here! I understand the logic they said about that same and different bits but by their code, every time I perform a XOR with the next bit a new number pops out so how can then we guarantee that only the number which occurs once will be evolved?
I feel clueless. It might not be a good approach to have posted a code snippet and then ask for an explanation but I feel the solution is extremely intriguing and am dying to learn how bitwise XOR helped out! Please help if anyone has an idea! Thanks in advance.
Note : In the list, all numbers occur twice except for one.
The general constraints for this XOR solution are a bit different:
You have a list where exactly one number occurs an odd number of times and all other numbers occur an even number of times and need to find the odd one.
XOR flips the bits if they are identical:
b1 XOR b1 > b0
b0 XOR b0 > b0
b1 XOR b0 > b1
b0 XOR b1 > b1
You have an arbitrary start value (the first number in your list), f.e.:
01010111011111101
If all other numbers occur an even number of times, you see each bit pattern at least twice.
The first time you have pairs of bits and
flip every 0,1 bit to 1
flip every 1,0 bit to 1
flip every 1,1 bit to 0
then you encounter the same number again (because number even times in it):
you have a 1 bit from last flipping and encounter a 1 bit again so you flip to 0
you have a 1 bit from last flipping and encounter a 0 bit again so you stay at 1
you have a 0 bit from last flipping and encounter a 1 bit again so you flip to 1
and you are now back at your original number because even times occuring numbers cancel out:
101010010
xor 101010010
-------------
000000000
and if you XOR any other number with 000000000 you get exactly that other number back.
The relevant properties of the XOR operation are:
It is associative, so (a ^ b) ^ c == a ^ (b ^ c)
It is commutative, so a ^ b == b ^ a
It has 0 as an identity element, so a ^ 0 == 0 ^ a == a
Every element is its own inverse, so a ^ a == 0
So consider what happens when your input list is like [3, 4, 5, 4, 3]. From the first two properties, we have that (((3 ^ 4) ^ 5) ^ 4) ^ 3 is the same as (3 ^ 3) ^ (4 ^ 4) ^ 5, from the fourth property this is the same as 0 ^ 0 ^ 5, and from the third property the result is 5. Effectively, every element which appears an even number of times cancels out with itself, leaving just the element which occurs once (or an odd number of times).
I think without knowing the properties of XOR operation, it is difficult to understand what that code does
This is the property of XOR operation
If the bits are the same, the result is 0. If the bits are different, the result is 1.
Below table will give a clear idea of the above XOR property.
+---+---+---------+
| a | b | a XOR b |
+---+---+---------+
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 0 |
+---+---+---------+
The Solution to your question is based on this concept.
An interesting property of XOR is that
when you XOR the same number by itself even number of times you get a Zero
when you XOR the same number by itself odd number of times you get the number itself
Take for example - Number 2 (2 in binary is - 0010)
Let us
Perform XOR of 2 (even no. times)
0010
0010
----
0000 <- XOR (Value is 0)
Perform XOR of 2 (odd number of times)
0010
0010
0010
----
0010 <- XOR (Value is 2)
Whatever numbers are occuring for even number of times will eventually result in a Zero and Zero XORed with the number occuring odd number of times will give you the same number (that occurs odd number of times)
And the reason why a is initialized to 0 is because - 0 XOR x will give x itself
Say x = 3
0 -> 0000
3 -> 0011
----
0011 <- XOR (Value is 3)
Edit - Based on OP's doubt
For the input arr = [1,2,2,4,1] you are asking why you are getting arbitrary values - 1,3,1,5,4
They are not arbitrary. They are the result of XOR operation between a and i.
Initial value of a = 0
First Iteration: i = 1(0001); a = 0 (0000)
i: 0000
a: 0001
----
0001 -> Decimal value is 1 stored in a
Second Iteration: i = 2(0010); a = 1 (0001)
i: 0010
a: 0001
----
0011 -> Decimal value is 3 stored in a
Third Iteration: i = 2(0010); a = 3 (0011)
i: 0010
a: 0011
----
0001 -> Decimal value is 1 stored in a
Fourth Iteration: i = 4(0100); a = 1 (0001)
i: 0100
a: 0001
----
0101 -> Decimal value is 5 stored in a
Fifth Iteration: i = 1(0001); a = 5 (0101)
i: 0001
a: 0101
----
0100 -> Decimal value is 4 stored in a
I'm a little confused by the ~ operator. Code goes below:
a = 1
~a #-2
b = 15
~b #-16
How does ~ do work?
I thought, ~a would be something like:
0001 = a
1110 = ~a
why not?
You are exactly right. It's an artifact of two's complement integer representation.
In 16 bits, 1 is represented as 0000 0000 0000 0001. Inverted, you get 1111 1111 1111 1110, which is -2. Similarly, 15 is 0000 0000 0000 1111. Inverted, you get 1111 1111 1111 0000, which is -16.
In general, ~n = -n - 1
The '~' operator is defined as:
"The bit-wise inversion of x is defined as -(x+1). It only applies to integral numbers."Python Doc - 5.5
The important part of this sentence is that this is related to 'integral numbers' (also called integers). Your example represents a 4 bit number.
'0001' = 1
The integer range of a 4 bit number is '-8..0..7'. On the other hand you could use 'unsigned integers', that do not include negative number and the range for your 4 bit number would be '0..15'.
Since Python operates on integers the behavior you described is expected. Integers are represented using two's complement. In case of a 4 bit number this looks like the following.
7 = '0111'
0 = '0000'
-1 = '1111'
-8 = '1000'
Python uses 32bit for integer representation in case you have a 32-bit OS. You can check the largest integer with:
sys.maxint # (2^31)-1 for my system
In case you would like an unsigned integer returned for you 4 bit number you have to mask.
'0001' = a # unsigned '1' / integer '1'
'1110' = ~a # unsigned '14' / integer -2
(~a & 0xF) # returns 14
If you want to get an unsigned 8 bit number range (0..255) instead just use:
(~a & 0xFF) # returns 254
It looks like I found simpler solution that does what is desired:
uint8: x ^ 0xFF
uint16: x ^ 0xFFFF
uint32: x ^ 0xFFFFFFFF
uint64: x ^ 0xFFFFFFFFFFFFFFFF
You could also use unsigned ints (for example from the numpy package) to achieve the expected behaviour.
>>> import numpy as np
>>> bin( ~ np.uint8(1))
'0b11111110'
The problem is that the number represented by the result of applying ~ is not well defined as it depends on the number of bits used to represent the original value. For instance:
5 = 101
~5 = 010 = 2
5 = 0101
~5 = 1010 = 10
5 = 00101
~5 = 11010 = 26
However, the two's complement of ~5 is the same in all cases:
two_complement(~101) = 2^3 - 2 = 6
two_complement(~0101) = 2^4 - 10 = 6
two_complement(~00101) = 2^5 - 26 = 6
And given that the two's complement is used to represent negative values, it makes sense to consider ~5 as the negative value, -6, of its complement.
So, more formally, to arrive at this result we have:
flipped zeros and ones (that's equivalent to taking the ones' complement)
taken two's complement
applied negative sign
and if x is a n-digit number:
~x = - two_complement(one_complement(x)) = - two_complement(2^n - 1 - x) = - (2^n - (2^n - 1 - x)) = - (x + 1)
I am trying to understand bitwise operators in Python 2.7 (<<, >>, &, |, ~ and ^), so my question is: what is the interpreter really seeing and executing? When:
>>> 3 >> 0
3
>>> 3 >> 1
1
and why
>>> 3 >> 2
0
and after that if you increment the second number by one the answer will continue to be 0.
They treat it as if it were a string of bits, written in twos-complement binary. But I do not understand what happens here.
Have a look at the binary representation of these numbers and imagine that there are infinite zeroes to the left:
...0000011 = 3
Now, what >> does is it shifts the binary representation to the right. This is the same as deleting the last digits:
If you shift by 0 places, nothing changes. Therefore, 3 >> 0 is 3.
If you shift by 1 place, you delete the last binary digit:
...0000011 >> 1 = ...000001 = 1
Therefore, 3 >> 1 is 1.
If you shift by 2 or more places, all of the binary ones will be deleted and you are left with only zeroes:
...0000011 >> 2 = ...00000 = 0
This is why 3 >> 2 (or more than 2) is always 0.
I've been trying to understand a Python script and I can't figure out what this assignment is doing:
data_byte2 = value >> 7 & 127
I've seen a kind of similar construct with the or operator but never with & and I have no idea at all what >> does (nothing's coming up on Google for it).
value >> 7 & 127
Is like writing:
(value >> 7) & 127 # see Python Operators Precedence
First you rightshift value by 7, then & result with 127.
127 in binary is 1111111, when you & with this number, you're clearing all bits on left of the number. For example, if you have 16 bit number:
1101011101111101
→
Shifting it 7 to the right will result in:
0000000110101110
& with 127 will keep only most right 8 bits:
0000000110101110
↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
0000000001111111 & (127)
----------------
0000000000101110
>> is the right bit-shift operator, and & is bitwise AND.
This is easier to see if you look at numbers in binary form.
>>> format(13, "08b")
'00001101'
>>> format(13 >> 1, "08b")
'00000110'
You can see that the binary digits are right-shifted by one place (this is equivalent to a division by 2, so x >> y is equivalent to x // (2 ** y)). On that basis, x >> 7 means "shift x's bits seven places to the right", or "divide x by 128".
>>> format(10, "08b")
'00001010'
>>> format(7, "08b")
'00000111'
>>> format(10 & 7, "08b")
'00000010'
Here the output includes a 1 for every bit where both inputs have 1, 0 otherwise. Given that
>>> format(127, "08b")
'01111111'
x & 127 means "take the last seven bits of x".
As >> has higher operator precedence than &, the expression
value >> 7 & 127
means "divide value by 128 then take the last seven bits".
>> is right shift operator
>>> 3 >> 1
1
binary of 3 is 00011, now shift one bit towards right the so it will be 00001 so we get answer 1
>>> 3 >> 2
0
binary of 3 is 00011, now shift two bit towards the right, so it will be 0000, so output is 0