Develop Reference

Pycharm output adding up when using threading

I was learning about Threads so I made a simple program in sublime text.
import time
from threading import Thread
def func():
for a in range(1, 101):
print(a)
Threads = []
for i in range(25):
t = Thread(target=func)
Threads.append(t)
for i in Threads:
i.start()
for i in Threads:
i.join()
But after a few minutes, I started to get annoyed by the bad quality of autocompletion.
So I switched to Pycharm Edu and something weird happened with output. In cmd it was like this
60
60
97
68
58
59
70
71
74
95
89
68
53
92
91
92
93
99
100
89
96
and in Pycharm the output was
6545
46
47
54
76
775981
66
6555
55
608264
67
48
I don't understand what's going on.

print is in truth two distinct writes to stdout: the text, and then a newline. So print(a) is this:
sys.stdout.write(str(a))
sys.stdout.write('\n')
If now multiple Threads write at the same time, it is similar to this:
sys.stdout.write(str(a))
sys.stdout.write('\n')
sys.stdout.write(str(a))
sys.stdout.write('\n')
Or, sometimes:
sys.stdout.write(str(a))
sys.stdout.write(str(a))
sys.stdout.write('\n')
sys.stdout.write('\n')
So you get two numbers on one line and then two newlines.
The easiest fix is to join the strings & newline before using print:
def func():
for a in range(1, 101):
print(f'{a}\n', end='')
This produces the correct result.
(as to why this didn't happen in CMD: probably just luck)

Why does using `or` within an except clause not cause a SyntaxError? Is there a valid use for it?

At work, I stumbled upon an except clause with an or operator:
try:
# Do something.
except IndexError or KeyError:
# ErrorHandling
I know the exception classes should be passed as a tuple, but it bugged me that it wouldn't even cause a SyntaxError.
So first I wanted to investigate whether it actually works. And it doesn't.
>>> def with_or_raise(exc):
... try:
... raise exc()
... except IndexError or KeyError:
... print('Got ya!')
...
>>> with_or_raise(IndexError)
Got ya!
>>> with_or_raise(KeyError)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in with_or_raise
KeyError
So it did not catch the second exception, and looking at the bytecode, it becomes clearer why:
>>> import dis
>>> dis.dis(with_or_raise)
2 0 SETUP_EXCEPT 10 (to 12)
3 2 LOAD_FAST 0 (exc)
4 CALL_FUNCTION 0
6 RAISE_VARARGS 1
8 POP_BLOCK
10 JUMP_FORWARD 32 (to 44)
4 >> 12 DUP_TOP
14 LOAD_GLOBAL 0 (IndexError)
16 JUMP_IF_TRUE_OR_POP 20
18 LOAD_GLOBAL 1 (KeyError)
>> 20 COMPARE_OP 10 (exception match)
22 POP_JUMP_IF_FALSE 42
24 POP_TOP
26 POP_TOP
28 POP_TOP
5 30 LOAD_GLOBAL 2 (print)
32 LOAD_CONST 1 ('Got ya!')
34 CALL_FUNCTION 1
36 POP_TOP
38 POP_EXCEPT
40 JUMP_FORWARD 2 (to 44)
>> 42 END_FINALLY
>> 44 LOAD_CONST 0 (None)
46 RETURN_VALUE
So we can see, instruction 14 first loads the IndexError class onto the stack. Then it checks whether that value is True, which it is because of Python truthiness and finally jumps directly to instruction 20 where the exception match is done. Since instruction 18 was skipped, KeyError was never loaded onto the stack and therefore doesn't match.
I tried with Python 2.7 and 3.6, same result.
But then, why is it valid syntax? I imagine it being one of the following:
It's an artifact from a really old version of Python.
There is actually a valid use case for using or within an except clause.
It's simply a limitation of the Python parser which might have to accept any expression after the except keyword.
My vote is on 3 (given I saw some discussion about a new parser for Python) but I'm hoping someone can confirm that hypothesis. Because if it was 2 for example, I want to know that use case!
Also, I'm a bit clueless on how I'd continue that exploration. I imagine I would have to dig into CPython parser's source code but idk where to find it and maybe there's an easier way?

In except e, e can be any valid Python expression:
try1_stmt ::= "try" ":" suite
("except" [expression ["as" identifier]] ":" suite)+
...
[..] For an except clause with an expression, that expression is evaluated, and the clause matches the exception if the resulting object is “compatible” with the exception. An object is compatible with an exception if it is the class or a base class of the exception object or a tuple containing an item compatible with the exception.
https://docs.python.org/3/reference/compound_stmts.html#the-try-statement
The expression IndexError or KeyError yields the value IndexError. So this is equivalent to:
except IndexError:
...

You should use a n-tuple of types instead of a logical expression (which just returns the first non-false element):
def with_or_raise(exc):
try:
raise exc()
except (IndexError,KeyError):
print('Got ya!')

RAW CAN decoding

I'm trying to import CAN data using a virtual CAN network and am getting strange results when I unpack my CAN packet of data. I'm using Python 3.3.7
Code:
import socket, sys, struct
sock = socket.socket(socket.PF_CAN, socket.SOCK_RAW, socket.CAN_RAW)
interface = "vcan0"
try:
sock.bind((interface,))
except OSError:
sys.stderr.write("Could not bind to interface '%s'\n" % interface)
fmt = "<IB3x8s"
while True:
can_pkt = sock.recv(16)
can_id, length, data = struct.unpack(fmt, can_pkt)
can_id &= socket.CAN_EFF_MASK
data = data[:length]
print(data, can_id , can_pkt)
So when I have a CAN packet looking like this.
candump vcan0: vcan0 0FF [8] 77 9C 3C 21 A2 9A B9 66
output in Python: b'\xff\x00\x00\x00\x08\x00\x00\x00w\x9c<!\xa2\x9a\xb9f'
Where vcan0 is the interface, [x] is the number of bytes in the payload, the rest is an 8 byte hex payload.
Do I have the wrong formatting? Has PF_CAN been updated for newer Python version? Am I using CAN_RAW when I should be using CAN_BCM for my protocol family? Or am I just missing how to decode the unpacked data?
Any direction or answer would be much appreciated.
Also, here are some script outputs to can-utils values I've plucked. If I can't find anything, I'm probably just going to make collect a ton of data then decode for the bytes of data that don't translate over properly. I feel that i'm over complicating things, and possibly missing one key aspect.
Python3 output == can-utils/socketCAN (hex)
M= == 4D 3D
~3 == 7E 33
p == 70
. == 2E
# == 40
r: == 0D 3A
c == 63
5g == 35 67
y == 79
a == 61
) == 29
E == 45
M == 4D
C == 43
P> == 50 3E
SGN == 53 47 4E
8 == 38

Rather than laboriously complete that table you started, just look at any ASCII code chart. When you simply print a string, any characters that are actually ASCII text will print as that character: only unprintable characters get shown as hexadecimal escapes. If you want everything in hex, you need to explicitly request that:
import binascii
print(binascii.hexlify(data))
for example.

I'm sure you've already run into the python-can library? If not we have a native python version of socketcan that correctly parse data out of CAN messages. Some of the source might help you out - or you might want to use it directly. CAN_RAW is probably what you want, if you plan on leaving virtual can for real hardware you might also want to get the timestamp from the hardware.
Not all constants have been exposed in Python's socket module so there is also a ctypes version which made in easier to experiment with things like the socketcan broadcast manager. Docs for both are here.

Python Encoding Issues

I have data that I would like to decode from and its in Windows-1252 basically I send code to a socket and it sends it back and I have to decode the message and use IEEE-754 to get a certain value from it but I can seem to figure out all this encoding stuff. Here is my code.
def printKinds ():
test = "x40\x39\x19\x99\x99\x99\x99\x9A"
print (byt1Hex(test))
test = byt1Hex(test).replace(' ', '')
struct.unpack('<d', binascii.unhexlify(test))
print (test)
printKinds()
def byt1Hex( bytStr ):
return ' '.join( [ "%02X" % ord( x ) for x in bytStr ] )
So I use that and then I have to get the value from that.. But it's not working and I can not figure out why.
The current output I am getting is
struct.unpack('<d', binascii.unhexlify(data))
struct.error: unpack requires a bytes object of length 8
That the error the expected output I am looking for is 25.1
but when I encode it, It actually changes the string into the wrong values so when I do this:
print (byt1Hex(data))
I expect to get this.
40 39 19 99 99 99 99 9A
But I actually get this instead
78 34 30 39 19 99 99 99 99 9A

>>> import struct
>>> struct.pack('!d', 25.1)
b'#9\x19\x99\x99\x99\x99\x9a'
>>> struct.unpack('!d', _) #NOTE: no need to call byt1hex, unhexlify
(25.1,)
You send, receive bytes over the network. No need hexlify/unhexlify them; unless the protocol requires it (you should mention the protocol in the question then).

You have:
test = "x40\x39\x19\x99\x99\x99\x99\x9A"
You need:
test = "\x40\x39\x19\x99\x99\x99\x99\x9A"

Python if check error

When I use the code:
def Jack():
global PHand
if 11 or 24 or 37 or 50 in PHand:
PHand.remove(11 or 24 or 37 or 50)
PHand.append("Jack")
I get an error saying list.remove(x) x is not in PHand, my question is, shouldn't the if check prevent this error?

You're basically checking to see whether 11 is true. It's non-zero, so your if always executes. What you want is:
if 11 in PHand or 24 in PHand or 37 in PHand or 50 in Phand:
Of course, your PHand.remove always tries to remove 11 for much the same reason. You can't tell remove to remove any of those (not sure where you got the idea that would even work, it's not in any documentation I've ever seen), so you should structure it so:
if 11 in PHand:
PHand.remove(11)
PHand.append("Jack")
if 24 in PHand:
PHand.remove(24)
PHand.append("Jack")
... and so on.
Of course you'd be better off refactoring that into a loop or even a function, rather than repeating all that code.

You need to iterate over each element:
for i in (11, 24, 37, 50): # assign i to 11, then 24, then 37, then 50
if i in PHand: # check each one in PHand
PHand.remove(i) # and remove that one
PHand.append("Jack") # your code
break # end the loop. remove this to check all
Otherwise, 11 or 24 or 37 or 50 in PHand outputs 11. Try it!
>>> 11 or 24 or 37 or 50 in PHand
11
Why? the way or works, it checks if the first side is truthy. If it is, it doesn't bother evaluating the rest, since the result couldn't change. If it weren't truthy, it would move on to the next argument, and so on.
And what of the in PHand? That actually gets evaluated first, to just the last number like this:
11 or 24 or 37 or (50 in PHand)
But again, 11 short-circuits all the ors.
Long story short:
or always returns a single value, not all values at once applied to functions repeatedly or however your syntax implies.

Just another way of solving it using filter:
def Jack():
T = [11,24,37,50]
found = filter(lambda x:x in T, PHand)
if found:
[PHand.remove(x) for x in found]
PHand.append('Jack')
PHand = range(10,55)
Jack()