Develop Reference

SocketCAN CAN bus Arbit-lost error increments once program started

I'm doing a project that will connect multiple subsystems (sensors, controller, etc) via CAN bus. I'm using SocketCAN and have settings as below:
root#ngtianxun-desktop:~# ip -details -statistic link show can0
5: can0: <NOARP,UP,LOWER_UP,ECHO> mtu 16 qdisc pfifo_fast state UP mode DEFAULT group default qlen 10
link/can promiscuity 0 minmtu 0 maxmtu 0
can state ERROR-ACTIVE (berr-counter tx 0 rx 0) restart-ms 10
bitrate 500000 sample-point 0.600
tq 100 prop-seg 3 phase-seg1 8 phase-seg2 8 sjw 4
RDC_CAN: tseg1 5..16 tseg2 3..8 sjw 1..4 brp 2..131072 brp-inc 2
clock 20000000
re-started bus-errors arbit-lost error-warn error-pass bus-off
0 0 337 0 0 0 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
RX: bytes packets errors dropped overrun mcast
27841616 3498429 0 0 0 0
TX: bytes packets errors dropped carrier collsns
9504120 2357958 0 0 0 0
root#ngtianxun-desktop:~#
Python scripts have been written to constantly monitor the subsystems and write to the subsystems upon request from the python end.
My question here is - Why am I seeing arbit-lost incrementing by 1 every time at an interval of about ~5 mins once my python program runs? Does it indicate any serious issue? Does it mean the data frame is lost? Is there any concern if I just let it be like this? Anyone who could help to answer and explain would be appreciated!
Worth noting: The programs started for about ~3 days. Only arbit-lost is observed, there's no re-started, bus-errors, error-warn, error-pass, and bus-off. And there are no errors, dropped, overrun, mcast, carrier, and collsns errors in TX/RX fields.

Usually you can safely ignore arbitration lost errors. It just means one message lost an arbitration in favor of another. Thankfully CAN has been made robust enough so that the "arbitration loser" will be sent again.
I recommend the following reads:
Wikipedia
SocketCAN

Python - Parse Ifconfig File

I am attempting to parse an ifconfig file that will have the following format:
Bond10G: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST> mtu 9000
inet 10.117.62.135 netmask 255.255.254.0 broadcast 10.117.63.255
ether 00:50:56:9e:89:10 txqueuelen 1000 (Ethernet)
RX packets 14315389 bytes 39499265855 (36.7 GiB)
RX errors 0 dropped 35686 overruns 0 frame 0
TX packets 13009616 bytes 38702751346 (36.0 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Bond1G: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST> mtu 1500
inet 10.117.60.135 netmask 255.255.254.0 broadcast 10.117.61.255
inet6 fe80::250:56ff:fe9e:ed0d prefixlen 64 scopeid 0x20<link>
ether 00:50:56:9e:ed:0d txqueuelen 1000 (Ethernet)
RX packets 1573455 bytes 172628399 (164.6 MiB)
RX errors 0 dropped 10946 overruns 0 frame 0
TX packets 185449 bytes 50369231 (48.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 9000
ether 00:50:56:9e:89:10 txqueuelen 1000 (Ethernet)
RX packets 13493291 bytes 39433797198 (36.7 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 13006856 bytes 38701854528 (36.0 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 9000
ether 00:50:56:9e:89:10 txqueuelen 1000 (Ethernet)
RX packets 822097 bytes 65468597 (62.4 MiB)
RX errors 0 dropped 35673 overruns 0 frame 0
TX packets 2760 bytes 896818 (875.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth2: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500
ether 00:50:56:9e:ed:0d txqueuelen 1000 (Ethernet)
RX packets 961003 bytes 127916200 (121.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 182704 bytes 49477386 (47.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth3: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500
ether 00:50:56:9e:ed:0d txqueuelen 1000 (Ethernet)
RX packets 612452 bytes 44712199 (42.6 MiB)
RX errors 0 dropped 10930 overruns 0 frame 0
TX packets 2745 bytes 891845 (870.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 3164912 bytes 12725232051 (11.8 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3164912 bytes 12725232051 (11.8 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
What I would like to get is a dictionary who's value is the interface, and the keys and values looking in this manner:
{'Bond10G' : {'mtu' : '9000', 'inet' : '10.117.62.135', 'netmask' : '255.255.254.0' # all remaining values that are space delimited},
'Bond1G' : {'mtu' : '9000', 'inet' : '10.117.60.135' # all remaining values that are space delimited} }
I have been able to split by new line to segregate each interface, however I am unsure as to how to continue. Sample code:
with open('ifconfig_file) as data:
for line in data:
temp_array = line.split("\n\n")
My logic would be (correct me if im wrong):
Split by colon to grab interface name to find the key (issue is the ether has colons in it).
While not empty line, take those values delimited by spaces and array[0] would be the key and array[1] would be the value.

You can leverage split() as well as regex. Here's an example where I parse mtu and inet:
import re
def parse_info(interface):
info = {}
mtuMatches = re.findall("mtu [0-9]*\s", interface) # find all matches for mtu
if (len(mtuMatches) > 0):
info['mtu'] = mtuMatches[0].replace("mtu", "").strip() # use the first match
inetMatches = re.findall("inet [0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\s", interface) # find all matches for inet
if (len(inetMatches) > 0):
info['inet'] = inetMatches[0].replace("inet", "").strip() # use the first match
# add more here
return info
def parse_name(interface):
parts = interface.split(":")
return parts[0] # grab the name
def parse_interface(interface):
name = parse_name(interface)
info = parse_info(interface)
return name, info
def parse_file(data):
interfaces = data.split("\n\n")
parsed = {}
for interface in interfaces:
name, info = parse_interface(interface)
parsed[name] = info
return parsed
with open('ifconfig.txt') as file:
print(parse_file(file.read()))
This is probably not the most performant way of doing it (if you want performance then maybe split by space and iterate over the result) but this is the cleanest in my opinion

Extract IPv4 address from ifconfig output using regex

Hey i need a little help trying to get the gateway IP from this using the 're' import in python. (I'm trying to get the 'broadcast' bit in eth0) can anyone help me with the regex expression. Many thanks.
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.2.15 netmask 255.255.255.0 broadcast 10.0.2.255
inet6 fe80::a00:27ff:feb4:9403 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:b4:94:03 txqueuelen 1000 (Ethernet)
RX packets 81508 bytes 94760761 (90.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 80331 bytes 134321242 (128.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Not sure why you would use regex against shell output of ifconfig which is highly susceptible to changes between distributions of *nix, bsd, etc. etc.
here is a pure python solution:
>>> import netifaces as ni
>>> ni.ifaddresses('en0')[2][0]['broadcast']
u'192.168.0.255'

Well, here it is both ways. Take your pick.
import re
s = """
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.2.15 netmask 255.255.255.0 broadcast 10.0.2.255
inet6 fe80::a00:27ff:feb4:9403 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:b4:94:03 txqueuelen 1000 (Ethernet)
RX packets 81508 bytes 94760761 (90.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 80331 bytes 134321242 (128.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
"""
# Bit values - see /usr/include/linux/if.h
IFF_UP = 1<<0
IFF_BROADCAST = 1<<1
IFF_RUNNING = 1<<6
IFF_MULTICAST = 1<<12
# Method 1 - Collect the flags integer value.
rf = re.compile(r'flags=(?P<flags>\d+)')
m = rf.search(s)
if m:
flags = int(m.group('flags'))
print('Your flags {0:d} (decimal) is {0:X}'.format(flags, flags))
# Note the bitwise and operator here:
if flags & IFF_BROADCAST:
print('BROADCAST bit is ON')
# Method 2 - Collect the flag names.
rn = re.compile(r'flags=\d+<(?P<names>.+)>')
m = rn.search(s)
if m:
names = m.group('names').split(',')
print('Your names are ', names)
if 'BROADCAST' in names:
print('BROADCAST bit is ON')

Best way to check checksum of bytes in python

I receive some packets from serial port. Packet example:
last 2 bytes
/\
[ data length ] [ data ] [packet crc]
I get crc, for example, b'w\x06'. Value is 125 (sum(b'w\x06') = 125). I calculate crc of packet: sum(data). And I get 1655. I know that 1655 is the same as b'w\06', but I do not know, how to translate it simply to 125 and compare with right checksum. What I need to convert 1655 to the same value as received packet bytes (125 or b'w\x06')? binascii/struct/something else?
Thanks

You can use int.from_bytes:
int.from_bytes(b'w\x06', 'little')
or with struct:
struct.unpack("<H", b'w\x06')[0]

parsing a text file for integers on a given line with python

I've got this test suite that outputs test results as a text file.
Here is a sample:
File Opened: Tuesday, June 26, 2016, 10:17:13 AM
File Opened: Tuesday, June 26, 2016, 10:17:29 AM
Radio Test BER LOOP BACK successful
Radio Test PAUSE successful
Radio Test BER LOOP BACK successful
File Opened: Tuesday, June 28, 2016, 10:18:11 AM
Bits received 10152
Bits in error 117
Access code bit errors 0
Packets received 49
Packets expected 2707
Packets w/ header error 0
Packets w/ CRC error 0
Packets w/ uncorr errors 0
Sync timeouts 3
==================================
Bits received 10368
Bits in error 85
Access code bit errors 0
Packets received 52
Packets expected 2758
Packets w/ header error 0
Packets w/ CRC error 0
Packets w/ uncorr errors 0
Sync timeouts 1
==================================
Bits received 10152
Bits in error 93
Access code bit errors 0
Packets received 49
Packets expected 2707
Packets w/ header error 0
Packets w/ CRC error 0
Packets w/ uncorr errors 0
Sync timeouts 3
I'm trying to extract the number after Bits received and Bits in error, and divide them to get a percentage.
Then, I'd like to plot those as a scatter plot with matplotlib.pyplot.
I'm having a hard time getting those numbers out of this file, however...I'm messing up something with how I'm parsing this.
Either way, I'm just feeling my way through this, and I'm sure I'm not doing this the most elegant way possible. This seems like such a simple task for Python and I'm surely making it much harder than it needs to be.
How would you handle this?
Thanks

Create two arrays, once for the received data and one for the error data, then just loop through the file and parse:
receivedData = []
errorData = []
with open("data.txt") as f:
for line in f:
if line.startswith("Bits received"):
receivedData.append(int(line.split()[-1]))
elif line.startswith("Bits in error"):
errorData.append(int(line.split()[-1]))
else:
#do normal stuff with other lines
pass

Another easy way would be to use the regex library re. (https://docs.python.org/2/library/re.html)
import re
pattern1 = re.compile(r'Bits received\s+(\d+)') # \d means any digit character
pattern2 = re.compile(r'Bits in error\s+(\d+)')
with open('path/file.txt', 'r') as f:
text = f.read()
received = int(pattern1.match(text).group(1))
in_error = int(pattern2.match(text).group(1))
value_of_interest = in_error/received
This approach assumes that every input file has those two lines. If that assumption can't be made, break up the matches to check for their presence:
match1 = pattern1.match(text) # re.MatchObject if the pattern is found
if match1: # None if it's not found
received = int(match1.group(1)) # re.MatchObject.group(1) is the first parenthesized group
match2 = pattern2.match(text)
if match2:
in_error = int(match2.group(1))