I'm trying to import CAN data using a virtual CAN network and am getting strange results when I unpack my CAN packet of data. I'm using Python 3.3.7
Code:
import socket, sys, struct
sock = socket.socket(socket.PF_CAN, socket.SOCK_RAW, socket.CAN_RAW)
interface = "vcan0"
try:
sock.bind((interface,))
except OSError:
sys.stderr.write("Could not bind to interface '%s'\n" % interface)
fmt = "<IB3x8s"
while True:
can_pkt = sock.recv(16)
can_id, length, data = struct.unpack(fmt, can_pkt)
can_id &= socket.CAN_EFF_MASK
data = data[:length]
print(data, can_id , can_pkt)
So when I have a CAN packet looking like this.
candump vcan0: vcan0 0FF [8] 77 9C 3C 21 A2 9A B9 66
output in Python: b'\xff\x00\x00\x00\x08\x00\x00\x00w\x9c<!\xa2\x9a\xb9f'
Where vcan0 is the interface, [x] is the number of bytes in the payload, the rest is an 8 byte hex payload.
Do I have the wrong formatting? Has PF_CAN been updated for newer Python version? Am I using CAN_RAW when I should be using CAN_BCM for my protocol family? Or am I just missing how to decode the unpacked data?
Any direction or answer would be much appreciated.
Also, here are some script outputs to can-utils values I've plucked. If I can't find anything, I'm probably just going to make collect a ton of data then decode for the bytes of data that don't translate over properly. I feel that i'm over complicating things, and possibly missing one key aspect.
Python3 output == can-utils/socketCAN (hex)
M= == 4D 3D
~3 == 7E 33
p == 70
. == 2E
# == 40
r: == 0D 3A
c == 63
5g == 35 67
y == 79
a == 61
) == 29
E == 45
M == 4D
C == 43
P> == 50 3E
SGN == 53 47 4E
8 == 38
Rather than laboriously complete that table you started, just look at any ASCII code chart. When you simply print a string, any characters that are actually ASCII text will print as that character: only unprintable characters get shown as hexadecimal escapes. If you want everything in hex, you need to explicitly request that:
import binascii
print(binascii.hexlify(data))
for example.
I'm sure you've already run into the python-can library? If not we have a native python version of socketcan that correctly parse data out of CAN messages. Some of the source might help you out - or you might want to use it directly. CAN_RAW is probably what you want, if you plan on leaving virtual can for real hardware you might also want to get the timestamp from the hardware.
Not all constants have been exposed in Python's socket module so there is also a ctypes version which made in easier to experiment with things like the socketcan broadcast manager. Docs for both are here.
Related
So i want to send this code : 01 00 14 00 58 over ModBus RTU utilizing minimalmodbus to my VIRTUAL COM Port (COM2).
So i do get the "code" but i also get other bytes before and after the code and i can seem to know where they come from and how i can resolve it.
Terminal Output Image
I did try to use another libary called pymodbus but i got the same result
PythonCode :
import serial
import minimalmodbus as mrtu
mrtu.BYTEORDER_BIG = 1
insmrt = mrtu.Instrument('COM2',1 ,mrtu.MODE_RTU,close_port_after_each_call= False,debug=True)
insmrt.precalculate_read_size= False
insmrt.clear_buffers_before_each_transaction= True
insmrt.serial.baudrate = 38400
insmrt.serial.bytesize = 8
insmrt.serial.parity = serial.PARITY_NONE
insmrt.serial.stopbits =1
insmrt.serial.timeout = 5
insmrt.handle_local_echo = None
def inscommmand():
#insmrt.write_string = "$0100140058"
insmrt.write_registers(0,[0x01,0x14,0x58])
while 1:
try:
inscommmand()
except:
continue
I don't really know what you are trying to do or what you mean exactly by send this code by Modbus but I'm afraid what the library (minimalModbus) is doing is exactly what it is supposed to.
If you call:
insmrt.write_registers(0,[0x01,0x14,0x58])
The library will build the following Modbus frame:
01 10 0000 0003 06 0001 0014 0058 9ABE
This is what each value on this frame means:
01: The Slave Address (default is address 1)
10: The Function Code 16 (Write Multiple Holding Registers, 16 = 10 hex)
0000: The address of the first register (0000 hex = 0, +40001 offset = register #40001).
0003: Number of registers to write since you are giving a 3-element list
06: Number of data bytes that follow (3 registers x 2 bytes each = 6 bytes).
0001: Value to write to register 40001
0014: Value to write to register 40002
0058: Value to write to register 40003
9ABE: The CRC (Cyclic Redundancy Check) for error checking.
The sequence 01 00 14 00 58 is not a valid Modbus frame because there is no function code 00(only values from 1 to 6, 15, and 16 are allowed).
Maybe what you want is just to send that sequence over the serial port? Otherwise, you should think about where those values are coming from.
Backstory
I have an IP-Camera on my lan, I have connected to it using VLC (CTRL + N, adding rtsp://192.168.1.10 <- IP camera configured ip address), and after the VLC connected, i started sniffing the traffic using Wireshark.
After about 30 seconds, on Wireshark, I clicked "Follow TCP stream", and dumped the traffic coming from the camera to my computer raw.
My goal
I want to be able to take the dumped RTSP stream from Wireshark (Or other network sniffing alternatives), extract the H264 data, and be able to play it afterwards on VLC.
What I dont want to do
I dont want to dump the stream using VLC
I dont want to dump the stream using FFMPEG
Things I have tried
So I've already managed to parse the RTSP and RTP headers, and understand where the H264 payload starts, for example:
00000000 24 00 05 A0 80 60 35 85 14 0C 1F DE 43 13 B7 58
00000010 7C 81 9A 26 26 64 33 FF 7C 11 99 87 4B 15 FA 06
00000020 06 47 12 C5 6E 39 56 56 82 0D E0 1F 8D 1B 8A A7
Starting at offset 0x00000000, there is the RTSP header 24 00 05 A0. Unpacking it using python:
magic, channel, length = struct.unpack(">BBH", data)
after that, between the offsets 0x00000004 to 0x00000010, i have the RTP header, parsing as
bits_header_a, bits_header_b, sequence_number, timestamp, ssrc_identifier = struct.unpack(">BBHII", data)
and from the offset 0x00000010 I have the actual payload.
I looked at the RFC, and the great answer given here on this question but haven't quite managed to understand what I'm doing wrong next to reconstruct the H264 data from the RTSP actual payload.
Here is a part of a larger piece of code I'm executing on every payload (frame) data I previously parsed (python), to create the data frames:
first_byte = "{0:>08s}".format(bin(ord(frame[0]))[2:])
second_byte = "{0:>08s}".format(bin(ord(frame[1]))[2:])
rest_of_data = frame[2:]
# First byte
nal_unit_a = int(first_byte[:3], 2)
fragment_type = int(first_byte[3:], 2)
# Second byte
start_bit = int(second_byte[0], 2)
end_bit = int(second_byte[1], 2)
nal_unit_b = int(second_byte[3:], 2)
# Video data
if 0x1C == fragment_type:
# Middle frame, just add
if 0 == start_bit and 0 == end_bit:
total_data += rest_of_data
continue
# First frame in sequence
elif 1 == start_bit:
print " [*] Found start frame"
nal_unit = chr((nal_unit_a << 5) | nal_unit_b)
if ord(nal_unit) != ( (ord(frame[0]) & 0xe0) | (ord(frame[1]) & 0x1F)):
print "Unexpected"
total_data += "\x00\x00\x00\x01"
total_data += nal_unit
total_data += rest_of_data
continue
# End
elif 0 == start_bit and 1 == end_bit:
print " [*] Found end frame"
total_data += rest_of_data
I'm reconstructing the NALU as I think I should. I'm executing this code for every payload. But the output is not playable on VLC (Even after setting the VLC Demux module to force H264 parsing). I'm pretty sure i am missing something, as I did not fix anything related to timestamps and the H264, so there might be a problem with that. The only values I get for the nal_unit_b, and nal_unit_a are 5 3, 1 3, being reconstructed to the whole NAL unit 0x61 and 0x65.
My questions
I know that at the beginning of the RTSP session, some data is sent to know better how to parse the RTSP session (Also called SDP), But I'm pretty sure all the data should also exist in the H264 payload and it should be playable even without telling it how to parse the data, alternatively I can always make an assumption of how the data is encoded, if i know for certain what IP-Camera vendor i am communicating with. Is that correct?
Is my python code snippet above correct? (Parsing regarded)
Why on my data stream I only receive video data packets (fragment_type = 0x1C), and I dont see anything else.
When simply connecting to the IP-Camera using VLC and clicking Tools-->Codec Information, the presented codec is: Codec: H264 - MPEG-4 AVC (part 10) (h264), I assumed it should be parsed as H264, I've also tried parsing it as MPEG but failed, was I correct to parse it as H264?
First of all, I know this might look like a duplicate of: ePassport reading with PN532, Keep Getting SW1 SW2 = 0x69 0x88 (Incorrect Secure Messaging Data Objects)
..but I'm one step further in the process.
I'm trying to read a MRTD (ePassport) using python and a PN7120 nfc reader. I used pyPassport 2.0 as a basis.
I know the reader is OK because I can read passports with the same device using a Android setup.
I followed the ICAO 9303 Part 11 documentation, and simulated the "worked example" (appendix D, same file).
The problem
When using a real passport, the steps "select applet", "get challenge", "do bac", and "select file" step work fine but read binary results in 69 88 (Incorrect Secure Messaging Data Objects).
When I simulate the "worked example" by injecting the ksmac/ssc I get to the exact same ProtectedAPDU outcome as stated on page 75 (AppD-7) point g.
Also, the step "select file" almost uses the exact same procedure (see def protect) with success (rAPDU 90 00).
I've compared everything in to extreme detail at least twice and really don't see where to look next. I hope someone can give some advice or insights.
The relevant log (error at the end)
Calculate Session Keys (KSenc and KSmac) using Appendix 5.1
KSenc: 3DE649F8AEA41C04FB6D4CD9043757AD
KSmac: 8C34AD61974F68CEBA3E0EAEA1456476
Calculate Send Sequence Counter
SSC: AB1D2F337FD997D6
Reading Common
Select File
APDU 00 A4 02 0C 02 [011E]
Mask class byte and pad command header
CmdHeader: 0CA4020C80000000
Pad data
Data: 011E800000000000
Encrypt data with KSenc 3DE649F8AEA41C04FB6D4CD9043757AD
EncryptedData: FF0E241E2F94B508
Build DO'87
DO87: 870901FF0E241E2F94B508
Concatenate CmdHeader and DO87
M: 0CA4020C80000000870901FF0E241E2F94B508
Compute MAC of M
Increment SSC with 1
SSC: AB1D2F337FD997D7
Concatenate SSC and M and add padding
N: AB1D2F337FD997D70CA4020C80000000870901FF0E241E2F94B5088000000000
Compute MAC over N with KSmac 8C34AD61974F68CEBA3E0EAEA1456476
CC: 22FF803EC3104336
Build DO'8E
DO8E: 8E0822FF803EC3104336
Construct and send protected APDU
ProtectedAPDU: 0CA4020C15870901FF0E241E2F94B5088E0822FF803EC310433600
[SM] - 0C A4 02 0C 15 [870901FF0E241E2F94B5088E0822FF803EC3104336] 00
[SM] - [990290008E08AAEA3B783FD6CA9D] 90 00
Receive response APDU of MRTD's chip
RAPDU: 990290008E08AAEA3B783FD6CA9DC29000
Read Binary
APDU 00 B0 00 00 [] 04
Mask class byte and pad command header
CmdHeader: 0CB0000080000000
Build DO'97
DO97: 970104
Concatenate CmdHeader and DO97
M: 0CB0000080000000970104
Compute MAC of M
Increment SSC with 1
SSC: AB1D2F337FD997D8
Concatenate SSC and M and add padding
N: AB1D2F337FD997D80CB00000800000009701048000000000
Compute MAC over N with KSmac 8C34AD61974F68CEBA3E0EAEA1456476
CC: 68DD9FD88472834A
Build DO'8E
DO8E: 8E0868DD9FD88472834A
Construct and send protected APDU
ProtectedAPDU: 0CB000000D9701048E0868DD9FD88472834A00
[SM] - 0C B0 00 00 0D [9701048E0868DD9FD88472834A] 00
[SM] - [] 69 88 //SM data objects incorrect
Thanks!!
Figured it out:
Due to a binary/hex/string conversion error (here) the SM validation step for the SELECT FILE response was skipped and thus the SSC wasn't incremented correctly.
I have data that I would like to decode from and its in Windows-1252 basically I send code to a socket and it sends it back and I have to decode the message and use IEEE-754 to get a certain value from it but I can seem to figure out all this encoding stuff. Here is my code.
def printKinds ():
test = "x40\x39\x19\x99\x99\x99\x99\x9A"
print (byt1Hex(test))
test = byt1Hex(test).replace(' ', '')
struct.unpack('<d', binascii.unhexlify(test))
print (test)
printKinds()
def byt1Hex( bytStr ):
return ' '.join( [ "%02X" % ord( x ) for x in bytStr ] )
So I use that and then I have to get the value from that.. But it's not working and I can not figure out why.
The current output I am getting is
struct.unpack('<d', binascii.unhexlify(data))
struct.error: unpack requires a bytes object of length 8
That the error the expected output I am looking for is 25.1
but when I encode it, It actually changes the string into the wrong values so when I do this:
print (byt1Hex(data))
I expect to get this.
40 39 19 99 99 99 99 9A
But I actually get this instead
78 34 30 39 19 99 99 99 99 9A
>>> import struct
>>> struct.pack('!d', 25.1)
b'#9\x19\x99\x99\x99\x99\x9a'
>>> struct.unpack('!d', _) #NOTE: no need to call byt1hex, unhexlify
(25.1,)
You send, receive bytes over the network. No need hexlify/unhexlify them; unless the protocol requires it (you should mention the protocol in the question then).
You have:
test = "x40\x39\x19\x99\x99\x99\x99\x9A"
You need:
test = "\x40\x39\x19\x99\x99\x99\x99\x9A"
I want to send some bytes via pySerial. I created virtual serial ports with socat for testing purposes:
socat PTY,link=./ptyp1,b9600 PTY,link=./ptyp2,b9600
Here's the python code:
ser = serial.Serial('./ptyp1')
x = struct.pack('B',2)
print binascii.hexlify(x) # 02
ser.write(x)
y = ser.read(2)
print binascii.hexlify(y) # 5e42
The ouput I get:
02 # x
5e42 # y
The output I expect:
02 # x
02 # y
What am I doing wrong here? Is it socat or python?
Edit:
I just noticed some other strange behavior for different x values. Here the ouput:
x = 12 => y = 5E 52 0D 0A 5E 50
x = 100 => y = 100 # why does it work here?
Solution:
The problem was that I read on the same port I wrote to. If I get it right socat "connects" the two ports as "in" and "out". So I have to read on ./ptyp2 if I write to ./ptyp1. After that, everything is fine.
The problem was that I read on the same port I wrote to. If I get it right socat "connects" the two ports as "in" and "out". So I have to read on ./ptyp2 if I write to ./ptyp1. After that, everything is fine.
I have installed socat to test your code. I have run this line :
socat PTY,link=./ptyp1,b9600 PTY,link=./ptyp2,b9600
Then, the following code works :
from binascii import hexlify
from serial import Serial, struct
ser = Serial('ptyp1')
x = struct.pack('B', 2)
print hexlify(x) # 02
ser.write(x)
y = ser.read()
print hexlify(y) # 5E
y = ser.read()
print hexlify(y) # 42
Ouput :
02
5e
42
What you seem to be getting back is the string "^B". It's possible that socat (or something else along the way) is interpreting the byte you're sending (\x02) as a control code of some sort.
Off the top of my head, Ctrl-B is the page-back mnemonic, but I'm not sure.