Develop Reference

Read and write to port with unit8 format?

I have Matlab code that communicates over serial port, and I am trying to translate the code into python, however I am not getting the same "read" messages.
Here is the matlab code:
s = serial('COM3','BaudRate',115200,'InputBufferSize',1e6,'Timeout',2); %OPEN COM PORT
fopen(s);
string=[];
while(length(st)<1)
fwrite(s,30,'uint8'); %REQUEST CONNECTION STRING
pause(0.1);
st = fread(s,5); %READ CONNECTION (5BYTES, "Ready")
disp(st)
end
fwrite(s,18,'uint8'); % START ACQUISITION
while(1)
st(1:131) = fread(s,131); .....
disp(st)
OUT:
%first disp(st)
82
101
97
100
121
%from disp(st) second time
106
85
106
59
106
61
106
0
106...
Here is my attempt of python code:
# Open serial -
import serial
import time
s = serial.Serial('COM3', baudrate=115200, timeout=2 )
s.set_buffer_size(rx_size = 1000000, tx_size = 1000000)
#serieal is open
print ("serial is open: ", s.isOpen())
s.write(30) #request connection string
time.sleep(0.1)
string = s.read(5) #read connection string (5 BYTESm "Ready)
print (string)
# start aquisition
s.write(18) #request connection string
print (s.read(131))
however the output is, OUT:
serial is open: True
b'Uj.jg'
b'jVj3j-i\xefjOj8jajJj"i\xb8j\x19j4j,j\x17\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00j\x9aj\x9aj\x8djfjkj/j\xa0j\x97jbjKj#i\xb9j\x1bj5j-j\x19\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00/\xaaUj\xe5j\xb7'
As you can see they aren't the same, so:
How do I send via pyserial a 'uint8' encoded number like in matlabs: fwrite(s,30,'uint8');
How do I read and display from pyserial similar to matlabs: st = fread(s,5);

How do I read and display from pyserial similar to matlabs: st = fread(s,5);
uint8 means 8-bit number. So 1 byte.
In python, you get a bytes object which is a sequence of bytes. You can iterate over it to get each value. - Which is exactly what you want, because that would be a value of one byte, 0-255
first = b'Uj.jg'
for i in first:
print(i)
This gives you:
85
106
46
106
103
How do I send via pyserial a 'uint8' encoded number like in matlabs: fwrite(s,30,'uint8');
You can convert your int to bytes object using int.to_bytes:
print((30).to_bytes(1, byteorder="big"))
Results in
b'\x1e'
First argument is number of bytes - in our case 1. Second argument is byte order - which is useless when we use 1 byte but it's still required.
So what you had as s.write(30) will be
s.write(b'\x1e')
or to keep the "30" thing visible just directly paste the conversion:
s.write((30).to_bytes(1, byteorder="big"))
Same for sending 18: s.write((18).to_bytes(1, byteorder="big"))

minimalmodbus : Wrong Message being sent over Serial

So i want to send this code : 01 00 14 00 58 over ModBus RTU utilizing minimalmodbus to my VIRTUAL COM Port (COM2).
So i do get the "code" but i also get other bytes before and after the code and i can seem to know where they come from and how i can resolve it.
Terminal Output Image
I did try to use another libary called pymodbus but i got the same result
PythonCode :
import serial
import minimalmodbus as mrtu
mrtu.BYTEORDER_BIG = 1
insmrt = mrtu.Instrument('COM2',1 ,mrtu.MODE_RTU,close_port_after_each_call= False,debug=True)
insmrt.precalculate_read_size= False
insmrt.clear_buffers_before_each_transaction= True
insmrt.serial.baudrate = 38400
insmrt.serial.bytesize = 8
insmrt.serial.parity = serial.PARITY_NONE
insmrt.serial.stopbits =1
insmrt.serial.timeout = 5
insmrt.handle_local_echo = None
def inscommmand():
#insmrt.write_string = "$0100140058"
insmrt.write_registers(0,[0x01,0x14,0x58])
while 1:
try:
inscommmand()
except:
continue

I don't really know what you are trying to do or what you mean exactly by send this code by Modbus but I'm afraid what the library (minimalModbus) is doing is exactly what it is supposed to.
If you call:
insmrt.write_registers(0,[0x01,0x14,0x58])
The library will build the following Modbus frame:
01 10 0000 0003 06 0001 0014 0058 9ABE
This is what each value on this frame means:
01: The Slave Address (default is address 1)
10: The Function Code 16 (Write Multiple Holding Registers, 16 = 10 hex)
0000: The address of the first register (0000 hex = 0, +40001 offset = register #40001).
0003: Number of registers to write since you are giving a 3-element list
06: Number of data bytes that follow (3 registers x 2 bytes each = 6 bytes).
0001: Value to write to register 40001
0014: Value to write to register 40002
0058: Value to write to register 40003
9ABE: The CRC (Cyclic Redundancy Check) for error checking.
The sequence 01 00 14 00 58 is not a valid Modbus frame because there is no function code 00(only values from 1 to 6, 15, and 16 are allowed).
Maybe what you want is just to send that sequence over the serial port? Otherwise, you should think about where those values are coming from.

How can I play an already streaming H264 RTSP data with VLC - dumped to file with wireshark

Backstory
I have an IP-Camera on my lan, I have connected to it using VLC (CTRL + N, adding rtsp://192.168.1.10 <- IP camera configured ip address), and after the VLC connected, i started sniffing the traffic using Wireshark.
After about 30 seconds, on Wireshark, I clicked "Follow TCP stream", and dumped the traffic coming from the camera to my computer raw.
My goal
I want to be able to take the dumped RTSP stream from Wireshark (Or other network sniffing alternatives), extract the H264 data, and be able to play it afterwards on VLC.
What I dont want to do
I dont want to dump the stream using VLC
I dont want to dump the stream using FFMPEG
Things I have tried
So I've already managed to parse the RTSP and RTP headers, and understand where the H264 payload starts, for example:
00000000 24 00 05 A0 80 60 35 85 14 0C 1F DE 43 13 B7 58
00000010 7C 81 9A 26 26 64 33 FF 7C 11 99 87 4B 15 FA 06
00000020 06 47 12 C5 6E 39 56 56 82 0D E0 1F 8D 1B 8A A7
Starting at offset 0x00000000, there is the RTSP header 24 00 05 A0. Unpacking it using python:
magic, channel, length = struct.unpack(">BBH", data)
after that, between the offsets 0x00000004 to 0x00000010, i have the RTP header, parsing as
bits_header_a, bits_header_b, sequence_number, timestamp, ssrc_identifier = struct.unpack(">BBHII", data)
and from the offset 0x00000010 I have the actual payload.
I looked at the RFC, and the great answer given here on this question but haven't quite managed to understand what I'm doing wrong next to reconstruct the H264 data from the RTSP actual payload.
Here is a part of a larger piece of code I'm executing on every payload (frame) data I previously parsed (python), to create the data frames:
first_byte = "{0:>08s}".format(bin(ord(frame[0]))[2:])
second_byte = "{0:>08s}".format(bin(ord(frame[1]))[2:])
rest_of_data = frame[2:]
# First byte
nal_unit_a = int(first_byte[:3], 2)
fragment_type = int(first_byte[3:], 2)
# Second byte
start_bit = int(second_byte[0], 2)
end_bit = int(second_byte[1], 2)
nal_unit_b = int(second_byte[3:], 2)
# Video data
if 0x1C == fragment_type:
# Middle frame, just add
if 0 == start_bit and 0 == end_bit:
total_data += rest_of_data
continue
# First frame in sequence
elif 1 == start_bit:
print " [*] Found start frame"
nal_unit = chr((nal_unit_a << 5) | nal_unit_b)
if ord(nal_unit) != ( (ord(frame[0]) & 0xe0) | (ord(frame[1]) & 0x1F)):
print "Unexpected"
total_data += "\x00\x00\x00\x01"
total_data += nal_unit
total_data += rest_of_data
continue
# End
elif 0 == start_bit and 1 == end_bit:
print " [*] Found end frame"
total_data += rest_of_data
I'm reconstructing the NALU as I think I should. I'm executing this code for every payload. But the output is not playable on VLC (Even after setting the VLC Demux module to force H264 parsing). I'm pretty sure i am missing something, as I did not fix anything related to timestamps and the H264, so there might be a problem with that. The only values I get for the nal_unit_b, and nal_unit_a are 5 3, 1 3, being reconstructed to the whole NAL unit 0x61 and 0x65.
My questions
I know that at the beginning of the RTSP session, some data is sent to know better how to parse the RTSP session (Also called SDP), But I'm pretty sure all the data should also exist in the H264 payload and it should be playable even without telling it how to parse the data, alternatively I can always make an assumption of how the data is encoded, if i know for certain what IP-Camera vendor i am communicating with. Is that correct?
Is my python code snippet above correct? (Parsing regarded)
Why on my data stream I only receive video data packets (fragment_type = 0x1C), and I dont see anything else.
When simply connecting to the IP-Camera using VLC and clicking Tools-->Codec Information, the presented codec is: Codec: H264 - MPEG-4 AVC (part 10) (h264), I assumed it should be parsed as H264, I've also tried parsing it as MPEG but failed, was I correct to parse it as H264?

RAW CAN decoding

I'm trying to import CAN data using a virtual CAN network and am getting strange results when I unpack my CAN packet of data. I'm using Python 3.3.7
Code:
import socket, sys, struct
sock = socket.socket(socket.PF_CAN, socket.SOCK_RAW, socket.CAN_RAW)
interface = "vcan0"
try:
sock.bind((interface,))
except OSError:
sys.stderr.write("Could not bind to interface '%s'\n" % interface)
fmt = "<IB3x8s"
while True:
can_pkt = sock.recv(16)
can_id, length, data = struct.unpack(fmt, can_pkt)
can_id &= socket.CAN_EFF_MASK
data = data[:length]
print(data, can_id , can_pkt)
So when I have a CAN packet looking like this.
candump vcan0: vcan0 0FF [8] 77 9C 3C 21 A2 9A B9 66
output in Python: b'\xff\x00\x00\x00\x08\x00\x00\x00w\x9c<!\xa2\x9a\xb9f'
Where vcan0 is the interface, [x] is the number of bytes in the payload, the rest is an 8 byte hex payload.
Do I have the wrong formatting? Has PF_CAN been updated for newer Python version? Am I using CAN_RAW when I should be using CAN_BCM for my protocol family? Or am I just missing how to decode the unpacked data?
Any direction or answer would be much appreciated.
Also, here are some script outputs to can-utils values I've plucked. If I can't find anything, I'm probably just going to make collect a ton of data then decode for the bytes of data that don't translate over properly. I feel that i'm over complicating things, and possibly missing one key aspect.
Python3 output == can-utils/socketCAN (hex)
M= == 4D 3D
~3 == 7E 33
p == 70
. == 2E
# == 40
r: == 0D 3A
c == 63
5g == 35 67
y == 79
a == 61
) == 29
E == 45
M == 4D
C == 43
P> == 50 3E
SGN == 53 47 4E
8 == 38

Rather than laboriously complete that table you started, just look at any ASCII code chart. When you simply print a string, any characters that are actually ASCII text will print as that character: only unprintable characters get shown as hexadecimal escapes. If you want everything in hex, you need to explicitly request that:
import binascii
print(binascii.hexlify(data))
for example.

I'm sure you've already run into the python-can library? If not we have a native python version of socketcan that correctly parse data out of CAN messages. Some of the source might help you out - or you might want to use it directly. CAN_RAW is probably what you want, if you plan on leaving virtual can for real hardware you might also want to get the timestamp from the hardware.
Not all constants have been exposed in Python's socket module so there is also a ctypes version which made in easier to experiment with things like the socketcan broadcast manager. Docs for both are here.

Python Encoding Issues

I have data that I would like to decode from and its in Windows-1252 basically I send code to a socket and it sends it back and I have to decode the message and use IEEE-754 to get a certain value from it but I can seem to figure out all this encoding stuff. Here is my code.
def printKinds ():
test = "x40\x39\x19\x99\x99\x99\x99\x9A"
print (byt1Hex(test))
test = byt1Hex(test).replace(' ', '')
struct.unpack('<d', binascii.unhexlify(test))
print (test)
printKinds()
def byt1Hex( bytStr ):
return ' '.join( [ "%02X" % ord( x ) for x in bytStr ] )
So I use that and then I have to get the value from that.. But it's not working and I can not figure out why.
The current output I am getting is
struct.unpack('<d', binascii.unhexlify(data))
struct.error: unpack requires a bytes object of length 8
That the error the expected output I am looking for is 25.1
but when I encode it, It actually changes the string into the wrong values so when I do this:
print (byt1Hex(data))
I expect to get this.
40 39 19 99 99 99 99 9A
But I actually get this instead
78 34 30 39 19 99 99 99 99 9A

>>> import struct
>>> struct.pack('!d', 25.1)
b'#9\x19\x99\x99\x99\x99\x9a'
>>> struct.unpack('!d', _) #NOTE: no need to call byt1hex, unhexlify
(25.1,)
You send, receive bytes over the network. No need hexlify/unhexlify them; unless the protocol requires it (you should mention the protocol in the question then).

You have:
test = "x40\x39\x19\x99\x99\x99\x99\x9A"
You need:
test = "\x40\x39\x19\x99\x99\x99\x99\x9A"