I receive some characters over a radio chip which I try to read out from a serial port. I can read it out fine in this Python code, which gives me this:
received: counter: 2703
received: counter: 2704
received: counter: 2705
So using the go-serial package I wrote some code to do the same in Go:
package main
import "fmt"
import "log"
import "github.com/jacobsa/go-serial/serial"
import "io"
import "encoding/hex"
func main() {
// Set up options.
options := serial.OpenOptions{
PortName: "/dev/ttyUSB0",
BaudRate: 9600,
DataBits: 7,
StopBits: 2,
MinimumReadSize: 4,
}
// Open the port.
port, err := serial.Open(options)
if err != nil {
log.Fatalf("serial.Open: %v", err)
}
defer port.Close()
for {
buf := make([]byte, 32)
n, err := port.Read(buf)
if err != nil {
if err != io.EOF {
fmt.Println("Error reading from serial port: ", err)
}
} else {
buf = buf[:n]
fmt.Println("received: ", buf)
fmt.Println("received: ", hex.EncodeToString(buf))
}
}
}
As you can see I print out the received buffer both raw AND converted from hex to string. The result is this:
received: [99 111 117 110 116 101 114 58 32 51 48 50 52 10]
received: 636f756e7465723a20333032340a
received: [99 111 117 110 116 101 114 58 32 51 48 50 53 10]
received: 636f756e7465723a20333032350a
received: [99 111 117 110 116 101 114 58 32 51 48 50 54 10]
received: 636f756e7465723a20333032360a
I guess those numbers represent counter: 2704, but as you can see the conversion to string doesn't give me the result I expect.
What am I doing wrong here? How can I convert those numbers to a string?
The text that came in is already a valid string. It's just that you have the bytes stored in buf which is []byte. To convert the existing []byte to a value of type string:
asString := string(buf)
While hex.EncodeToString returns a string, it returns the hexadecimal representation of each byte. For instance, the UTF-8 / ASCII for lowercase c, code 99 decimal, is 0x63, so the first two characters of hex.EncodeToString are 6 and 3.
(Meanwhile, you should figure out what to do with actual errors. Your code currently ignores them, after announcing any that are not io.EOF. If your device goes into an error state, you will loop over and over again getting the same error.)
I found that the solution is very simple. Instead of either of these:
fmt.Println("received: ", buf)
fmt.Println("received: ", hex.EncodeToString(buf))
I simply had to do this:
fmt.Println("received: ", string(buf))
Related
I have Matlab code that communicates over serial port, and I am trying to translate the code into python, however I am not getting the same "read" messages.
Here is the matlab code:
s = serial('COM3','BaudRate',115200,'InputBufferSize',1e6,'Timeout',2); %OPEN COM PORT
fopen(s);
string=[];
while(length(st)<1)
fwrite(s,30,'uint8'); %REQUEST CONNECTION STRING
pause(0.1);
st = fread(s,5); %READ CONNECTION (5BYTES, "Ready")
disp(st)
end
fwrite(s,18,'uint8'); % START ACQUISITION
while(1)
st(1:131) = fread(s,131); .....
disp(st)
OUT:
%first disp(st)
82
101
97
100
121
%from disp(st) second time
106
85
106
59
106
61
106
0
106...
Here is my attempt of python code:
# Open serial -
import serial
import time
s = serial.Serial('COM3', baudrate=115200, timeout=2 )
s.set_buffer_size(rx_size = 1000000, tx_size = 1000000)
#serieal is open
print ("serial is open: ", s.isOpen())
s.write(30) #request connection string
time.sleep(0.1)
string = s.read(5) #read connection string (5 BYTESm "Ready)
print (string)
# start aquisition
s.write(18) #request connection string
print (s.read(131))
however the output is, OUT:
serial is open: True
b'Uj.jg'
b'jVj3j-i\xefjOj8jajJj"i\xb8j\x19j4j,j\x17\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00j\x9aj\x9aj\x8djfjkj/j\xa0j\x97jbjKj#i\xb9j\x1bj5j-j\x19\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00/\xaaUj\xe5j\xb7'
As you can see they aren't the same, so:
How do I send via pyserial a 'uint8' encoded number like in matlabs: fwrite(s,30,'uint8');
How do I read and display from pyserial similar to matlabs: st = fread(s,5);
How do I read and display from pyserial similar to matlabs: st = fread(s,5);
uint8 means 8-bit number. So 1 byte.
In python, you get a bytes object which is a sequence of bytes. You can iterate over it to get each value. - Which is exactly what you want, because that would be a value of one byte, 0-255
first = b'Uj.jg'
for i in first:
print(i)
This gives you:
85
106
46
106
103
How do I send via pyserial a 'uint8' encoded number like in matlabs: fwrite(s,30,'uint8');
You can convert your int to bytes object using int.to_bytes:
print((30).to_bytes(1, byteorder="big"))
Results in
b'\x1e'
First argument is number of bytes - in our case 1. Second argument is byte order - which is useless when we use 1 byte but it's still required.
So what you had as s.write(30) will be
s.write(b'\x1e')
or to keep the "30" thing visible just directly paste the conversion:
s.write((30).to_bytes(1, byteorder="big"))
Same for sending 18: s.write((18).to_bytes(1, byteorder="big"))
I have a python program wherein I am using Pickle to store the object using the following:
pickle.dump(sample, open( "Pickled_files/sample.p", "wb" ))
I can extract and unpickle this object in Python using the following:
sample_extracted= pickle.load(open( "Pickled_files/sample.p", "rb" ))
However, I need to extract this object in a Golang application. Thus I need to know a way by which objects pickled using Python are extracted in Golang.
Is there a way that this can be achieved? And if yes, I would really appreciate if someone can point me to a sample reference or example.
ogórek (2) is Go library for decoding/encoding Python pickles.
A pickle saved to the file can be loaded in Go as follows:
f, err := os.Open("Pickled_files/sample.p")
d := ogórek.NewDecoder(f)
obj, err := d.Decode()
Pickle is Python specific format. AFAIK there are no pickle-parsers outside of Python. You can try to write one for Go but you will most likely only waste lots of time and mental health. On the other hand that would be an interesting project, indeed.
Anyway, instead of pickling use any language independent format, i.e. xml, json, google's protobuf or even a custom one, whatever suits your needs. Always pick tool for a job, never other way around.
Is there a way that this can be achieved?
Depends on your understanding of this. There are a lot of better options than Pickle -- even in pure Python environments. If you understand this as exchanging data between golang and Python, you should consider the following example:
You can serialize everything in Python, like
import msgpack
import msgpack_numpy as m
m.patch()
import numpy as np
data = {'string': 'Hello World',
'number': 42,
'matrix': np.random.randn(2, 3)}
with open('data.msgp', 'wb') as f:
f.write(msgpack.packb(data, use_bin_type=True))
Reading it is pretty simple
// go get -u github.com/vmihailenco/msgpack
package main
import (
"fmt"
"github.com/vmihailenco/msgpack"
"io/ioutil"
)
func main() {
buf, err := ioutil.ReadFile("data.msgp")
if err != nil {
panic(err)
}
var out map[string]interface{}
err = msgpack.Unmarshal(buf, &out)
if err != nil {
panic(err)
}
for k, v := range out {
fmt.Printf("key[%v] value[%v]\n", k, v)
}
}
This gives you
key[matrix] value[map[data:[145 106 174 12 61 187 235 63 128 225 138 214 167 154 231 191 156 205 144 51 50 116 244 191 251 147 235 33 187 149 251 63 207 56 134 174 206 146 220 63 7 23 246 148 34 235 226 63] type:[60 102 56]
kind:[] nd:true shape:[2 3]]]
key[string] value[[72 101 108 108 111 32 87 111 114 108 100]]
key[number] value[42]
All is left is converting the byte sequences into the object you would like to have.
[ Python ]
I have a string and I know it's size in memory. I want to get the number of characters (a rough estimate) will it contain.
Actual case, I want to send a report through mail, the content of the mail is exceeding the permitted size is allowed for it. I want to split the mail into multiple according to maximum size. But I don't have a way to co-relate the maximum size to number of characters in the string.
import smtplib
smtp = smtplib.SMTP('server.name')
smtp.ehlo()
max_size = smtp.esmtp_features['size']
message_data = <some string data exceeding the max_size>
# Now, how can I get the number of characters in message_data which exceeds the max_szie
Thanks,
The number of chars in a string is the size in memory in bytes to which you must deduct 37 (python 2.7 / mac os)
import sys
def estimate_chars():
"size in bytes"
s = ""
for idx in range(100):
print idx * 10, sys.getsizeof(s), len(s)
s += '1234567890'
estimate_chars()
result: (chars | bytes | len)
0 37 0
10 47 10
20 57 20
30 67 30
40 77 40
50 87 50
...
960 997 960
970 1007 970
980 1017 980
990 1027 990
sorry for my english.
My Arduino serial give 3 values like this, at 300 Hz:
-346 54 -191
-299 12 -123
-497 -214 77
-407 -55 -19
45 129 46
297 123 -197
393 71 -331
544 115 -273
515 -355 -89
510 -183 -47
Whit this python code I read and write correctly serial to file but after the while cycle do not terminate, and the shell remain open, and do not print stop:
...
ard=serial.Serial(portname,baudrate)
print"start"
while True:
x = ard.readline()
#print x
a=open(filename,'ab')
a.write(x)
a.close
print "stop"
...
I a biginner programmer, can you tell me a solution, to write serial to file and go forward.
Tanks
You're never breaking from the while loop. You should:
Add a timeout to the serial reader
When there are no bytes received, break the loop
Taking your code as a base, try something like this:
...
ard=serial.Serial(addr,baud)
ard.timeout = 1 # in seconds
print"start"
while True:
x = ard.readline()
if len(x) == 0:
break
a=open(fname,'ab')
a.write(x)
a.close
print "stop"
...
It works!
I have use ard.timeout and if condition (just if condition alone do not work).
An other question,
My arduino serial start and terminate like this:
Start
-663 -175 76
361 47 157
425 -229 -174
531 -283 -288
518 -40 -28
538 -228 206
581 188 174
445 5 176
end
It's possible to start write file after "Start" string and terminate before "end" string?
I have tried something like this but do not work:
while True:
x = ard.readline()
if x=="end":
break
#print x
a=open(fname,'ab')
a.write(x)
a.close
Blockquote
enter code here
I have data that I would like to decode from and its in Windows-1252 basically I send code to a socket and it sends it back and I have to decode the message and use IEEE-754 to get a certain value from it but I can seem to figure out all this encoding stuff. Here is my code.
def printKinds ():
test = "x40\x39\x19\x99\x99\x99\x99\x9A"
print (byt1Hex(test))
test = byt1Hex(test).replace(' ', '')
struct.unpack('<d', binascii.unhexlify(test))
print (test)
printKinds()
def byt1Hex( bytStr ):
return ' '.join( [ "%02X" % ord( x ) for x in bytStr ] )
So I use that and then I have to get the value from that.. But it's not working and I can not figure out why.
The current output I am getting is
struct.unpack('<d', binascii.unhexlify(data))
struct.error: unpack requires a bytes object of length 8
That the error the expected output I am looking for is 25.1
but when I encode it, It actually changes the string into the wrong values so when I do this:
print (byt1Hex(data))
I expect to get this.
40 39 19 99 99 99 99 9A
But I actually get this instead
78 34 30 39 19 99 99 99 99 9A
>>> import struct
>>> struct.pack('!d', 25.1)
b'#9\x19\x99\x99\x99\x99\x9a'
>>> struct.unpack('!d', _) #NOTE: no need to call byt1hex, unhexlify
(25.1,)
You send, receive bytes over the network. No need hexlify/unhexlify them; unless the protocol requires it (you should mention the protocol in the question then).
You have:
test = "x40\x39\x19\x99\x99\x99\x99\x9A"
You need:
test = "\x40\x39\x19\x99\x99\x99\x99\x9A"