I'm trying to send a Numpy arrays of floats (every array has 3 members) like:
[-1.73921636 -0.47675802 -0.2068679 ]
to arduino via serial port.
it looks like that I know so little to solve this problem! I'm using these lines of code in Python:
import serial
arduino=serial.Serial('com4',9600)
while 1:
array = function(x,y,z);
data=bytearray(array)
arduino.write(data)
time.sleep(.5)
of course I can't use Arduino serial monitor, So I use a 16x2 LCD to see what I'm exactly receiving from python. All I see is random charachters (often japanese!) I think problem it with decoding numpy array because I can see the string if I put it in arduino.write(data).
Arduino code:
#include <Wire.h>
#include <LiquidCrystal_I2C.h>
char inputBuffer[32];
LiquidCrystal_I2C lcd(0x27, 16, 2);
void setup() {
Serial.begin(9600);
lcd.begin();
lcd.backlight();
}
void loop() {
if (Serial.available()) {
lcd.clear();
delay(200);
if (Serial.available() > 0) {
//lcd.print(" ");
Serial.readBytesUntil('\n', inputBuffer, 32);
//Serial.read();
lcd.setCursor(0, 0);
lcd.print(inputBuffer);
memset(inputBuffer, 0, sizeof(inputBuffer));
}
}
}
When you call bytearray in python, it converts the numpy array to an array of bytes. For example, data = bytearray(np.array([-1.73921636, -0.47675802, -0.2068679])) gives you a byte array with 24 elements (each element in the numpy array is a double which is 8 bytes each).
When you read this on the Arduino, you need to convert this array of bytes into doubles again before doing any useful operation. union would be useful for this. I do not have an LCD to verify this, but something similar should correctly read the bytes as a double.
unsigned char inputBuffer[32];
union {float f; unsigned char b[8];} num1;
union {float f; unsigned char b[8];} num2;
union {float f; unsigned char b[8];} num3;
void loop() {
if (Serial.available()) {
lcd.clear();
delay(200);
if (Serial.available() > 0) {
Serial.readBytesUntil('\n', inputBuffer, 32);
for (int i = 0; i < 8; i++) {
num1.b[i] = inputBuffer[i];
num2.b[i] = inputBuffer[8 + i];
num3.b[i] = inputBuffer[16 + i];
}
// you can call the read float numbers using num1.f, num2.f, ...
// other operations to write to the LCD go here
}
}
Also, you might want to consider sending CRC checksum at the end of the message to be safe. When you send a stream of bytes, loosing one byte can lead to really weird numbers.
Related
I'm trying to figure out, how I could achieve this:
I'm having a Python script, which in the end produces a Numpy array, an array of arrays of floats, to be more specific. I have all properly set: I can pass parameters from C to Python, launch Py functions from C, and process returned scalar values in C.
What I'm currently not being able to do, is to return such a Numpy array as a result of a Py function to C.
Could somebody probably provide me a pointer, how to achieve this?
TIA
What you need to look at is Inter Process communication (IPC). There are several ways to perform it.
You can use one of:
Files (Easy to use)
Shared memory (really fast)
Named pipes
Sockets (slow)
See Wikipedia's IPC page and find the best approach for your needs.
Here's a small working demo example (1D, not 2D array! it's not perfect, adjust to your needs).
# file ./pyscript.py
import numpy as np
# print inline
np.set_printoptions(linewidth=np.inf)
x = np.random.random(10)
print(x)
# [0.52523722 0.29053534 0.95293405 0.7966214 0.77120688 0.22154705 0.29398872 0.47186567 0.3364234 0.38107864]
~ demo.c
// file ./demo.c
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
int main()
{
int fd[2];
pipe(fd); // create pipes
char buf[4096];
pid_t pid=fork();
if (pid==0) { // child process
dup2(fd[1],1);
close(fd[0]);
close(fd[1]);
char *pyscript = "./pyscript.py";
char *args[] = {"python3", pyscript, (char*)NULL};
execv("/usr/bin/python3",args);
}
else {
int status;
close(fd[1]);
int bytes = read(fd[0], buf, sizeof(buf));
printf("Python script output: %.*s\n", bytes, buf);
char* values[10];
int count = 0;
values[count++] = &buf[1]; // ignore the '[' coming from numpy array output
char* p = buf;
while (*p) {
if (*p == ' ') {
*p = 0;
values[count++] = p + 1;
}
p++;
}
float a[10];
float f;
for (int i = 0; i < 10; i++) {
printf("%f\n", f = atof(values[i]) ); // float values
a[i] = f;
}
waitpid(pid, &status, 0);
}
return 0;
}
Sample output
# cc demo.c
# ./a.out
Python script output: [0.23286839 0.54437959 0.37798547 0.17190732 0.49473837 0.48112695 0.93113395 0.20877592 0.96032973 0.30025713]
0.23286839
0.54437959
0.232868
0.544380
0.377985
0.171907
0.494738
0.481127
0.931134
0.208776
0.960330
0.300257
a will be your desired result, an array of float.
One has to use the PyList API for decoding list objects from Python to C
https://docs.python.org/3.3/c-api/list.html?highlight=m
Solved.
I have a simple cpp server which receives a char * buffer from a python client and unpacks it in order to use the data.
the python client sends a buffer which includes 2 "different" data types - string and int.
the buffer should look like this -
which means if the client wants to send the message code 200, and the data "ok", he would have to send the buffer [2002ok].
But I have decided that the client would send the buffer as chars.
so, the buffer would look like this- [Èok]
(È = 200's ascii value, = 2's ascii value)
(edit: I don't know why, but the ASCII value of 2 cannot be shown here..)
The problem is, That when I unpack the 3 parts of the buffer, they are somehow distorted.
here is my client side (Python):
msg = chr(200) + chr(0) + chr(0) + chr(0) + chr(2) + "ok"
print(">>>>" + (msg))
sock.send((msg.encode()))
and here is my server side(CPP):
uint8_t msgCode = helpMe.getCode(client_socket);
std::cout << "The message code is " << static_cast<unsigned int>(msgCode) << std::endl;
int DataLen = helpMe.getLength(client_socket);
std::string StrData = helpMe.getString(client_socket, DataLen);
Here are the "Helper" functions I used (unpacking the data):
using std::string;
uint8_t Helper::getCode(SOCKET sc)
{
uint8_t code;
getPartFromSocket(sc, reinterpret_cast<char*>(&code), sizeof(code), 0);
return code;
}
uint32_t Helper::getLength(SOCKET sc)
{
uint32_t length;
getPartFromSocket(sc, reinterpret_cast<char*>(&length), sizeof(length), 0);
return length;
}
std::string Helper::getString(SOCKET sc, size_t length)
{
std::string s(length + 1, 0);
getPartFromSocket(sc, (char*)s.data(), length, 0);
// possible since C++17 ^
return s;
}
void Helper::getPartFromSocket(SOCKET sc, char * buffer, size_t bytesNum, int flags)
{
if (bytesNum == 0)
{
return;
}
int res = recv(sc, buffer, bytesNum, flags);
if (res == INVALID_SOCKET)
{
std::string s = "Error while recieving from socket: ";
s += std::to_string(sc);
throw std::exception(s.c_str());
}
}
the client seems to work fine - it's output is:
È ok
but the server's output, which is supposed to be -
The message code is 200
is actually
The message code is ├
Where is my mistake?
Thanks, M.
You should change the way you receive data:
void Helper::getPartFromSocket(SOCKET sc, char* buffer, size_t bytesNum, int flags);
instead of internally creating an array. Then you can do:
uint8_t Helper::getCode(SOCKET sc)
{
uint8_t code;
getPartFromSocket(sc, reinterpret_cast<char*>(&code), sizeof(code), 0);
return code;
}
uint32_t Helper::getLength(SOCKET sc)
{
uint32_t length;
getPartFromSocket(sc, reinterpret_cast<char*>(&length), sizeof(length), 0);
return length;
}
std::string Helper::getString(SOCKET sc, size_t length)
{
std::string s(length, 0);
getPartFromSocket(sc, s.data(), length, 0);
// possible since C++17 ^
return s;
}
i. e. you write the data directly to where it shall be placed. At the same time, you solve your memory leak issue...
Problem remains with endianness... You obviously write big endian on python side, but as is shown above, you'll (most likely – it's machine dependent, but big endian machines got very rare these days...) read little endian. To get independent of machine's byte order on C++ side, too, you could modify the code as follows:
uint32_t length = 0
for(unsigned int i = 0; i < sizeof(length); ++i)
{
uint8_t byte;
getPartFromSocket(sc, reinterpret_cast<char*>(&byte), sizeof(byte), 0);
// little endian tranmitted:
// length |= static_cast<uint32_t>(byte) << 8*i;
// big endian transmitted:
length |= static_cast<uint32_t>(byte) << 8*(sizeof(length) - (i + 1));
// simpler: just adjust loop variable; = 1, <= sizeof ^
}
return length;
Edit: some remarks from the comments, as these have been moved away:
Well, actually, there's already a function doing this stuff: ntohl (thanks, WhozCraig, for the hint), so you can get it much easier:
uint32_t length;
getPartFromSocket(sc, reinterpret_cast<char*>(&length), sizeof(length), 0);
return ntohl(length);
Another problem spotted during discussion, this time on python side:
sock.send((msg.encode()))
encode by default delivers an utf-8-encoded string, which is certainly not what we want in this case (200 will be converted to two bytes). Instead we need to use local machine's encoding (on a windows host, quite likely cp1252 for western Europe or cp1250 for central and eastern Europe).
how do i change the if statement in arduino in order to move the servo the same degrees as whats randomized in python?
First he's the python code:
import serial
import time
import random
arduinoData = serial.Serial('com4',9600)
while True:
low = 0; high = 180
ran_number = random.randint(low, high)
print ran_number
time.sleep(1)
arduinoData.write(ran_number)
python code is fine works with other things.
Now the arduino code, fix this:
#include <Servo.h>
Servo myservo;
int data;
int pin=9;
int pos = 0;
void setup() {
myservo.attach(9);
pinMode(pin, OUTPUT);
digitalWrite (pin, LOW);
Serial.begin(9600);
}
void loop() {
while (Serial.available()){
data = Serial.read();
}
if statement here.....
}
}
What you are looking for is the ParseInt method. Instead of using read in a loop and constructing your number, Serial.ParseInt does exactly that for you.
The correct code would be:
#include <Servo.h>
Servo myservo;
int pin = 9;
void setup() {
myservo.attach(9);
pinMode(pin, OUTPUT);
digitalWrite(pin, LOW);
Serial.setTimeout(100);
Serial.begin(9600);
}
void loop() {
int number;
if (Serial.available()) {
number = Serial.parseInt();
myservo.write(number);
delay(0.5);
}
}
Notice that I set the serial timeout period to 100ms. This is to prevent parseInt to wait too much before deciding that it has read an entire int; otherwise when receiving a value (say, 42) it waits for about 1 second waiting for some other digits.
The Python script also has a problem or two. First, you should wait for a bit after establishing the connection because every time the serial is opened the Arduino resets, so it won't read the first couple of values. You could also get a cleaner solution by printing some string from the Arduino when ready (as the last instruction of the setup function, for example) and wait for it in the Python script with readLine.
Secondly, arduinoData.write takes a byte array as input, whereas you pass an int. You need to encode such int in bytes, first converting it to a string with str and then encoding it with encode.
I got the following program working:
import serial
import time
import random
arduinoData = serial.Serial('/dev/ttyUSB0',9600)
# Let Arduino some time to reset
time.sleep(2)
while True:
low = 0; high = 180
ran_number = random.randint(low, high)
print(ran_number)
arduinoData.write(str(ran_number).encode())
time.sleep(1)
I want to use a dll function which returns AP ssid list in Python, But it takes a preallocated struct with dyamic length array inside. I don't know how to define such a structure, without knowing the returned array length in advance.
Below is how the definition looks like in the C# demo; specifically the SSID byte array length in this struct varies.
public extern static bool D300SysUI_WiFiGetAroundSsidStatus(IntPtr SSIDList, int nMaxCount);
public struct SSIDLISTNET
{
public uint ATIMWindow;
public D300SysUI.NDIS_802_11_AUTHENTICATION_MODE AuthenticationMode;
public uint BeaconPeriod;
public uint DSConfig;
public uint DwellTime;
public uint HopPattern;
public uint HopSet;
public D300SysUI.NDIS_802_11_NETWORK_INFRASTRUCTURE InfrastructureMode;
public byte[] MacAddress;
public D300SysUI.NDIS_802_11_NETWORK_TYPE NetworkTypeInUse;
public uint NumberOfItems;
public uint Privacy;
public byte[] Reserved;
public int Rssi;
public byte[] Ssid;
public uint SsidLength;
public byte[] SupportedRates;
}
Do I need to create_string_buffer long enough by estimation ? And loop through the returned buffer, byte by byte and assmebly the bytes into element by size?
If that is the right way, how do I determine the end of the dymamic arrays ? (please pardon my ignorance, I am new to ctypes/c++)
PS: Example from the C# SDK
//D300SysUI.SSIDLIST[] items= new D300SysUI.SSIDLIST[30];
//IntPtr[] ptArray = new IntPtr[1];
//ptArray[0] = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(D300SysUI.SSIDLIST)) * 30);
//bool b = D300SysUI.D300SysUI_WiFiGetAroundSsidStatus(ptArray[0], 30);
//string message = "";
//string mac = "";
//if (b)
//{
// items[0] = (D300SysUI.SSIDLIST)Marshal.PtrToStructure((IntPtr)((UInt32)ptArray[0]), typeof(D300SysUI.SSIDLIST));
// for (int i =0;i<6;i++)
// {
// mac += String.Format("{0:X2} ", items[0].MacAddress[i]);
// }
// message += string.Format("AP:{0},MAC:{1},dBm:{2} \r\n",Encoding.GetEncoding("ASCII").GetString(items[0].Ssid,0,(int)(items[0].SsidLength)),mac,items[0].Rssi);
// for (int j = 1; j < items[0].NumberOfItems; j++)
// {
// items[j] = (D300SysUI.SSIDLIST)Marshal.PtrToStructure((IntPtr)((UInt32)ptArray[0] + j * Marshal.SizeOf(typeof(D300SysUI.SSIDLIST))), typeof(D300SysUI.SSIDLIST));
// mac = "";
// for (int i = 0; i < 6; i++)
// {
// mac += String.Format("{0:X2} ", items[j].MacAddress[i]);
// }
// message += string.Format("AP:{0},MAC:{1},dBm:{2} \r\n", Encoding.GetEncoding("ASCII").GetString(items[j].Ssid, 0, (int)(items[j].SsidLength)), mac, items[j].Rssi);
// }
//}
//Marshal.FreeHGlobal(ptArray[0]);
//MessageBox.Show(message);
If you have defined SSIDLISTNET as a cTypes structure, you just allocate the desired number of these. If the maximum the API will return is 30, allocating that number of instances is a simple and straightforward solution.
SSIDlist = SSIDLISTNET * 30
If you are very constrained on memory, you can probably copy over the required actual number of items to a new list which only holds that many, and del SSIDlist to free up the memory you reserved for this list (or let it go out of scope).
As my first foray into programming with python (previous exp only with vba and very basic java), I am trying to replicate some functionality of a GUI application, osPID Frontend, written in Processing, which communicates with an Arduino sketch, osPID-Firmware, over USB serial.
The front end takes a float array, converts it to a byte array, and then sends this over the serial link.
I have been trying to test sending a byte array to the device using the following python code:
import serial
def Send_Dash(myPort):
#To_Controller()
byteSet = 1,2,3 #bytes(1),bytes(4),bytes(3)
toSend = bytearray(byteSet)
myPort.write('5')
myPort.write('1')
myPort.write(toSend)
myPort = serial.Serial('/dev/ttyUSB0',9600,8,'N',timeout=None) # open first serial port
if myPort.isOpen():
myPort.read(10)
#typeReq = bytearray('0,0')
#myPort.write(typeReq)
Send_Dash(myPort)
Unfortunately which I can connect ok, I don't seem to be able to send any successful commands. I'd like to come up with some generic function I can use to send commands to the arduino (either by replicating the Processing code in python, or doing away with the byte array structure entirely (if possible). The original code was written to allow values such as 300000 and 0.000001 to the device. I will be unlikely to send anything greater than 300, or with decimals below 0.001, but as I hope to release this open source, others might.
For reference, the relevant arduino and Processing functions are below:
The function which converts the float array to a byte array is:
byte[] floatArrayToByteArray(float[] input)
{
int len = 4*input.length;
int index=0;
byte[] b = new byte[4];
byte[] out = new byte[len];
ByteBuffer buf = ByteBuffer.wrap(b);
for(int i=0;i<input.length;i++)
{
buf.position(0);
buf.putFloat(input[i]);
for(int j=0;j<4;j++) out[j+i*4]=b[3-j];
}
return out;
}
An example of one of the functions which packs the data is:
// Sending Floating point values to the arduino
// is a huge pain. if anyone knows an easier
// way please let know. the way I'm doing it:
// - Take the 6 floats we need to send and
// put them in a 6 member float array.
// - using the java ByteBuffer class, convert
// that array to a 24 member byte array
// - send those bytes to the arduino
void Send_Dash()//To_Controller()
{
float[] toSend = new float[3];
toSend[0] = float(SPField.getText());
toSend[1] = float(InField.getText());
toSend[2] = float(OutField.getText());
Byte a = (AMLabel.valueLabel().getText()=="Manual")?(byte)0:(byte)1;
byte identifier = 1;
myPort.write(identifier);
myPort.write(a);
myPort.write(floatArrayToByteArray(toSend));
}
A more simple function is:
void Run_Profile()
{
byte[] toSend = new byte[2];
toSend[0]=8;
toSend[1]=1;
myPort.write(toSend);
}
The byte array is made into a union:
boolean ackDash = false, ackTune = false;
union { // This Data structure lets
byte asBytes[32]; // us take the byte array
float asFloat[8]; // sent from processing and
} // easily convert it to a
foo; // float array
And read by SerialReceive(), which takes action based on the data:
void SerialReceive()
{
// read the bytes sent from Processing
byte index=0;
byte identifier=0;
byte b1=255,b2=255;
boolean boolhelp=false;
while(Serial.available())
{
byte val = Serial.read();
if(index==0){
identifier = val;
Serial.println(int(val));
}
else
{
switch(identifier)
{
case 0: //information request
if(index==1) b1=val; //which info type
else if(index==2)boolhelp = (val==1); //on or off
break;
case 1: //dasboard
case 2: //tunings
case 3: //autotune
if(index==1) b1 = val;
else if(index<14)foo.asBytes[index-2] = val;
break;
case 4: //EEPROM reset
if(index==1) b1 = val;
break;
case 5: //input configuration
if (index==1)InputSerialReceiveStart();
InputSerialReceiveDuring(val, index);
break;
case 6: //output configuration
if (index==1)OutputSerialReceiveStart();
OutputSerialReceiveDuring(val, index);
break;
case 7: //receiving profile
if(index==1) b1=val;
else if(b1>=nProfSteps) profname[index-2] = char(val);
else if(index==2) proftypes[b1] = val;
else foo.asBytes[index-3] = val;
break;
case 8: //profile command
if(index==1) b2=val;
break;
default:
break;
}
}
index++;
}
//we've received the information, time to act
switch(identifier)
{
case 0: //information request
switch(b1)
{
case 0:
sendInfo = true;
sendInputConfig=true;
sendOutputConfig=true;
break;
case 1:
sendDash = boolhelp;
break;
case 2:
sendTune = boolhelp;
break;
case 3:
sendInputConfig = boolhelp;
break;
default:
break;
}
break;
case 1: //dashboard
if(index==14 && b1<2)
{
setpoint=double(foo.asFloat[0]);
//Input=double(foo.asFloat[1]); //Not used
if(b1==0)
{
output=double(foo.asFloat[2]);
}
}
break;
case 2: //Tune
if(index==14 && (b1<=1))
{
kp = double(foo.asFloat[0]); //
ki = double(foo.asFloat[1]); //
kd = double(foo.asFloat[2]); //
}
break;
case 3: //ATune
if(index==14 && (b1<=1))
{
aTuneStep = foo.asFloat[0];
aTuneNoise = foo.asFloat[1];
aTuneLookBack = (unsigned int)foo.asFloat[2];
}
break;
default:
break;
}
}
Edit:
Sorry Lesto, yes I did read the duplicate answers, but I've only just figured out how to use struct.pack (I think!).
It appears that the existing Processing code packs (up to) 6 floats into a 32 byte array, using ByteBuffer. 6 floats is 24 bytes using struct.pack - although I gather I can pad this out by adding 1s and 0s into the format string, provided I know where they should go?
Following the comments by lesto, I have had another try (but still not quite there).
import struct
import serial
from array import array
def packByteArray(formatToSend):
bytesToSend = bytearray()
#bytesToSend.extend(struct.pack('#fffff',1,0,20,0,10) #Different result?
bytesToSend.extend(struct.pack(formatToSend,1)) #request Type
bytesToSend.extend(struct.pack(formatToSend,0)) #identifier
bytesToSend.extend(struct.pack(formatToSend,20)) #Setpoint
bytesToSend.extend(struct.pack(formatToSend,0)) #Input
bytesToSend.extend(struct.pack(formatToSend,10)) #Output
return bytesToSend
def main():
myPort = serial.Serial('/dev/ttyUSB0',9600,8,'N',timeout=None)
#Check if port is open, and if so, send the byta array.
if myPort.isOpen():
thisVar = packByteArray('>f')
print len(thisVar)
myPort.write(thisVar)
if __name__ == '__main__':
main()