Stack Overflow gods I ask for your mercy...
my error: ^[[A?????????????????????????????????????????q????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????c?????????????????????????????????????,23:24:05,01/06/2015,ourfleet
Traceback (most recent call last):
File "ourfleet.py", line 33, in <module>
result = requests.post(firebase_url + '/' + gps_location + '/gps_c.json'+ auth_token, data=json.dumps(data))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 243, in dumps
return _default_encoder.encode(obj)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 270, in iterencode
return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 0: invalid start byte
I am passing GPS data through serial from an arduino uno. This is UTF-8? Why is this happening?
My Arduino code:
#include <TinyGPS++.h>
#include <SoftwareSerial.h>
/*
This sample sketch demonstrates the normal use of a TinyGPS++ (TinyGPSPlus) object.
It requires the use of SoftwareSerial, and assumes that you have a
4800-baud serial GPS device hooked up on pins 4(rx) and 3(tx).
*/
static const int RXPin = 4, TXPin = 3;
static const uint32_t GPSBaud = 9600;
// The TinyGPS++ object
TinyGPSPlus gps;
// The serial connection to the GPS device
SoftwareSerial ss(RXPin, TXPin);
void setup()
{
Serial.begin(115200);
ss.begin(GPSBaud);
}
void loop()
{
// This sketch displays information every time a new sentence is correctly encoded.
while (ss.available() > 0)
if (gps.encode(ss.read()))
displayInfo();
if (millis() > 5000 && gps.charsProcessed() < 10)
{
while(true);
}
}
void displayInfo()
{
Serial.print(gps.location.lat(), 6);
Serial.print(F(","));
Serial.print(gps.location.lng(), 6);
Serial.println();
}
My Python code that sends the data through https. I am also getting an error that is saying the connection is not https and, instead, it is http. I have tried to update my Python libraries using pip etc etc to no avail. I am using an 07 MacBook pro)
import serial
import time
import requests
import json
firebase_url = 'https://ourfleet.firebaseio.com'
auth_token = '0sBNZjz4uQvLteDoCZTDUD3RNOT852aKyqahGzl4'
#Connect to Serial Port for communication
ser = serial.Serial('/dev/tty.wchusbserial410', 9600, timeout=0)
#Setup a loop to send GPS values at fixed intervals
#in seconds
fixed_interval = 10
while 1:
try:
#GPS value obtained from Arduino + Ublox
gps_c = ser.readline()
#current time and date
time_hhmmss = time.strftime('%H:%M:%S')
date_mmddyyyy = time.strftime('%d/%m/%Y')
#current location name
gps_location = 'ourfleet'
print gps_c + ',' + time_hhmmss + ',' + date_mmddyyyy + ',' + gps_location
#insert record
data = {'date':date_mmddyyyy,'time':time_hhmmss,'location':gps_c}
result = requests.post(firebase_url + '/' + gps_location + '/gps_c.json'+ auth_token, data=json.dumps(data))
#insert record
print 'Record inserted. Result Code = ' + str(result.status_code) + ',' + result.text
time.sleep(fixed_interval)
except IOError:
print('Error! Something went wrong.')
time.sleep(fixed_interval)
I am lacking the know how on this subject to modify my programs accordingly.
In your Python code, the first character of the data being read from the serial port is '\xff' in which case json.dumps() will throw the exception shown:
>>> json.dumps({'gps_c': '\xffblah'})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.7/json/__init__.py", line 243, in dumps
return _default_encoder.encode(obj)
File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 0: invalid start byte
The data read from the serial port looks corrupted - it should be 14 bytes long (6 lat + 6 lon + ',' + '\n'), however, the data printed shows that it is far longer than that.
I suspect that the baud rates are not properly matched. In your C/C++ code you set one port to 155200 and another (SoftwareSerial) to 9600. The Python code expects 9600. Are you sure that you have configured the correct speeds?
Related
I am trying to send a file from python client to a c# server and present it on screen by saving it first and then showing it on my MainWindow.
I came across a couple of problems I can't figure out why happen (I'm new to C#)
I followed this guide : http://snippetbank.blogspot.com/2014/04/csharp-client-server-file-transfer-example-1.html
The problems were :
1. The file was not being saved to my folder.
2. when I used message box to try and detect if it passes all the info it looks like it gets stuck in the middle .
I've been stuck on this for quite some time now but can't figure out what I'm missing
Python code :
def send_file(conn, name):
try:
full_path = "Output/"+name
file_to_send = open(full_path, "rb")
size = os.path.getsize(full_path)
file_name = name + "\n"
size_to_send = str(size) + "\n"
conn.send(size_to_send.encode())
conn.send(file_name.encode())
while size > 0:
data = file_to_send.read(1024)
conn.send(data)
size -= len(data)
file_to_send.close()
return conn
except IOError:
print("FILE IS ALREADY OPEN")
C# CODE :
public static string ReceiveFile(StreamReader reader, TcpClient tcpClient)
{
string folder = #"C:\Users\Roy\Desktop\GUI243\GUI243\";
// The first message from the client is the file size
string cmdFileSize = reader.ReadLine();
MessageBox.Show(cmdFileSize);
// The first message from the client is the filename
string cmdFileName = reader.ReadLine();
MessageBox.Show(cmdFileName);
string full_path = folder + cmdFileName;
int length = Convert.ToInt32(cmdFileSize);
byte[] buffer = new byte[length];
int received = 0;
int read = 0;
int size = 1024;
int remaining = 0;
// Read bytes from the client using the length sent from the client
while (received < length)
{
remaining = length - received;
if (remaining < size)
{
size = remaining;
}
read = tcpClient.GetStream().Read(buffer, received, size);
if (read == 0)
{
break;
}
received += read;
}
// Save the file using the filename sent by the client
using (FileStream fStream = new FileStream(Path.GetFileName(cmdFileName), FileMode.Create))
{
fStream.Write(buffer, 0, buffer.Length);
fStream.Flush();
fStream.Close();
}
return full_path;
}
In your C# code looks like a while cycle is missing, unless you call the function ReceiveFile() iterating somewhere else.
The class StreamReader needs to constantly check the tcp client socket to see if new data has been received because this is how a TCP stream works.
You don't know when the client will connect and send the data so you can't just call the ReceiveFile() function once.
In the example there is a perpetual while (true) cycle that makes that reader.Readline() work:
while (true)
{
// Accept a TcpClient
TcpClient tcpClient = tcpListener.AcceptTcpClient();
Console.WriteLine("Connected to client");
StreamReader reader = new StreamReader(tcpClient.GetStream());
// The first message from the client is the file size
string cmdFileSize = reader.ReadLine();
...
}
Do you have an equivalent in your code?
Also, is not pratical to use MessageBox for debugging purposes.
Try:
Debug.WriteLine ("Expected size is: " + cmdFileSize);
Note you need System.Diagnostics to use that.
I would like to control an actuator with a python script in MODBUS RTU
master. I tried to use the library minimalmodbus to communicate (write
bit, write & read registers) with my slave.
When I start my code, I have some errors. So, someone can I help me to find a solution?
My code:
import minimalmodbus
import os
import struct
import sys
import serial
import time
instrument = minimalmodbus.Instrument('/dev/ttyRS485', 1)
instrument.serial.port
instrument.serial.baudrate = 9600
instrument.serial.parity = serial.PARITY_NONE
instrument.serial.bytesize = 8
instrument.serial.stopbits = 1
instrument.mode = minimalmodbus.MODE_RTU
instrument.serial.timeout = 0.05
modbus = instrument.write_bit(0x0427, 1)
print (modbus)
alarme = instrument.write_bit(0x0404, 1)
print (alarme)
alarme = instrument.write_bit(0x0404, 0)
print (alarme)
on = instrument.write_bit(0x0403, 1)
print (on)
home = instrument.write_bit(0x040B, 1)
print (home)
position = instrument.write_register(0x9900, 0, number_of_decimals=2,functioncode=16, signed=False)
print (position)
posi = instrument.write_register(0x9901, 6000, number_of_decimals=2,functioncode=16, signed=False)
print (posi)
Errors:
========================= RESTART: /home/pi/test.py =========================
None
None
None
None
None
None
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.5/site-packages/minimalmodbus.py", line 2448, in _pack
result = struct.pack(formatstring, value)
struct.error: 'H' format requires 0 <= number <= 65535
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/test.py", line 36, in <module>
posi = instrument.write_register(0x9901, 6000, number_of_decimals=2, functioncode=16, signed=False)
File "/home/pi/.local/lib/python3.5/site-packages/minimalmodbus.py",line 518, in write_register
payloadformat=_PAYLOADFORMAT_REGISTER,
File "/home/pi/.local/lib/python3.5/site-packages/minimalmodbus.py",line 1166, in _generic_command
payloadformat,
File "/home/pi/.local/lib/python3.5/site-packages/minimalmodbus.py",line 1514, in _create_payload
value, number_of_decimals, signed=signed
File "/home/pi/.local/lib/python3.5/site-packages/minimalmodbus.py", line 1991, in
_num_to_twobyte_string outstring = _pack(formatcode, integer)
File "/home/pi/.local/lib/python3.5/site-packages/minimalmodbus.py", line 2454, in _pack
raise ValueError(errortext.format(value, formatstring))
ValueError: The value to send is probably out of range, as the num-to-bytestring conversion failed.
Value: 600000 Struct format code is: >H
In response to your request in the comments for an alternative library, here is what I use to read modbus with the pymodbus library:
import pymodbus
from pymodbus.pdu import ModbusRequest
from pymodbus.client.sync import ModbusSerialClient as ModbusClient
from pymodbus.transaction import ModbusRtuFramer
client = ModbusClient(
method = 'rtu'
,port='/dev/tty.usbserial-AQ00BYCR'
,baudrate=38400
,parity = 'O'
,timeout=1
)
connection = client.connect()
registers = client.read_holding_registers(0,100,unit=1)# start_address, count, slave_id
print (registers.registers)
Note that in the above, the reading begins from address 0 and continues to address 100, for slave_id 1.
To write registers, do the following:
write = client.write_register(1,425,unit=1)# address = 1, value to set = 425, slave ID = 1
New in python, I am reading JSON object from the server, size of JSON object is not fix. I am getting data from the server as per the buffer size given in socket.recv(1024). How to check that JSON object from server-socket received is full/complete, Because while parsing that JSON I am getting an error.
Please note that my JSON object is not nested.
**** Sample code ****
def get_data():
s = socket.socket()
host = 'IP_Address'
port = 'Port_Number'
# connection to hostname on the port.
s.connect((host, port))
msg=''
while(True):
msg = s.recv(1024)
print(msg.decode('ascii'))
jsonObject=json.loads(msg.decode('ascii'))
s.close()
Below is the error
Traceback (most recent call last):
File "d:/xxxxxxxxxxxxx/Python_Test.py", line 26, in <module>
get_data()
File "d:/xxxxxxxxxxxxx/Python_Test.py", line 20, in get_data
temp=json.loads(msg.decode('ascii'))
File "xxxxxxxxxxxxx\Python\Python37\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "xxxxxxxxxxxxx\Python\Python37\lib\json\decoder.py", line 340, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 777)
You recieve 1024 bytes in every loop and if your json object is bigger than that you have to handle uncompleted json string.
Also you may have two json object in 1024 bytes or even more. you can change your code to below code
def get_data():
s = socket.socket()
host = 'IP_Address'
port = 'Port_Number'
s.connect((host, port))
msg=''
while True:
r = s.recv(1024)
msg += r.decode('ascii')
while True:
start = msg.find("{")
end = msg.find("}")
if start==-1 or end==-1: # if can not find both { and } in string
break
jsonObject=json.loads(msg[start:end+1]) # only read { ... } and not another uncompleted data
# do whatever you want with jsonObject here
msg = msg[end+1:]
s.close()
NOTE : this code work correct only if you have not any nested json in your data (like this: {"device_id": {"another_json": "something"}})
You need to wait till the complete json is received and then parse it, something like this should work:
msg = ''
while(True):
response = s.recv(1024)
if not response:
break
msg += response
jsonObject=json.loads(msg.decode('ascii'))
Working on a script to poll a smart plug and parse the json response for further use.
This works perfectly:
while True:
try:
ip = "192.168.1.130"
port = 9999
sock_tcp = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock_tcp.connect((ip, port))
cmd_emeter = commands['emeter']
sock_tcp.send(encrypt(cmd_emeter))
data_emeter = sock_tcp.recv(2048)
sock_tcp.close()
# decrypt, parse, and relay emeter response
emeter = decrypt(data_emeter[4:])
parsed_json_emeter = json.loads(emeter)
voltage = parsed_json_emeter['emeter']['get_realtime']['voltage']
power = parsed_json_emeter['emeter']['get_realtime']['power']
print voltage
print power
publish.single("myHome/smartplug1/voltage", voltage, hostname="192.168.1.120")
publish.single("myHome/smartplug1/power", power, hostname="192.168.1.120")
time.sleep(5)
except socket.error:
quit("Cound not connect to host " + ip + ":" + str(port))
This does not:
while True:
try:
ip = "192.168.1.130"
port = 9999
sock_tcp = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock_tcp.connect((ip, port))
cmd_emeter = commands['emeter']
sock_tcp.send(encrypt(cmd_emeter))
data_emeter = sock_tcp.recv(2048)
cmd_info = commands['info']
sock_tcp.send(encrypt(cmd_info))
data_info = sock_tcp.recv(2048)
#cmd_daystats = commands['daystats']
#sock_tcp.send(encrypt(cmd_daystats))
#data_daystats = sock_tcp.recv(2048)
sock_tcp.close()
# decrypt, parse, and relay emeter response
emeter = decrypt(data_emeter[4:])
parsed_json_emeter = json.loads(emeter)
voltage = parsed_json_emeter['emeter']['get_realtime']['voltage']
power = parsed_json_emeter['emeter']['get_realtime']['power']
print voltage
print power
publish.single("myHome/smartplug1/voltage", voltage, hostname="192.168.1.120")
publish.single("myHome/smartplug1/power", power, hostname="192.168.1.120")
# decrypt, parse, and relay info response
info = decrypt(data_info[4:])
parsed_json_info = json.loads(info)
relay_state = parsed_json_info['system']['get_sysinfo']['relay_state']
print relay_state
#publish.single("myHome/smartplug1/voltage", voltage, hostname="192.168.1.120")
#publish.single("myHome/smartplug1/power", power, hostname="192.168.1.120")
time.sleep(5)
except socket.error:
quit("Cound not connect to host " + ip + ":" + str(port))
I need help figuring out why. Perhaps some sort of buffer issue with the sock_tcp.recv(2048) lines.
Here is the error message:
pi#raspberrypi:/usr/share/openhab/scripts $ python HS110-relay.py
119.622975
0
Traceback (most recent call last):
File "HS110-relay.py", line 89, in <module>
parsed_json_info = json.loads(info)
File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Thanks.
Baobab
I am having an issue with JSON, I can't seem to figure out why this is not working. This is supposed to output JSON.
Here is my code
#!/usr/bin/env python
import socket
import struct
import json
def unpack_varint(s):
d = 0
i = 0
while True:
b = ord(s.recv(1))
d |= (b & 0x7F) << 7*i
i += 1
if not b & 0x80:
return d
def pack_data(d):
return struct.pack('>b', len(d)) + d
def pack_port(i):
return struct.pack('>H', i)
def get_info(host, port=25565):
# Connect
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
# Send handshake + status request
s.send(pack_data("\x00\x00" + pack_data(host.encode('utf8')) + pack_port(port) + "\x01"))
s.send(pack_data("\x00"))
# Read response
unpack_varint(s) # Packet length
unpack_varint(s) # Packet ID
l = unpack_varint(s) # String length
d = ""
while len(d) < l:
d += s.recv(1024)
# Close our socket
s.close()
# Load json and return
return json.loads(d.decode('utf8'))
get_info('162.213.43.124');
I am getting this error
Traceback (most recent call last):
File "main.py", line 46, in
get_info('162.213.43.124');
File "main.py", line 45, in get_info
return json.loads(d.decode('utf8'))
File "/usr/local/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python2.7/json/decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python2.7/json/decoder.py", line 383, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
If anyone could come to the rescue that would be awesome!
It seems that you have invalid JSON. In that case, that's totally dependent on the data the server sends you which you have not shown. I would suggest running the response through a JSON validator.