I'm working on a project where the output size is very important. As my outputs are numbers between 0 and 100, I'm trying to write them as bytes (or unsigned chars).
However, I'm getting errors when trying to read them.
Here is a simple example:
test_filename='test.b'
g=(3*ones(shape=[1000])).astype('c')
g.tofile(test_filename)
with open(test_filename, "rb") as f:
bytes = f.read(1)
num = int(bytes.encode('hex'), 1)
print num
Here is the error I get, somehow the bytes.encode thingy excepts a binary string or something of that sort (not sure of course):
ValueError Traceback (most recent call last)
<ipython-input-43-310a447041fe> in <module>()
----> 1 num = int(bytes.encode('hex'), 1)
2 print num
ValueError: int() base must be >= 2 and <= 36
I should state that I would later need to read the output files in C++.
Thanks in advance,
Gil
There is some iffiness to this based on the version of python you are using.
If python2, which I assume you are using because of the print statement, the main problem you have is that you are getting a string from the read, so if the value is say 50 you would get an ascii value of 2 if you print it. You need to tell python that those bits should be in an int type not a str type and a simple cast does not do that.
I personally would use the struct package and do the following:
with open(test_filename, "rb") as f:
bytes = f.read(1)
num = struct.unpack("B", bytes)[0]
print num
Another option would be to encode the string to hex and read it in as a hex string (which looks like is what you are trying):
num = int(bytes.encode("hex_codec"), 16))
print num
One final option would be to put the string in a bytearray and pull the first byte:
num = bytearray(bytes)[0]
print num
If you are actually using python 3 this is simpler because you will get back a bytes object (if so dont name a variable bytes, very confusing). With a bytes object you can just pull the first element out which will be pulled out as an int:
num = bytes[0]
print num
Related
I ask a Measurement Device to give me some Data. At first it tells me how many bytes of data are in the storage. It is always 14. Then it gives me the data which i have to encode into hex. It is Python 2.7 can´t use newer versions. Line 6 to 10 tells the Device to give me the measured data.
Line 12 to 14 is the encoding to Hex. In other Programs it works. but when i print result(Line 14) then i get a Hex number with 13 Bytes PLUS 1 which can not be correct because it has an L et the end. I guess it is some LONG or whatever. and i dont need the last Byte. but i do think it changes the Data too, which is picked out from Line 15 and up. at first in Hex. Then it is converted into Int.
Is it possible that the L has an effect on the Data or not?
How can i fix it?
1 ap.write(b"ML\0")
rmemb = ap.read(2)
print(rmemb)
rmemb = int(rmemb)+1
5 rmem = rmemb #must be and is 14 Bytes
addmem = ("MR:%s\0" % rmem)
# addmem = ("MR:14\0")
ap.write(addmem.encode())
10 time.sleep(1)
test = ap.read(rmem)
result = hex(int(test.encode('hex'), 16))
print(result)
15 ftflash = result[12:20]
ftbg = result[20:28]
print(ftflash)
print(ftbg)
ftflash = int(ftflash, 16)
20 # print(ftflash)
ftbg = int(ftbg, 16)
# print(ftbg)
OUTPUT:
14
0x11bd5084c0b000001ce00000093L
b000001c
e0000009
Python 2 has two built-in integer types, int and long. hex returns a string representing a Python hexadecimal literal, and in Python 2, that means that longs get an L at the end, to signify that it's a long.
I have a Python code from a third party which uses Python 2.7 and create_string_buffer and string.join on ctypes tools. I want to convert the code to Python 3.8.3, but I got an error on the following part. Here's the code after I converted it to Python3 using 2to3 tools:
for dev in self.devices:
handle = libusb0.usb_open(dev)
self.handles.append(handle) # open and store handle
sn_char = create_string_buffer('\000'*16)
libusb0.usb_get_string_simple(handle, offset, sn_char, 16)
ser_num = ''.join(sn_char).split(b'\0',1)[0] # treat first null-byte as stop character
self.sn.append(ser_num)
The error I got is:
sn_char = create_string_buffer('\000'*16)
File "C:\Python\Python383\lib\ctypes\__init__.py", line 65, in create_string_buffer
raise TypeError(init)
TypeError:
I have also already tried to make an init variable in create_string_buffer to byte (sn_char = create_string_buffer(b'\000'*16), but I still got an error like this:
ser_num = ''.join(sn_char).split(b'\0',1)[0] # treat first null-byte as stop character
TypeError: sequence item 0: expected str instance, bytes found
hope will get solution in here, thanks...
When you use .split and provide parameter of type bytes, the object you work on must be also of type bytes.
You can easily solve this by adding b before the literal string:
ser_num = b''.join(sn_char).split(b'\0',1)[0] # treat first null-byte as stop character
thanks for the fast response, yea it can run, but now I get the new error at the other code which uses create_string_buffer, at this code
def write_data(self, sn, words_in):
num_words = len(words_in)
dev_num = self.sn_to_devnum(sn)
handle = self.handles[dev_num]
buf = create_string_buffer(b'\000'*(num_words*2-1))
for n in range(num_words):
buf[2*n] = chr(words_in[n] % 0x100);
buf[2*n+1] = chr(words_in[n] // 0x100);
ret = libusb0.usb_bulk_write(handle, self.in_ep, buf, num_words*2,
self.usb_write_timeout);
#wr_buf = [ord(buf[n]) for n in range(num_bytes)]
#print "write buffer = ", wr_buf
return ret;
The error is :
buf[2*n] = chr(words_in[n] % 0x100);
TypeError: one character bytes, bytearray or integer expected
Sorry if my question is repeatly and too simple, cause I new in python... thanks
ok, im fairly new to python but not programming, I know php, C, bash, etc... My question is:
How do I convert data = "b'\x16'" to binary "0001 0110" ??
im trying to read the response from an esc printer from DLE
x = 1
while x:
time.sleep(3)
ser.write("\x10\x04\x01".encode())
bytesToRead = ser.inWaiting()
data = ser.read(bytesToRead)
while data:
print(data)
data = ""
all that ends up printing is: b'\x16' i assume hex but a simple hex to bin is not working because of the b?
What you get back is a bytes object. (think: raw array of bytes) You can get the number itself from the first byte via data[0]. That will give you 0x16 as an int, which you can convert however you want.
I am trying to send a value over serial port. I am unsure how to convert it into the correct format. I tried bin() for binary and format() but it didn't work.
result = 2
ser.open()
ser.write(b'1')
time.sleep(3)
ser.write(result) # wrong format
ser.write("{0:b}".format(result)) # TypeError: unicode strings
# are not supported, please
# encode to bytes: '10'
The first write operation sends '1' as binary string. Now I want the variable to be send as binary string as well.
write() requires a bytes object.
>>> help(serial.Serial.write)
Help on function write in module serial.serialwin32:
write(self, data)
Output the given byte string over the serial port.
To convert an integer to bytes, call int.to_bytes().
>>> result = 2
>>> b = result.to_bytes(4, 'little')
>>> b
b'\x02\x00\x00\x00'
>>> # to convert back to an integer
>>> int.from_bytes(b, 'little')
2
Like this :
import binascii
def write(num):
pack = binascii.unlexlify("%04X"%num)
ser.write(pack)
Important point: Which number system used on device (8,16,32,64 bit) ?
8 Bit = 1 Byte(0-255)
16Bit = 2 Byte(0-65535)
32Bit = 4 Byte(like upside)(0-4294967295)
All ranges are UNSIGNED(look up), but float
got extra definition !
You can't type binary 1 value with keyboard :
binascii.unlexlify("%01X"%1) so equal to \x01 (of course you can use struct package)
write() method accepts a string parameter. You can convert result to string with str() builtin function like this.
result = str(result)
Python 2:
result = 2
ser.open()
ser.write(b'1')
time.sleep(3)
ser.write(str(result))
Python 3:
You have to encode string in bytes.
result = 2
ser.open()
ser.write(b'1')
time.sleep(3)
ser.write(str(result).encode('utf-8'))
I've a script which reads temperature data:
def get_temp(socket, channels):
data = {}
for ch in channels:
socket.sendall('KRDG? %s\n' % ch)
time.sleep(0.2)
temp = socket.recv(32).rstrip('\r\n')
data[ch] = float(temp)
Sometimes, the script fails on the line which converts the values to float:
File "./projector.py", line 129, in get_temp
data[ch] = float(temp)
ValueError: invalid literal for float(): +135.057E+0
+078.260E+0
+00029
but this is NOT an invalid literal. If I enter this into any python shell,
float(+135.057E+0)
then it correctly returns 135.057.
So what is the problem?
I would all but guarantee that the issue is some sort of non-printing character that's present in the value you pulled off your socket. It looks like you're using Python 2.x, in which case you can check for them with this:
print repr(temp)
You'll likely see something in there that's escaped in the form \x00. These non-printing characters don't show up when you print directly to the console, but their presence is enough to negatively impact the parsing of a string value into a float.
-- Edited for question changes --
It turns this is partly accurate for your issue - the root cause however appears to be that you're reading more information than you expect from your socket or otherwise receiving multiple values. You could do something like
map(float, temp.strip().split('\r\n'))
In order to convert each of the values, but if your function is supposed to return a single float value this is likely to cause confusion. Anyway, the issue certainly revolves around the presence of characters you did not expect to see in the value you retrieved from your socket.
I had a similar issue reading the serial output from a digital scale. I was reading [3:12] out of a 18 characters long output string.
In my case sometimes there is a null character "\x00" (NUL) which magically appears in the scale's reply string and is not printed.
I was getting the error:
> ' 0.00'
> 3 0 fast loop, delta = 10.0 weight = 0.0
> ' 0.00'
> 1 800 fast loop, delta = 10.0 weight = 0.0
> ' 0.00'
> 6 0 fast loop, delta = 10.0 weight = 0.0
> ' 0\x00.0'
> Traceback (most recent call last):
> File "measure_weight_speed.py", line 172, in start
> valueScale = float(answer_string)
> ValueError: invalid literal for float(): 0
After some research I wrote few lines of code that work in my case.
replyScale = scale_port.read(18)
answer = replyScale[3:12]
answer_decode = answer.replace("\x00", "")
answer_strip = str(answer_decode.strip())
print(repr(answer_strip))
valueScale = float(answer_strip)
The answers in these posts helped:
How to get rid of \x00 in my array of bytes?
Invalid literal for float(): 0.000001, how to fix error?
Watch out for possible unintended literals in your argument
for example you can have a space within your argument, rendering it to a string / literal:
float(' 0.33')
After making sure the unintended space did not make it into the argument, I was left with:
float(0.33)
Like this it works like a charm.
Take away is:
Pay Attention for unintended literals (e.g. spaces that you didn't see) within your input.