I've a script which reads temperature data:
def get_temp(socket, channels):
data = {}
for ch in channels:
socket.sendall('KRDG? %s\n' % ch)
time.sleep(0.2)
temp = socket.recv(32).rstrip('\r\n')
data[ch] = float(temp)
Sometimes, the script fails on the line which converts the values to float:
File "./projector.py", line 129, in get_temp
data[ch] = float(temp)
ValueError: invalid literal for float(): +135.057E+0
+078.260E+0
+00029
but this is NOT an invalid literal. If I enter this into any python shell,
float(+135.057E+0)
then it correctly returns 135.057.
So what is the problem?
I would all but guarantee that the issue is some sort of non-printing character that's present in the value you pulled off your socket. It looks like you're using Python 2.x, in which case you can check for them with this:
print repr(temp)
You'll likely see something in there that's escaped in the form \x00. These non-printing characters don't show up when you print directly to the console, but their presence is enough to negatively impact the parsing of a string value into a float.
-- Edited for question changes --
It turns this is partly accurate for your issue - the root cause however appears to be that you're reading more information than you expect from your socket or otherwise receiving multiple values. You could do something like
map(float, temp.strip().split('\r\n'))
In order to convert each of the values, but if your function is supposed to return a single float value this is likely to cause confusion. Anyway, the issue certainly revolves around the presence of characters you did not expect to see in the value you retrieved from your socket.
I had a similar issue reading the serial output from a digital scale. I was reading [3:12] out of a 18 characters long output string.
In my case sometimes there is a null character "\x00" (NUL) which magically appears in the scale's reply string and is not printed.
I was getting the error:
> ' 0.00'
> 3 0 fast loop, delta = 10.0 weight = 0.0
> ' 0.00'
> 1 800 fast loop, delta = 10.0 weight = 0.0
> ' 0.00'
> 6 0 fast loop, delta = 10.0 weight = 0.0
> ' 0\x00.0'
> Traceback (most recent call last):
> File "measure_weight_speed.py", line 172, in start
> valueScale = float(answer_string)
> ValueError: invalid literal for float(): 0
After some research I wrote few lines of code that work in my case.
replyScale = scale_port.read(18)
answer = replyScale[3:12]
answer_decode = answer.replace("\x00", "")
answer_strip = str(answer_decode.strip())
print(repr(answer_strip))
valueScale = float(answer_strip)
The answers in these posts helped:
How to get rid of \x00 in my array of bytes?
Invalid literal for float(): 0.000001, how to fix error?
Watch out for possible unintended literals in your argument
for example you can have a space within your argument, rendering it to a string / literal:
float(' 0.33')
After making sure the unintended space did not make it into the argument, I was left with:
float(0.33)
Like this it works like a charm.
Take away is:
Pay Attention for unintended literals (e.g. spaces that you didn't see) within your input.
Related
I am running this hashlib code and it runs almost all the way:
def generate_hashes(peaks, fan_value=DEFAULT_FAN_VALUE):
if PEAK_SORT:
sorted(peaks,key=itemgetter(1))
# bruteforce all peaks
peaks=list(peaks)
len_peaks=len(peaks)
for i in range(len_peaks):
for j in range(1, fan_value):
if (i + j) < len(peaks):
# take current & next peak frequency value
freq1 = peaks[i][IDX_FREQ_I]
freq2 = peaks[i + j][IDX_FREQ_I]
# take current & next -peak time offset
t1 = peaks[i][IDX_TIME_J]
t2 = peaks[i + j][IDX_TIME_J]
# get diff of time offsets
t_delta = t2 - t1
# check if delta is between min & max
if t_delta >= MIN_HASH_TIME_DELTA and t_delta <= MAX_HASH_TIME_DELTA:
h = hashlib.sha1(("%s|%s|%s") % (str(freq1), str(freq2), str(t_delta)))
yield (h.hexdigest()[0:FINGERPRINT_REDUCTION], t1)
However, it returns this error:
h = hashlib.sha1(("%s|%s|%s") % (str(freq1), str(freq2), str(t_delta)))
TypeError: Unicode-objects must be encoded before hashing
I am honestly completely lost and don't know how to fix it. If you guys have any follow up questions regarding details about the code I will try my best to answer. Any feedback would be appreciated.
The answer is in the error message: use encode on your text string before hashing.
h = hashlib.sha1(("%s|%s|%s" % (str(freq1), str(freq2), str(t_delta))).encode('utf-8'))
The reason this is necessary is because hashlib.sha1() requires a bytes object due to the way it works internally. Normal Python strings (since version 3.0) are made of Unicode codepoints, which don't fit into a byte. They need an encoding which defines how the translation between codepoints and bytes occurs. UTF-8 is the most popular encoding, because it can handle every Unicode codepoint yet remain backwards compatible with older encodings like ASCII.
I have a Python code from a third party which uses Python 2.7 and create_string_buffer and string.join on ctypes tools. I want to convert the code to Python 3.8.3, but I got an error on the following part. Here's the code after I converted it to Python3 using 2to3 tools:
for dev in self.devices:
handle = libusb0.usb_open(dev)
self.handles.append(handle) # open and store handle
sn_char = create_string_buffer('\000'*16)
libusb0.usb_get_string_simple(handle, offset, sn_char, 16)
ser_num = ''.join(sn_char).split(b'\0',1)[0] # treat first null-byte as stop character
self.sn.append(ser_num)
The error I got is:
sn_char = create_string_buffer('\000'*16)
File "C:\Python\Python383\lib\ctypes\__init__.py", line 65, in create_string_buffer
raise TypeError(init)
TypeError:
I have also already tried to make an init variable in create_string_buffer to byte (sn_char = create_string_buffer(b'\000'*16), but I still got an error like this:
ser_num = ''.join(sn_char).split(b'\0',1)[0] # treat first null-byte as stop character
TypeError: sequence item 0: expected str instance, bytes found
hope will get solution in here, thanks...
When you use .split and provide parameter of type bytes, the object you work on must be also of type bytes.
You can easily solve this by adding b before the literal string:
ser_num = b''.join(sn_char).split(b'\0',1)[0] # treat first null-byte as stop character
thanks for the fast response, yea it can run, but now I get the new error at the other code which uses create_string_buffer, at this code
def write_data(self, sn, words_in):
num_words = len(words_in)
dev_num = self.sn_to_devnum(sn)
handle = self.handles[dev_num]
buf = create_string_buffer(b'\000'*(num_words*2-1))
for n in range(num_words):
buf[2*n] = chr(words_in[n] % 0x100);
buf[2*n+1] = chr(words_in[n] // 0x100);
ret = libusb0.usb_bulk_write(handle, self.in_ep, buf, num_words*2,
self.usb_write_timeout);
#wr_buf = [ord(buf[n]) for n in range(num_bytes)]
#print "write buffer = ", wr_buf
return ret;
The error is :
buf[2*n] = chr(words_in[n] % 0x100);
TypeError: one character bytes, bytearray or integer expected
Sorry if my question is repeatly and too simple, cause I new in python... thanks
I am working on the following python code:
import wave
from bitstring import BitArray
w = wave.open('file.wav','rb')
totalFrames = w.getnframes() #Total number of samples
bytesData = w.readframes(totalFrames)
binData = BitArray(bytesData)
bin2Data = (binData.bin)
The file.wav has 88200 samples at a sampling rate of 44.1KHz.
My goal is to be able to get the 2's compliment of the binary data I obtain from file.wav. 'binData.bin' gives me the binary form of the bytes (\x00\x00N\x00n\xff..) obtained through w.readframes but in a string format.
I was using this to obtain 2'scompliment:
2comp = ~(bin2Data) + 0b1
but in vain. It would show the following error:
Traceback (most recent call last):
File "speaker_bin.py", line 16, in <module>
bin2Data = ~((binData.bin)) + 0b1
TypeError: bad operand type for unary ~: 'str'
I tried int(bin2Data) to convert it but it would not work (It would not print anything at all. I guess because of the size of the data.)
What am I doing wrong?
I would really appreciate any feedback. (even a simple nudge in the right direction)
You need to use
int(binData.bin, 2)
To create an int, you can specify the base as a second parameter, otherwise it will just assume the value is in base 10. As you can see from the docs, the default base is 10, which is why you need to specify a different base other than 10
Also do the same with 0b1
Here's the code snippet from my RFID wiegand reader on my Raspberry Pi I use already.
def main():
set_procname("Wiegand Reader")
global bits
global timeout
GPIO.add_event_detect(D0, GPIO.FALLING, callback=one)
GPIO.add_event_detect(D1, GPIO.FALLING, callback=zero)
GPIO.add_event_detect(S1, GPIO.FALLING, callback=unlockDoor)
while 1:
if bits:
timeout = timeout -1
time.sleep(0.001)
if len(bits) > 1 and timeout == 0:
#print "Binary:", int(str(bits),2)
c1 = int(str(bits),2)
result = ((~c1) >> 1) & 0x0FFFFFF;
checkAccess(result, doorID)
else:
time.sleep(0.001)
if __name__ == '__main__':
main()
On a normal USB RFID reader, I get 0000119994 and that's what's printed on the card. But with this code it reads 119994. I've tried multiple cards. It always drops the zeros at the front .
I even tried a card with a zero in it. 0000120368 and it shows 120368
I thought it was taking off the first 4 characters but then I tried a key fob that only had 3 zeros in front. 0004876298 and it reads 4876298. Only dropping the front zeros.
Python will remove the front few bits if they are zero, this also applies to integers. For example
>>> a = 0003
>>> a
3
>>> b = 0b0011
>>> bin(b)
0b11
From what I see, all RFID's will have 10 numbers. You can make a simple program to add those numbers in and store the value as a string:
def rfid_formatter(value):
str_value = str(value)
for s in range(10 - len(str_value)):
str_value = "0" + str_value
return str_value
Your test cases:
print rfid_formatter(120368)
print "0000120368"
print rfid_formatter(4876298)
print "0004876298"
As mentioned already, leading zeros are removed in binary sequences and also when you explicitly convert a string to decimal using int().
What hasn't been mentioned already is that, in Python 2.x, integers with leading zeros are treated as octal values.
>>> a = 0003
>>> a
3
>>> a = 000127
>>> a
87
Since this was causing confusion, the implicit octal conversion was removed in Python 3 and any number of leading zeros in numerical values will raise a SyntaxError.
>>> a = 000127
File "<stdin>", line 1
a = 000127
^
SyntaxError: invalid token
>>>
You can read the rationale behind these decisions in PEP 3127.
Anyway, I mention all of this simply to arrive at an assumption: you're probably not working with octal representations. Instead, I think you're converting result to a string in checkAccess so you can do a string comparison. If this assumption is correct, you can simply use the string method zfill (zero fill):
>>> str(119994).zfill(10)
'0000119994'
>>>
>>> str(4876298).zfill(10)
'0004876298'
>>>
I'm working on a project where the output size is very important. As my outputs are numbers between 0 and 100, I'm trying to write them as bytes (or unsigned chars).
However, I'm getting errors when trying to read them.
Here is a simple example:
test_filename='test.b'
g=(3*ones(shape=[1000])).astype('c')
g.tofile(test_filename)
with open(test_filename, "rb") as f:
bytes = f.read(1)
num = int(bytes.encode('hex'), 1)
print num
Here is the error I get, somehow the bytes.encode thingy excepts a binary string or something of that sort (not sure of course):
ValueError Traceback (most recent call last)
<ipython-input-43-310a447041fe> in <module>()
----> 1 num = int(bytes.encode('hex'), 1)
2 print num
ValueError: int() base must be >= 2 and <= 36
I should state that I would later need to read the output files in C++.
Thanks in advance,
Gil
There is some iffiness to this based on the version of python you are using.
If python2, which I assume you are using because of the print statement, the main problem you have is that you are getting a string from the read, so if the value is say 50 you would get an ascii value of 2 if you print it. You need to tell python that those bits should be in an int type not a str type and a simple cast does not do that.
I personally would use the struct package and do the following:
with open(test_filename, "rb") as f:
bytes = f.read(1)
num = struct.unpack("B", bytes)[0]
print num
Another option would be to encode the string to hex and read it in as a hex string (which looks like is what you are trying):
num = int(bytes.encode("hex_codec"), 16))
print num
One final option would be to put the string in a bytearray and pull the first byte:
num = bytearray(bytes)[0]
print num
If you are actually using python 3 this is simpler because you will get back a bytes object (if so dont name a variable bytes, very confusing). With a bytes object you can just pull the first element out which will be pulled out as an int:
num = bytes[0]
print num