I am debugging an hardware IP and since I am looking at waveforms it's more tidy and organized if I keep the numbers represented in the hex format ( I am saying this because I could change the representation in the waveform viewer as "decimal", but if there is a way to avoid that, I'd prefer it. And the reason I prefer it is because with decimal I lose track of the single byte values). Example of numbers in the waveform:
The drawback however is that whenever I need to check them against my results in python I need to wrap every number with hex() before printing it. Note that I am using Python as book keeping and to check results on the spot, I don't care about performance or whatever else.
import numpy as np
#Test HW dot product - This is just an example
e = np.arange(0x5a,0x61)
a = np.arange(0x1,0x8)
# Intermediate result
r1 = a[0:4]*e[0:4]
#Final result
c = np.dot(a,e)
In the spyder console then I type the variable to display the content:
>>>c
Out[6]: 2632
>>>r1
Out[7]: array([ 90, 182, 276, 372])
However I would like them to be displayed as hex. Is there any console setting or Pyhton representation setting to make this happen?
I know that I can create a wrapper around a print function that calls hex, but I don't want to have prints all around my code, I like that I can just type a variable name in the console and see the value.
Related
So I have been trying to make a program that can display a countdown clock to a GUI. It requests a JSON file from a website and then uses that data- getting the JSON file and it's data is working for other parts of my program except using the data inside datetime.
import time
import requests
import json
import datetime as dt
response = requests.get("https://website.website/mission.json")
data = json.loads(response.text)
a = dt.datetime(data['launch'])
b = dt.datetime.now()
print("T- " + time.strftime("%H:%M:%S", time.gmtime(round((a-b).total_seconds()))))
JSON file:
{
"mission": "GlobeSat2",
"launch": "2020,9,13,19,00,00",
"status": "Go"
}
When I use a = dt.datetime(data['launch']) it gives an error of TypeError: an integer is required (got type str) but when I use a = dt.datetime(2020,9,13,19,00,00) there is no error.
The value is the same as if I put it in myself
>print(data['launch'])
>2020,9,13,19,00,00
What I've tried:
I changed the value of the JSON file to "launch": "20200913120000" and gave an error of OverflowError: Python int too large to convert to C long.
I tried to do int(data['launch']) which obviously didn't work.
If your problem is to change that date to datetime format you can easily use this one line of code to do that
a=dt.datetime(*([int(i) for i in data['launch'].split(",")]))
then a will get converted into datetime.
Beginning programmers often make similar mistakes to yours here, confusing print statements with return values and confusing string representations of values with the actual value.
Remember that standard in and out, or the console are just a text-based user interface. They feel technical and close to the language, but they are still just a user interface.
If you run this script:
x = 123
print(123)
Or you do the same thing on the Python CLI:
>>> x = 123
>>> x
123
You might thing Python showed you the value of the x variable of type int, 123. But 123 is just the text representation of that value. There's many ways you could represent 123, for example 0b1111011 or 0x7B - all valid string representations of the exact same integer number.
Your problem is similar, in that you expect these two things to be the same:
a = dt.datetime(2020,9,13,19,00,00)
parameter = '2020,9,13,19,00,00'
b = dt.datetime(parameters)
But in the case of assigning to a, you call .datetime() with 6 integer values, while in the case of assigning to b, you call .datetime() with a single string value. The fact that the string looks very similar when printed to how you would normally write those 6 integer values in code really means nothing to Python. Note the 'when printed' vs. 'in code'.
You can have Python interpret a string to obtain the values you know it represents, but you'll have to tell it how (there's many approaches).
For example:
a = dt.datetime(*[int(part) for part in data['launch'].split(',')])
This takes the string value of data['launch'], splits it over the ',' into 6 strings in a list (['2020','9','13','19','00','00']) and then turns each part of that list into an integer with the int() function, which can turn a string that represents a single integer into its int value. The resulting list is then spread over the parameters using the * operator on the list.
There's many different approaches, but of course you could also consider fixing the JSON if you wrote the server that returned the value:
{
"mission": "GlobeSat2",
"launch": [2020,9,13,19,0,0],
"status": "Go"
}
By replacing the "" with [], JSON now represents the value of "launch" as a list and when read by Python, it will be read as a list of integers.
If that was your JSON, this would be your code:
a = dt.datetime(*data['launch'])
Still using the * to spread it over the parameters of the function, but everything else would be done automatically when interpreting the JSON.
I am attempting to decode some data from a Shark 100 Power Meter via TCP modbus. I have successfully pulled down the registers that I need, and am left with two raw values from the registers like so:
[17138, 59381]
From the manual, I know that I need to convert these two numbers into a 32bit IEEE floating-point number. I also know from the manual that "The lower-addressed register is the
high order half (i.e., contains the exponent)." The first number in the list shown above is the lower-addressed register.
Using Python (any library will do if needed), how would I take these two values and make them into a 32 bit IEEE floating point value.
I have tried to use various online converters and calculators to figure out a non-programmatic way to do this, however, anything I have tried gets me a result that is way out of bounds (I am reading volts in this case so the end result should be around 120-122 from the supplied values above).
Update for Python 3.6+ (f-strings).
I am not sure why the fill in #B.Go's answer was only 2. Also, since the byte order was big-endian, I hardcoded it as such.
import struct
a = 17138
b = 59381
struct.unpack('>f', bytes.fromhex(f"{a:0>4x}" + f"{b:0>4x}"))[0]
Output: 121.45304107666016
The following code works:
import struct
a=17138
b=59381
struct.unpack('!f', bytes.fromhex('{0:02x}'.format(a) + '{0:02x}'.format(b)))
It gives
(121.45304107666016,)
Adapted from Convert hex to float and Integer to Hexadecimal Conversion in Python
I read in the comments, and #Sanju had posted this link: https://github.com/riptideio/pymodbus/blob/master/examples/common/modbus_payload.py
For anyone using pymodbus, the BinaryPayloadDecoder is useful as it's built in. It's very easy to pass a result.registers, as shown in the example. Also, it has a logging integrated, so you can help debug why a conversion isn't working (ex: wrong endianness).
As such, I made a working example for this question (using pymodbus==2.3.0):
from pymodbus.constants import Endian
from pymodbus.payload import BinaryPayloadDecoder
a = 17138
b = 59381
registers = [a, b]
decoder = BinaryPayloadDecoder.fromRegisters(registers, byteorder=Endian.Big)
decoder.decode_32bit_float() # type: float
Output: 121.45304107666016
I used netCDF Python library to read netCDF variable which has list(variable) returns correct decimal precision as in the image (using PyCharm IDE). However, when I try to get the element by index, e.g: variable[0], it returns the rounded value instead (e.g: 5449865.55794), while I need 5449865.55793999997.
How can I iterate this list with correct decimal precision ?
Some basic code
from netCDF4 import Dataset
nc_dataset = Dataset(self.get_file().get_filepath(), "r")
variable_name = "E"
// netCDF file contains few variables (axis dimensions)
variable = nc_dataset.variables[variable_name]
variable is not a list but a netCDF object, however when using list() or variable[index] will return element value of the axis dimension.
The decimals you are chasing are bogus. The difference is only in the way these numbers are represented on your screen, not in how they are stored in your computer.
Try the following to convince yourself
a = 5449865.55793999997
a
# prints 5449865.55794
The difference between the two numbers if we were to take them literally is 3x10^-11. The smallest difference a 64 bit floating point variable at the size of a can resolve is more than an order of magnitude larger. So your computer cannot tell these two decimal numbers apart.
But look at the bright side. Your data aren't corrupted by some mysterious process.
Hope this is what suits your needs:
import decimal
print decimal.Decimal('5449865.55793999997')
I have to read a binary file in python. This is first written by a Fortran 90 program in this way:
open(unit=10,file=filename,form='unformatted')
write(10)table%n1,table%n2
write(10)table%nH
write(10)table%T2
write(10)table%cool
write(10)table%heat
write(10)table%cool_com
write(10)table%heat_com
write(10)table%metal
write(10)table%cool_prime
write(10)table%heat_prime
write(10)table%cool_com_prime
write(10)table%heat_com_prime
write(10)table%metal_prime
write(10)table%mu
if (if_species_abundances) write(10)table%n_spec
close(10)
I can easily read this binary file with the following IDL code:
n1=161L
n2=101L
openr,1,file,/f77_unformatted
readu,1,n1,n2
print,n1,n2
spec=dblarr(n1,n2,6)
metal=dblarr(n1,n2)
cool=dblarr(n1,n2)
heat=dblarr(n1,n2)
metal_prime=dblarr(n1,n2)
cool_prime=dblarr(n1,n2)
heat_prime=dblarr(n1,n2)
mu =dblarr(n1,n2)
n =dblarr(n1)
T =dblarr(n2)
Teq =dblarr(n1)
readu,1,n
readu,1,T
readu,1,Teq
readu,1,cool
readu,1,heat
readu,1,metal
readu,1,cool_prime
readu,1,heat_prime
readu,1,metal_prime
readu,1,mu
readu,1,spec
print,spec
close,1
What I want to do is reading this binary file with Python. But there are some problems.
First of all, here is my attempt to read the file:
import numpy
from numpy import *
import struct
file='name_of_my_file'
with open(file,mode='rb') as lines:
c=lines.read()
I try to read the first two variables:
dummy, n1, n2, dummy = struct.unpack('iiii',c[:16])
But as you can see I had to add to dummy variables because, somehow, the fortran programs add the integer 8 in those positions.
The problem is now when trying to read the other bytes. I don't get the same result of the IDL program.
Here is my attempt to read the array n
double = 8
end = 16+n1*double
nH = struct.unpack('d'*n1,c[16:end])
However, when I print this array I get non sense value. I mean, I can read the file with the above IDL code, so I know what to expect. So my question is: how can I read this file when I don't know exactly the structure? Why with IDL it is so simple to read it? I need to read this data set with Python.
What you're looking for is the struct module.
This module allows you to unpack data from strings, treating it like binary data.
You supply a format string, and your file string, and it will consume the data returning you binary objects.
For example, using your variables:
import struct
content = f.read() #I'm not sure why in a binary file you were using "readlines",
#but if this is too much data, you can supply a size to read()
n, T, Teq, cool = struct.unpack("dddd",content[:32])
This will make n, T, Teq, and cool hold the first four doubles in your binary file. Of course, this is just a demonstration. Your example looks like it wants lists of doubles - conveniently struct.unpack returns a tuple, which I take for your case will still work fine (if not, you can listify them). Keep in mind that struct.unpack needs to consume the whole string passed into it - otherwise you'll get a struct.error. So, either slice your input string, or only read the number of characters you'll use, like I said above in my comment.
For example,
n_content = f.read(8*number_of_ns) #8, because doubles are 8 bytes
n = struct.unpack("d"*number_of_ns,n_content)
Did you give scipy.io.readsav a try?
Simply read you file like this:
mydict = scipy.io.readsav('name_of_file')
It looks like you are trying to read the cooling_0000x.out file generated by RAMSES.
Note that the first two integers (n1, n2) provide the dimensions of the two dimentional tables (arrays) that follow in the body of the file... So you need to first process those two integers before you know how much real*8 data is in the rest of the file.
scipy should be of help -- it lets you read arbitrary dimensioned binary data:
http://wiki.scipy.org/Cookbook/InputOutput#head-e35c7736718209eea00ebf37a7e1dfb91df696e1
If you already have this python code, please let me know as I was going to write it today (17Sep2014).
Rick
I am using python to read a currency value from excel. The returned from the range.Value method is a tuple that I don't know how to parse.
For example, the cell appears as $548,982, but in python the value is returned as (1, 1194857614).
How can I get the numerical amount from excel or how can I convert this tuple value into the numerical value?
Thanks!
Try this:
import struct
try: import decimal
except ImportError:
divisor= 10000.0
else:
divisor= decimal.Decimal(10000)
def xl_money(i1, i2):
byte8= struct.unpack(">q", struct.pack(">ii", i1, i2))[0]
return byte8 / divisor
>>> xl_money(1, 1194857614)
Decimal("548982.491")
Money in Microsoft COM is an 8-byte integer; it's fixed point, with 4 decimal places (i.e. 1 is represented by 10000). What my function does, is take the tuple of 4-byte integers, make an 8-byte integer using struct to avoid any issues of sign, and then dividing by the constant 10000. The function uses decimal.Decimal if available, otherwise it uses float.
UPDATE (based on comment): So far, it's only COM Currency values being returned as a two-integer tuple, so you might want to check for that, but there are no guarantees that this will always be successful. However, depending on the library you use and its version, it's quite possible that later on, after some upgrade, you will be receiving decimal.Decimals and not two-integer tuples anymore.
I tried this with Excel 2007 and VBA. It is giving correct value.
1) Try pasting this value in a new excel workbook
2) Press Alt + F11. Gets you to VBA Editor.
3) Press Ctrl + G. Gets you to immediate window.
4) In the immediate window, type ?cells("a1").Value
here "a1" is the cell where you have pasted the value.
I am doubting that the cell has some value or character due to which it is interpreted this way.
Post your observations here.