Develop Reference

How to read complex data from TB size binary file, fast and keep the most accuracy?

Use Python 3.9.2 read the beginning of TB size binary file (piece of it) as below:
file=open(filename,'rb')
bytes=file.read(8)
print(bytes)
b'\x14\x00\x80?\xb5\x0c\xf81'
I tried np.fromfile np.fromfile(np.complex64) ways to read the file filename.
float_data1 = np.fromfile(filename,np.float32)
float_data2 = np.fromfile(filename,np.complex64)
As the binary file always bigger than 500GB,even TB size,how to read complex data from TB size binary file, fast and keep the most acuuracy?

This is related to your ham post.
samples = np.fromfile(filename, np.complex128)
and
Those codes equal to -1.9726906072368233e-31,+3.6405886029665884e-23.
No, they don't equal that. That's just your interpretation of bytes as float64. That interpretation is incorrect!
You assume these are 64-bit floating point numbers. They are not; you really need to stop assuming that; it's wrong, and we can't help you if you still act as if it were 64-bit floats forming a 128 bit complex value.
Besides documents,I compare the byte content in the answer,that is more than reading docs.
As I already pointed out, that is wrong. Your computer can read anything as any type, just as you tell them, even if it's not the original type it's been stored in. You stored complex64, but read complex128. That's why your values are so inplausible.
It's 32-bit floats, forming a 64 bit complex value. The official block documentation for the file sink also points that out, and even explains the numpy dtype you need to use!
Anyways, you can use numpy's memmap functionality to map the file contents without reading them all to RAM. That works. Again, you need to use the right dtype, which is, to repeat this the 10th time, not complex128.
It's really easy:
data = numpy.memmap(filename, dtype=numpy.complex64)
done.

Grabbing full file version of an exe in Python

I was wondering if there was a easy way to grab the full file version of an exe in python (for instance if you right click into a file's properties and go to Details, you will find something like File Version 1.1.1.0).
I found something close by using win32api.GetFileVersionInfo, however when listing the file properties, under FileVersionLS it seems to have given only one digit, the far right digit of the file version (so in the case of the example above it gave the 0 in 1.1.1.0) when I needed the whole version number.
Hopefully this makes sense, please let me know if there is something I need to elaborate on.
Thanks for reading!

FileVersionLS and FileVersionMS act together to represent the complete version number as a 64-bit integer:
dwFileVersionMS
Type: DWORD
The most significant 32 bits of the file's binary version number. This member is used with dwFileVersionLS to form a 64-bit value used for numeric comparisons.
dwFileVersionLS
Type: DWORD
The least significant 32 bits of the file's binary version number. This member is used with dwFileVersionMS to form a 64-bit value used for numeric comparisons.
They are each bit-packed with two 16-bit numbers, so you have to bit-shift out the individual numbers. You can use win32api.LOWORD() and win32api.HIWORD() for that, eg:
def get_file_version(self, path):
info = win32api.GetFileVersionInfo(path, '\\')
ms = info['FileVersionMS']
ls = info['FileVersionLS']
return (win32api.HIWORD(ms), win32api.LOWORD(ms),
win32api.HIWORD(ls), win32api.LOWORD(ls))

Convert two raw values to 32-bit IEEE floating point number

I am attempting to decode some data from a Shark 100 Power Meter via TCP modbus. I have successfully pulled down the registers that I need, and am left with two raw values from the registers like so:
[17138, 59381]
From the manual, I know that I need to convert these two numbers into a 32bit IEEE floating-point number. I also know from the manual that "The lower-addressed register is the
high order half (i.e., contains the exponent)." The first number in the list shown above is the lower-addressed register.
Using Python (any library will do if needed), how would I take these two values and make them into a 32 bit IEEE floating point value.
I have tried to use various online converters and calculators to figure out a non-programmatic way to do this, however, anything I have tried gets me a result that is way out of bounds (I am reading volts in this case so the end result should be around 120-122 from the supplied values above).

Update for Python 3.6+ (f-strings).
I am not sure why the fill in #B.Go's answer was only 2. Also, since the byte order was big-endian, I hardcoded it as such.
import struct
a = 17138
b = 59381
struct.unpack('>f', bytes.fromhex(f"{a:0>4x}" + f"{b:0>4x}"))[0]
Output: 121.45304107666016

The following code works:
import struct
a=17138
b=59381
struct.unpack('!f', bytes.fromhex('{0:02x}'.format(a) + '{0:02x}'.format(b)))
It gives
(121.45304107666016,)
Adapted from Convert hex to float and Integer to Hexadecimal Conversion in Python

I read in the comments, and #Sanju had posted this link: https://github.com/riptideio/pymodbus/blob/master/examples/common/modbus_payload.py
For anyone using pymodbus, the BinaryPayloadDecoder is useful as it's built in. It's very easy to pass a result.registers, as shown in the example. Also, it has a logging integrated, so you can help debug why a conversion isn't working (ex: wrong endianness).
As such, I made a working example for this question (using pymodbus==2.3.0):
from pymodbus.constants import Endian
from pymodbus.payload import BinaryPayloadDecoder
a = 17138
b = 59381
registers = [a, b]
decoder = BinaryPayloadDecoder.fromRegisters(registers, byteorder=Endian.Big)
decoder.decode_32bit_float() # type: float
Output: 121.45304107666016

Converting a string to a wav file in python

I'm new to Python, and to programming in general, so please don't take it too hard on me
I am currently trying to figure out how to write a new wav file using a string (which was derived from another wave file's data)
I performed a fourier transform on that file's data, so now I'm trying to get the values from the Fourier transform written into a new wav file.
I can only use numpy and the included Python library, not scipy
According to the documentation, I have to use wave_write(), but I have no idea what the code is supposed to look like for this function.
I think I'm supposed to do something pertaining to
wave_write.writeframesraw(data)
Then again, not totally sure of what to do.
Any help is greatly appreciated!

Two functions in NumPy can help you with this: astype and tostring.
If you have an array of sound samples, say X then you can convert it to the right format using astype. This will depend on what data type is used in the wav file, and the library you are using to save it. But let us for this example say you want to store it as 16 bit integer. You'll need to scale X according to the data type selected - so in this case the range will be -32768 to 32767 for a signed 16 bit int. If you sample goes from -1.0 to 1.0 then you can simply multiply with 32767.
The next part is simply to convert it to a string using tostring, it could look something the following:
scaled = X * 32767
scaled.astype('<i2').tostring()
You can find the documentation for the functions here:
https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.astype.html
https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.tostring.html

ARINC429 Word Construction

Just as a preamble I am using python 3 and the bitstring library.
So Arinc429 words are 32 bit data words.
Bits 1-8 are used to store the label. Say for example I want the word to set the latitude, according to the label docs, set latitude is set to the octal
041
I can model this in python by doing:
label = BitArray(oct='041')
print(label.bin)
>> 000100001
The next two bits can be used to send a source, or extend the label by giving an equipment ID. Equipment IDs are given in hex, the one I wish to use is
002
So again, I add it to a new BitArray object and convert it to binary
>> 000000010
Next comes the data field which spans from bits 11-29. Say I want to set the latitude to the general area of London (51.5072). This is where I'm getting stuck as floats can only be 32/64 bits long.
There are 2 other parts of the word, but before I go there I am just wondering, if I am going along the right track, or way off how you would construct such a word?
Thanks.

I think you're on the right track, but you need to either know or decide the format for your data field.
If the 19 bits you want to represent a float are documented somewhere as being a float then look how that conversion is done (as that's not at all a standard number of bits for a floating point number). If those bits are free-form and you can choose both the encode and decode then just pick something appropriate.
There is a standard for 16-bit floats which is occasionally used, but if you only want to represent a latitude I'd go for something simpler. As it can only got from 0 to 360 just scale that to an integer from 0 to 2^19 and store the integer.
So 51.5072 becomes (51.5072/360*(2**19)) = 75012
Then store this as a unsigned integer
> latitude = BitArray(uint=75012, length=19)
This gives you a resolution of about 0.0007 degrees, which is the best you can hope for. To convert back:
> latitude.uint*360.0/2**19
51.50665283203125