Develop Reference

python lightweight solution for typecasting float(64) to bytes

I'd like to have a very simple solution in displaying the raw bytes for a float value (or more consecutive ones in memory). in my understanding, this is named typecasting (reading the memory values in byte) not to be misunderstood as casting (reading the value and interpreting that in byte).
The simplest test seem to be:
import numpy
a=3.14159265
print(a.hex())
# Returns 0x1.921fb53c8d4f1p+1
b=numpy.array(a)
print(b.tobytes())
# returns b'\xf1\xd4\xc8S\xfb!\t#'
# expected is something like 'F1' 'D4' 'C8' '53' 'FB' '21' '09' '40'
but the method hex() returns an Interpretation of the IEEE FLOAT represantation in hex. The second method shows four hex-Byte markers \x but I wonder as a float64 should read 8 Bytes. Further I'm wondering by the other characters.
I good old simple C I would have implemented that by simply using an unsigned int pointer on the memory address of the float and printing 8 values from that unsigned int "Array" (pointer.)
I know, that I can use C within python - but are there other simple solutions?
Maybe as it is of interest: I need that functionalaty to save many big float vectors to a BLOB into a database.
I think, similiar Problems are to be found in Correct interpretation of hex byte, convert it to float reading that if possible (displayable characters) they are not printed in \x-form. How can I change that?

You can use the built-in module struct for this. If you have Python 3.5 or later:
import struct
struct.pack('d', a).hex()
It gives:
'f1d4c853fb210940'
If you have Python older than 3.5:
import binascii
binascii.hexlify(struct.pack('d', a))
Or:
hex(struct.unpack('>Q', struct.pack('d', a))[0])
If you have an array of floats and want to use NumPy:
import numpy as np
np.set_printoptions(formatter={'int':hex})
np.array([a]).view('u8')

How python variables look like in memory

I want to know how a Python variable (int, list, tuple) look like in memory. And this is where I am right now.
from ctypes import string_at
from sys import getsizeof
from binascii import hexlify
string_at(id(a), getsizeof(a))
I expect it will return the hex representation of a variable in the memory.
However, here is the output when I assign value 1,2,3 to variable 'a':
1 - '\xd6\x05\x00\x00\x00\x00\x00\x00\xc0\x92\x17\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00'
2 - '\x17\x02\x00\x00\x00\x00\x00\x00\xc0\x92\x17\x00\x01\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00'
3 - '\xdc\x00\x00\x00\x00\x00\x00\x00\xc0\x92\x17\x00\x01\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00'
4 - '\x06\x01\x00\x00\x00\x00\x00\x00\xc0\x92\x17\x00\x01\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00'
Somewhere close to the middle, I can see \x01, \x02...etc. However, here are my other questions:
At the beginning, I can see two other bytes changing, what are those values?
Except for those \x00, I can see a few other bytes like ...\xc0\x92\x17\x00\x01... how to interpret those values?
Is there any resource available for me to learn how python store variables in memory?

Download the Python C sources and study them. You'll see that "almost everything" is a PyObject* -- a pointer to a PyObject struct. The \x01 &c you see are just random bytes within some of those pointers, nothing to do with the 1,2,3 tuple you're seeing, not directly at least!

python 32 bit float conversion

Python 2.6 on Redhat 6.3
I have a device that saves 32 bit floating point value across 2 memory registers, split into most significant word and least significant word.
I need to convert this to a float.
I have been using the following code found on SO and it is similar to code I have seen elsewhere
#!/usr/bin/env python
import sys
from ctypes import *
first = sys.argv[1]
second = sys.argv[2]
reading_1 = str(hex(int(first)).lstrip("0x"))
reading_2 = str(hex(int(second)).lstrip("0x"))
sample = reading_1 + reading_2
def convert(s):
i = int(s, 16) # convert from hex to a Python int
cp = pointer(c_int(i)) # make this into a c integer
fp = cast(cp, POINTER(c_float)) # cast the int pointer to a float pointer
return fp.contents.value # dereference the pointer, get the float
print convert(sample)
an example of the register values would be ;
register-1;16282 register-2;60597
this produces the resulting float of
1.21034872532
A perfectly cromulent number, however sometimes the memory values are something like;
register-1;16282 register-2;1147
which, using this function results in a float of;
1.46726675314e-36
which is a fantastically small number and not a number that seems to be correct. This device should be producing readings around the 1.2, 1.3 range.
What I am trying to work out is if the device is throwing bogus values or whether the values I am getting are correct but the function I am using is not properly able to convert them.
Also is there a better way to do this, like with numpy or something of that nature?
I will hold my hand up and say that I have just copied this code from examples on line and I have very little understanding of how it works, however it seemed to work in the test cases that I had available to me at the time.
Thank you.

If you have the raw bytes (e.g. read from memory, from file, over the network, ...) you can use struct for this:
>>> import struct
>>> struct.unpack('>f', '\x3f\x9a\xec\xb5')[0]
1.2103487253189087
Here, \x3f\x9a\xec\xb5 are your input registers, 16282 (hex 0x3f9a) and 60597 (hex 0xecb5) expressed as bytes in a string. The > is the byte order mark.
So depending how you get the register values, you may be able to use this method (e.g. by converting your input integers to byte strings). You can use struct for this, too; this is your second example:
>>> raw = struct.pack('>HH', 16282, 1147) # from two unsigned shorts
>>> struct.unpack('>f', raw)[0] # to one float
1.2032617330551147

The way you've converting the two ints makes implicit assumptions about endianness that I believe are wrong.
So, let's back up a step. You know that the first argument is the most significant word, and the second is the least significant word. So, rather than try to figure out how to combine them into a hex string in the appropriate way, let's just do this:
import struct
import sys
first = sys.argv[1]
second = sys.argv[2]
sample = int(first) << 16 | int(second)
Now we can just convert like this:
def convert(i):
s = struct.pack('=i', i)
return struct.unpack('=f', s)[0]
And if I try it on your inputs:
$ python floatify.py 16282 60597
1.21034872532
$ python floatify.py 16282 1147
1.20326173306

flip bytes using Python's struct module

I would like to flip from big to little endian this string:
\x00\x40
to have it like this:
\x40\x00
I guess the proper function to use would be struct.pack, but I can't find a way to make it properly work.
A small help would be very appreciated !
Thanks

You're not showing the whole code, so the simplest solution would be:
data = data[1] + data[0]
If you insist on using struct:
>>> from struct import pack, unpack
>>> unpack('<H', '\x12\x13')
(4882,)
>>> pack('>H', *unpack('<H', '\x12\x13'))
'\x13\x12'
Which first unpacks the string as a little-endian unsigned short, and then packs it back as big-endian unsigned short. You can have it the other way around, of course. When converting between BE and LE it doesn't matter which way you're converting - the conversion function is bi-directional.

data[::-1] works for any number of bytes.

little_endian = big_endian[1] + big_endian[0]

I could image that what would really want to do here is to convert the input data from big endian (or network) byteorder to your host byteorder (whatever that may be).
>>> from struct import unpack
>>> result = unpack('>H', '\x00\x40')
This would be a more portable approach than just swapping, which is bound to fail when the code is moved to a big endian machine that would not need to swap at all.

What is a suitable buffer for Python's struct module

In Python I'm accessing a binary file by reading it into a string and then using struct.unpack(...). Now I want to write to that string using struct.pack_into(...), but I get the error "Cannot use string as modifiable buffer". What would be a suitable buffer for use with the struct module?

As noted in another answer, struct_pack is probably all you need and should use. However, objects of type array support the buffer protocol and can be modified:
>>> import array, struct
>>> a = array.array('c', ' ' * 1000)
>>> c = 'a'; i = 1
>>> struct.pack_into('ci', a, -0, c, i)
>>> a
array('c', 'a\x00\x00\x00\x01\x00\x00\x00 ...
The original buffer protocol was a bit of a hack primarily for C extensions. It has been deprecated and replaced by a new C-level buffer API and memoryview objects in Python 3 (and in the upcoming 2.7).

If you aren't trying to pack it into a specific object, just use struct.pack to return a string.
Otherwise, ctypes.create_string_buffer is one way to obtain a mutable buffer.

Two possibilities leap immediately to mind:
You can use the Python stringio module to make a read/write buffer with file semantics.
You can use the Python array module to get a buffer you can treat like a list, but which will contain just binary bytes.