I have looked at the modules humanize and humanfriendly, and neither can convert a large bit value to human readable bit output (e.g. Mbits, Gbits, Tbits, ..etc). Has anyone come across such a module? Example:
mbits = 1000000
gbits = 1000000000
Then
print(human.bits(mbits)) # would output "1 Mbit"
print(human.bits(gbits)) # would output "1 Gbit"
...etc, up to exabit.
You can try hurry.filesize
>>> from hurry.filesize import size
>>> size(11000)
'10K'
There is another library bitmath
>>> from bitmath import *
>>> small_number = MiB(10000)
>>> print small_number.best_prefix()
9.765625 GiB
Related
I am using the answer provided here to convert string IP to integer. But I am not getting expected output. Specifically, the method is
>>> ipstr = '1.2.3.4'
>>> parts = ipstr.split('.')
>>> (int(parts[0]) << 24) + (int(parts[1]) << 16) + \
(int(parts[2]) << 8) + int(parts[3])
but when I provide it the value 172.31.22.98 I get 2887718498 back. However, I expect to see value -1407248798 as provided by Google's Guava library https://google.github.io/guava/releases/20.0/api/docs/com/google/common/net/InetAddresses.html#fromInteger-int-
Also, I've verified that this service provides expected output but all of the answers provided by the aforementioned StackOverflow answer return 2887718498
Note that I cannot use any third party library. So I am pretty much limited to using a hand-written code (no imports)
A better way is to use the library method
>>> from socket import inet_aton
>>> int.from_bytes(inet_aton('172.31.22.98'), byteorder="big")
2887718498
This is still the same result you had above
Here is one way to view it as a signed int
>>> from ctypes import c_long
>>> c_long(2887718498)
c_long(-1407248798)
To do it without imports (but why? The above is all first party CPython)
>>> x = 2887718498
>>> if x >= (1<<31): x-= (1<<32)
>>> x
-1407248798
Found this post whilst trying to do the same as OP. Developed the below for my simple mind to understand and for someone to benefit from.
baseIP = '192.168.1.0'
baseIPFirstOctet = (int((baseIP).split('.')[0]))
baseIPSecondOctet = (int((baseIP).split('.')[1]))
baseIPThirdOctet = (int((baseIP).split('.')[2]))
baseIPFourthOctet = (int((baseIP).split('.')[3]))
I have binary data inside a bytearray that I would like to gzip first and then post via requests. I found out how to gzip a file but couldn't find it out for a bytearray. So, how can I gzip a bytearray via Python?
Have a look at the zlib-module of Python.
Python 3: zlib-module
A short example:
import zlib
compressed_data = zlib.compress(my_bytearray)
You can decompress the data again by:
decompressed_byte_data = zlib.decompress(compressed_data)
Python 2: zlib-module
A short example:
import zlib
compressed_data = zlib.compress(my_string)
You can decompress the data again by:
decompressed_string = zlib.decompress(compressed_data)
As you can see, Python 3 uses bytearrays while Python 2 uses strings.
In case the bytearray is not too large to be stored in memory more than once and known as b, you can just:
b_gz = str(b).encode('zlib')
If you need to do deocding first, have a look at the decode() method of the bytearray.
The zlib module of Python Standard Library should meet your requirements :
>>> import zlib
>>> a = b'abcdefghijklmn' * 10
>>> ca = zlib.compress(a)
>>> len(a)
140
>>> len(ca)
25
>>> b = zlib.decompress(ca)
>>> b == a
True
>>> b
b'abcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmnabcdefghijklmn'
This is the output under Python3.4, but it works same under Python 2.7 -
import zlib
import binascii
def compress_packet(packet):
return zlib.compress(buffer(packet),1)
def decompress_packet(compressed_packet):
return zlib.decompress(compressed_packet)
def demo_zlib() :
packet1 = bytearray()
packet1.append(0x41)
packet1.append(0x42)
packet1.append(0x43)
packet1.append(0x44)
print "before compression: packet:{0}".format(binascii.hexlify(packet1))
cpacket1 = compress_packet(packet1)
print "after compression: packet:{0}".format(binascii.hexlify(cpacket1))
print "before decompression: packet:{0}".format(binascii.hexlify(cpacket1))
dpacket1 = decompress_packet(buffer(cpacket1))
print "after decompression: packet:{0}".format(binascii.hexlify(dpacket1))
def main() :
demo_zlib()
if __name__ == '__main__' :
main()
This should do. The zlib requires access to bytearray content, use buffer() for that.
I need to save a tuple of 4 numbers inside a column that only accepts numbers (int or floats)
I have a list of 4 number like -0.0123445552, -29394.2393339, 0.299393333, 0.00002345556.
How can I "store" all these numbers inside a number and be able to retrieve the original tuple in Python?
Thanks
Following up on #YevgenYampolskiy's idea of using numpy:
You could use numpy to convert the numbers to 16-bit floats, and then view the array as one 64-bit int:
import numpy as np
data = np.array((-0.0123445552, -29394.2393339, 0.299393333, 0.00002345556))
stored_int = data.astype('float16').view('int64')[0]
print(stored_int)
# 110959187158999634
recovered = np.array([stored_int], dtype='int64').view('float16')
print(recovered)
# [ -1.23443604e-02 -2.93920000e+04 2.99316406e-01 2.34842300e-05]
Note: This requires numpy version 1.6 or better, as this was the first version to support 16-bit floats.
If by int you mean the datatype int in Python (which is unlimited as of the current version), you may use the following solution
>>> x
(-0.0123445552, -29394.2393339, 0.299393333, 2.345556e-05)
>>> def encode(data):
sz_data = str(data)
import base64
b64_data = base64.b16encode(sz_data)
int_data = int(b64_data, 16)
return int_data
>>> encode(x)
7475673073900173755504583442986834619410853148159171975880377161427327210207077083318036472388282266880288275998775936614297529315947984169L
>>> def decode(data):
int_data = data
import base64
hex_data = hex(int_data)[2:].upper()
if hex_data[-1] == 'L':
hex_data = hex_data[:-1]
b64_data = base64.b16decode(hex_data)
import ast
sz_data = ast.literal_eval(b64_data)
return sz_data
>>> decode(encode(x))
(-0.0123445552, -29394.2393339, 0.299393333, 2.345556e-05)
You can combine 4 integers into a single integer, or two floats into a double using struct module:
from struct import *
s = pack('hhhh', 1, -2, 3,-4)
i = unpack('Q', pack('Q', i[0]))
print i
print unpack('hhhh', s)
s = pack('ff', 1.12, -2.32)
f = unpack('d', s)
print f
print unpack('ff', pack('d', f[0]))
prints
(18445618190982447105L,)
(1, -2, 3, -4)
(-5.119999879002571,)
(1.1200000047683716, -2.319999933242798)
Basically in this example tuple (1,-2,3,-4) gets packed into an integer 18445618190982447105, and tuple ( 1.12, -2.32) gets packed into -5.119999879002571
To pack 4 floats into a single float you will need to use half-floats, however this is a problem here:
With half-float it looks like there is no native support in python as of now:
http://bugs.python.org/issue11734
However numpy module do have some support for half-floats (http://docs.scipy.org/doc/numpy/user/basics.types.html). Maybe you can use it somehow to pack 4 floats into a single float
This does not really answer your question, but what you're trying to do violates 1NF. Is changing the DB schema to introduce an intersection table really not an option?
my idea is weird; but will it work??
In [31]: nk="-0.0123445552, -29394.2393339, 0.299393333, 0.00002345556"
In [32]: nk1="".join(str(ord(x)) for x in nk)
In [33]: nk1
Out[33]: '454846484950515252535353504432455057515752465051575151515744324846505757515751515151443248464848484850515253535354'
In [34]: import math
In [35]: math.log(long(nk1), 1000)
Out[36]: 37.885954947611985
In [37]: math.pow(1000,_)
Out[37]: 4.548464849505043e+113
you can easily unpack this string(Out[33]); for example split it at 32; its for space.
also this string is very long; we can make it to a small number by math.log; as we got in Out[36].
import binascii
f = open('file.ext', 'rb')
print binascii.hexlify(f.read(4))
f.close()
This prints:
84010100
I know that I must retrieve the hex number 184 out of this data.
How can it be done in python? I've used the struct module before, but I don't know if its little endian, big..whatever.. how can I get 184 from this number using struct?
>>> x = b'\x84\x01\x01\x00'
>>> import struct
>>> struct.unpack_from('<h', x)
(388,)
>>> map(hex, struct.unpack_from('<h', x))
['0x184']
< means little endian, h means read a 16-bit integer ("short"). Detail is in the package doc.
Is there anyway i can know how much bytes taken by particular variable in python. E.g; lets say i have
int = 12
print (type(int))
it will print
<class 'int'>
But i wanted to know how many bytes it has taken on memory? is it possible?
You can find the functionality you are looking for here (in sys.getsizeof - Python 2.6 and up).
Also: don't shadow the int builtin!
import sys
myint = 12
print(sys.getsizeof(myint))
if you want to know size of int, you can use struct
>>> import struct
>>> struct.calcsize("i")
4
otherwise, as others already pointed out, use getsizeof (2.6). there is also a recipe you can try.
In Python >= 2.6 you can use sys.getsizeof.
Numpy offers infrastructure to control data size. Here are examples (py3):
import numpy as np
x = np.float32(0)
print(x.nbytes) # 4
a = np.zeros((15, 15), np.int64)
print(a.nbytes) # 15 * 15 * 8 = 1800
This is super helpful when trying to submit data to the graphics card with pyopengl, for example.
You could also take a look at Pympler, especially its asizeof module, which unlike sys.getsizeof works with Python >=2.2.
on python command prompt, you can use size of function
$ import python
$ import ctypes
$ ctypes.sizeof(ctypes.c_int)
and read more on it from https://docs.python.org/2/library/ctypes.html
In Python 3 you can use sys.getsizeof().
import sys
myint = 12
print(sys.getsizeof(myint))
The best library for that is guppy:
import guppy
import inspect
def get_object_size(obj):
h = guppy.hpy()
callers_local_vars = inspect.currentframe().f_back.f_locals.items()
vname = "Constant"
for var_name, var_val in callers_local_vars:
if var_val == obj:
vname = str(var_name)
size = str("{0:.2f} GB".format(float(h.iso(obj).domisize) / (1024 * 1024)))
return str("{}: {}".format(vname, size))
The accepted answer sys.getsizeof is correct.
But looking at your comment about the accepted answer you might want the number of bits a number is occupying in binary. You can use bit_length
(16).bit_length() # '10000' in binary
>> 5
(4).bit_length() # '100' in binary
>> 3