I'm trying to convert some hex input into decimal RGB. Apologies if the code is bad, I'm new at python but I'm getting the error in the title even tho the input is a string? (I already printed the stuff to be converted into decimal before converting and they are all string)
v1 = input("Enter the first pixel value: ")
v2 = input("Enter the second pixel value: ")
#px_to_gen = input ("Enter the number of pixels to generate in between: ")
#group (r g b) values of v1 & v2 in a list
v1_ls = ["".join((v1[2],v1[3])),"".join((v1[4],v1[5])),"".join((v1[6],v1[7]))]
v2_ls = ["".join((v2[2],v2[3])),"".join((v2[4],v2[5])),"".join((v2[6],v2[7]))]
print(v1_ls)
print(v2_ls)
#convert values in v1_ls & v2_ls to decimal
i = 0
while (i <= 2):
v1_ls[i] = int(v1_ls[i], 16)
v2_ls[i] = int(v2_ls[i], 16)
print(v1_ls)
print(v2_ls)
I assume you're facing this error: TypeError: int() can't convert non-string with explicit base
This is because you're trying to convert a list directly to integer with base conversion.
To fix this you can try below code, (converting into string first).
v1_ls[i] = int(str(v1_ls[i]), 16)
v2_ls[i] = int(str(v2_ls[i]), 16)
But please remember that you need to fix a big issue with your while loop - it'll go in a infinite loop.
Here's the code snippet from my RFID wiegand reader on my Raspberry Pi I use already.
def main():
set_procname("Wiegand Reader")
global bits
global timeout
GPIO.add_event_detect(D0, GPIO.FALLING, callback=one)
GPIO.add_event_detect(D1, GPIO.FALLING, callback=zero)
GPIO.add_event_detect(S1, GPIO.FALLING, callback=unlockDoor)
while 1:
if bits:
timeout = timeout -1
time.sleep(0.001)
if len(bits) > 1 and timeout == 0:
#print "Binary:", int(str(bits),2)
c1 = int(str(bits),2)
result = ((~c1) >> 1) & 0x0FFFFFF;
checkAccess(result, doorID)
else:
time.sleep(0.001)
if __name__ == '__main__':
main()
On a normal USB RFID reader, I get 0000119994 and that's what's printed on the card. But with this code it reads 119994. I've tried multiple cards. It always drops the zeros at the front .
I even tried a card with a zero in it. 0000120368 and it shows 120368
I thought it was taking off the first 4 characters but then I tried a key fob that only had 3 zeros in front. 0004876298 and it reads 4876298. Only dropping the front zeros.
Python will remove the front few bits if they are zero, this also applies to integers. For example
>>> a = 0003
>>> a
3
>>> b = 0b0011
>>> bin(b)
0b11
From what I see, all RFID's will have 10 numbers. You can make a simple program to add those numbers in and store the value as a string:
def rfid_formatter(value):
str_value = str(value)
for s in range(10 - len(str_value)):
str_value = "0" + str_value
return str_value
Your test cases:
print rfid_formatter(120368)
print "0000120368"
print rfid_formatter(4876298)
print "0004876298"
As mentioned already, leading zeros are removed in binary sequences and also when you explicitly convert a string to decimal using int().
What hasn't been mentioned already is that, in Python 2.x, integers with leading zeros are treated as octal values.
>>> a = 0003
>>> a
3
>>> a = 000127
>>> a
87
Since this was causing confusion, the implicit octal conversion was removed in Python 3 and any number of leading zeros in numerical values will raise a SyntaxError.
>>> a = 000127
File "<stdin>", line 1
a = 000127
^
SyntaxError: invalid token
>>>
You can read the rationale behind these decisions in PEP 3127.
Anyway, I mention all of this simply to arrive at an assumption: you're probably not working with octal representations. Instead, I think you're converting result to a string in checkAccess so you can do a string comparison. If this assumption is correct, you can simply use the string method zfill (zero fill):
>>> str(119994).zfill(10)
'0000119994'
>>>
>>> str(4876298).zfill(10)
'0004876298'
>>>
I have the following:
I read 30 bits from a bitstream:
MMSI = b.readlist('uint:30')
This seems to work normally except when the values get higher.
MMSI = b.readlist('uint:30')
p = 972128254
# repr(MMSI)[:-1]
print p
print "MMSI :"
print MMSI
if MMSI == p:
The code above outputs:
972128254
MMSI :
[972128254L]
The whole if MMSI ==p: is skipped for it is not equal for some reason.
I do not understand why the value that is far lower than max.int:
>>> import sys
>>> sys.maxint
2147483647
I do not understand why I get a Long returned and not a uint?
If the value returned is 244123456 it works like a charm.
2147483647 is maxint, but an int is 32 bits and you're using 30 bits. So your max is 1/4 of that, or about 500 million.
Values will be 'long' if an intermediate value was a long. So for example 2**1000 / 2**999 will equal 2L. This is just to do with the internals of the method you called and shouldn't affect most code.
The real problem is the comparison you have in your code is comparing an int to a list, which is not what you want to do. You can either use the read method rather than readlist to return a single item, or take the first element of the returned list: if MMSI[0] == p:
Aha, I figured that if I read 30bits and returned it as uint value it would automatically be a 32bits value.
So what you are saying is that I would have to add leading zeros to make a 32 bit value and then it should work.
So that is what I have tested and now I am lost.
I figured lets encode the value I want to compare to in the same way. So this is what I did:
from bitarray import bitarray
from bitstring import BitArray, BitStream,pack
from time import sleep
import string
MMSI = b.readlist('uint:30')
x = pack('uint:30',972000000)
p = x.readlist('uint:30')
y = pack('uint:30',972999999)
q = y.read('uint:30')
print p
print q
print x
print y
print MMSI
resulting in:
p = [972000000L]
q = 972999999
x = 0b111001111011111000101100000000
y = 0b111001111111101100110100111111
MMSI = [972128254L]
How can it be that the higher value 972999999 is not a long?
I need to save a tuple of 4 numbers inside a column that only accepts numbers (int or floats)
I have a list of 4 number like -0.0123445552, -29394.2393339, 0.299393333, 0.00002345556.
How can I "store" all these numbers inside a number and be able to retrieve the original tuple in Python?
Thanks
Following up on #YevgenYampolskiy's idea of using numpy:
You could use numpy to convert the numbers to 16-bit floats, and then view the array as one 64-bit int:
import numpy as np
data = np.array((-0.0123445552, -29394.2393339, 0.299393333, 0.00002345556))
stored_int = data.astype('float16').view('int64')[0]
print(stored_int)
# 110959187158999634
recovered = np.array([stored_int], dtype='int64').view('float16')
print(recovered)
# [ -1.23443604e-02 -2.93920000e+04 2.99316406e-01 2.34842300e-05]
Note: This requires numpy version 1.6 or better, as this was the first version to support 16-bit floats.
If by int you mean the datatype int in Python (which is unlimited as of the current version), you may use the following solution
>>> x
(-0.0123445552, -29394.2393339, 0.299393333, 2.345556e-05)
>>> def encode(data):
sz_data = str(data)
import base64
b64_data = base64.b16encode(sz_data)
int_data = int(b64_data, 16)
return int_data
>>> encode(x)
7475673073900173755504583442986834619410853148159171975880377161427327210207077083318036472388282266880288275998775936614297529315947984169L
>>> def decode(data):
int_data = data
import base64
hex_data = hex(int_data)[2:].upper()
if hex_data[-1] == 'L':
hex_data = hex_data[:-1]
b64_data = base64.b16decode(hex_data)
import ast
sz_data = ast.literal_eval(b64_data)
return sz_data
>>> decode(encode(x))
(-0.0123445552, -29394.2393339, 0.299393333, 2.345556e-05)
You can combine 4 integers into a single integer, or two floats into a double using struct module:
from struct import *
s = pack('hhhh', 1, -2, 3,-4)
i = unpack('Q', pack('Q', i[0]))
print i
print unpack('hhhh', s)
s = pack('ff', 1.12, -2.32)
f = unpack('d', s)
print f
print unpack('ff', pack('d', f[0]))
prints
(18445618190982447105L,)
(1, -2, 3, -4)
(-5.119999879002571,)
(1.1200000047683716, -2.319999933242798)
Basically in this example tuple (1,-2,3,-4) gets packed into an integer 18445618190982447105, and tuple ( 1.12, -2.32) gets packed into -5.119999879002571
To pack 4 floats into a single float you will need to use half-floats, however this is a problem here:
With half-float it looks like there is no native support in python as of now:
http://bugs.python.org/issue11734
However numpy module do have some support for half-floats (http://docs.scipy.org/doc/numpy/user/basics.types.html). Maybe you can use it somehow to pack 4 floats into a single float
This does not really answer your question, but what you're trying to do violates 1NF. Is changing the DB schema to introduce an intersection table really not an option?
my idea is weird; but will it work??
In [31]: nk="-0.0123445552, -29394.2393339, 0.299393333, 0.00002345556"
In [32]: nk1="".join(str(ord(x)) for x in nk)
In [33]: nk1
Out[33]: '454846484950515252535353504432455057515752465051575151515744324846505757515751515151443248464848484850515253535354'
In [34]: import math
In [35]: math.log(long(nk1), 1000)
Out[36]: 37.885954947611985
In [37]: math.pow(1000,_)
Out[37]: 4.548464849505043e+113
you can easily unpack this string(Out[33]); for example split it at 32; its for space.
also this string is very long; we can make it to a small number by math.log; as we got in Out[36].
I want the shortest possible way of representing an integer in a URL. For example, 11234 can be shortened to '2be2' using hexadecimal. Since base64 uses is a 64 character encoding, it should be possible to represent an integer in base64 using even less characters than hexadecimal. The problem is I can't figure out the cleanest way to convert an integer to base64 (and back again) using Python.
The base64 module has methods for dealing with bytestrings - so maybe one solution would be to convert an integer to its binary representation as a Python string... but I'm not sure how to do that either.
This answer is similar in spirit to Douglas Leeder's, with the following changes:
It doesn't use actual Base64, so there's no padding characters
Instead of converting the number first to a byte-string (base 256), it converts it directly to base 64, which has the advantage of letting you represent negative numbers using a sign character.
import string
ALPHABET = string.ascii_uppercase + string.ascii_lowercase + \
string.digits + '-_'
ALPHABET_REVERSE = dict((c, i) for (i, c) in enumerate(ALPHABET))
BASE = len(ALPHABET)
SIGN_CHARACTER = '$'
def num_encode(n):
if n < 0:
return SIGN_CHARACTER + num_encode(-n)
s = []
while True:
n, r = divmod(n, BASE)
s.append(ALPHABET[r])
if n == 0: break
return ''.join(reversed(s))
def num_decode(s):
if s[0] == SIGN_CHARACTER:
return -num_decode(s[1:])
n = 0
for c in s:
n = n * BASE + ALPHABET_REVERSE[c]
return n
>>> num_encode(0)
'A'
>>> num_encode(64)
'BA'
>>> num_encode(-(64**5-1))
'$_____'
A few side notes:
You could (marginally) increase the human-readibility of the base-64 numbers by putting string.digits first in the alphabet (and making the sign character '-'); I chose the order that I did based on Python's urlsafe_b64encode.
If you're encoding a lot of negative numbers, you could increase the efficiency by using a sign bit or one's/two's complement instead of a sign character.
You should be able to easily adapt this code to different bases by changing the alphabet, either to restrict it to only alphanumeric characters or to add additional "URL-safe" characters.
I would recommend against using a representation other than base 10 in URIs in most cases—it adds complexity and makes debugging harder without significant savings compared to the overhead of HTTP—unless you're going for something TinyURL-esque.
All the answers given regarding Base64 are very reasonable solutions. But they're technically incorrect. To convert an integer to the shortest URL safe string possible, what you want is base 66 (there are 66 URL safe characters).
That code looks something like this:
from io import StringIO
import urllib
BASE66_ALPHABET = u"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz-_.~"
BASE = len(BASE66_ALPHABET)
def hexahexacontadecimal_encode_int(n):
if n == 0:
return BASE66_ALPHABET[0].encode('ascii')
r = StringIO()
while n:
n, t = divmod(n, BASE)
r.write(BASE66_ALPHABET[t])
return r.getvalue().encode('ascii')[::-1]
Here's a complete implementation of a scheme like this, ready to go as a pip installable package:
https://github.com/aljungberg/hhc
You probably do not want real base64 encoding for this - it will add padding etc, potentially even resulting in larger strings than hex would for small numbers. If there's no need to interoperate with anything else, just use your own encoding. Eg. here's a function that will encode to any base (note the digits are actually stored least-significant first to avoid extra reverse() calls:
def make_encoder(baseString):
size = len(baseString)
d = dict((ch, i) for (i, ch) in enumerate(baseString)) # Map from char -> value
if len(d) != size:
raise Exception("Duplicate characters in encoding string")
def encode(x):
if x==0: return baseString[0] # Only needed if don't want '' for 0
l=[]
while x>0:
l.append(baseString[x % size])
x //= size
return ''.join(l)
def decode(s):
return sum(d[ch] * size**i for (i,ch) in enumerate(s))
return encode, decode
# Base 64 version:
encode,decode = make_encoder("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/")
assert decode(encode(435346456456)) == 435346456456
This has the advantage that you can use whatever base you want, just by adding appropriate
characters to the encoder's base string.
Note that the gains for larger bases are not going to be that big however. base 64 will only reduce the size to 2/3rds of base 16 (6 bits/char instead of 4). Each doubling only adds one more bit per character. Unless you've a real need to compact things, just using hex will probably be the simplest and fastest option.
To encode n:
data = ''
while n > 0:
data = chr(n & 255) + data
n = n >> 8
encoded = base64.urlsafe_b64encode(data).rstrip('=')
To decode s:
data = base64.urlsafe_b64decode(s + '===')
decoded = 0
while len(data) > 0:
decoded = (decoded << 8) | ord(data[0])
data = data[1:]
In the same spirit as other for some “optimal” encoding, you can use 73 characters according to RFC 1738 (actually 74 if you count “+” as usable):
alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz_`\"!$'()*,-."
encoded = ''
while n > 0:
n, r = divmod(n, len(alphabet))
encoded = alphabet[r] + encoded
and the decoding:
decoded = 0
while len(s) > 0:
decoded = decoded * len(alphabet) + alphabet.find(s[0])
s = s[1:]
The easy bit is converting the byte string to web-safe base64:
import base64
output = base64.urlsafe_b64encode(s)
The tricky bit is the first step - convert the integer to a byte string.
If your integers are small you're better off hex encoding them - see saua
Otherwise (hacky recursive version):
def convertIntToByteString(i):
if i == 0:
return ""
else:
return convertIntToByteString(i >> 8) + chr(i & 255)
You don't want base64 encoding, you want to represent a base 10 numeral in numeral base X.
If you want your base 10 numeral represented in the 26 letters available you could use: http://en.wikipedia.org/wiki/Hexavigesimal.
(You can extend that example for a much larger base by using all the legal url characters)
You should atleast be able to get base 38 (26 letters, 10 numbers, +, _)
Base64 takes 4 bytes/characters to encode 3 bytes and can only encode multiples of 3 bytes (and adds padding otherwise).
So representing 4 bytes (your average int) in Base64 would take 8 bytes. Encoding the same 4 bytes in hex would also take 8 bytes. So you wouldn't gain anything for a single int.
a little hacky, but it works:
def b64num(num_to_encode):
h = hex(num_to_encode)[2:] # hex(n) returns 0xhh, strip off the 0x
h = len(h) & 1 and '0'+h or h # if odd number of digits, prepend '0' which hex codec requires
return h.decode('hex').encode('base64')
you could replace the call to .encode('base64') with something in the base64 module, such as urlsafe_b64encode()
If you are looking for a way to shorten the integer representation using base64, I think you need to look elsewhere. When you encode something with base64 it doesn't get shorter, in fact it gets longer.
E.g. 11234 encoded with base64 would yield MTEyMzQ=
When using base64 you have overlooked the fact that you are not converting just the digits (0-9) to a 64 character encoding. You are converting 3 bytes into 4 bytes so you are guaranteed your base64 encoded string would be 33.33% longer.
I maintain a little library named zbase62: http://pypi.python.org/pypi/zbase62
With it you can convert from a Python 2 str object to a base-62 encoded string and vice versa:
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> d = os.urandom(32)
>>> d
'C$\x8f\xf9\x92NV\x97\x13H\xc7F\x0c\x0f\x8d9}\xf5.u\xeeOr\xc2V\x92f\x1b=:\xc3\xbc'
>>> from zbase62 import zbase62
>>> encoded = zbase62.b2a(d)
>>> encoded
'Fv8kTvGhIrJvqQ2oTojUGlaVIxFE1b6BCLpH8JfYNRs'
>>> zbase62.a2b(encoded)
'C$\x8f\xf9\x92NV\x97\x13H\xc7F\x0c\x0f\x8d9}\xf5.u\xeeOr\xc2V\x92f\x1b=:\xc3\xbc'
However, you still need to convert from integer to str. This comes built-in to Python 3:
Python 3.2 (r32:88445, Mar 25 2011, 19:56:22)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> d = os.urandom(32)
>>> d
b'\xe4\x0b\x94|\xb6o\x08\xe9oR\x1f\xaa\xa8\xe8qS3\x86\x82\t\x15\xf2"\x1dL%?\xda\xcc3\xe3\xba'
>>> int.from_bytes(d, 'big')
103147789615402524662804907510279354159900773934860106838120923694590497907642
>>> x= _
>>> x.to_bytes(32, 'big')
b'\xe4\x0b\x94|\xb6o\x08\xe9oR\x1f\xaa\xa8\xe8qS3\x86\x82\t\x15\xf2"\x1dL%?\xda\xcc3\xe3\xba'
To convert from int to bytes and vice versa in Python 2, there is not a convenient, standard way as far as I know. I guess maybe I should copy some implementation, such as this one: https://github.com/warner/foolscap/blob/46e3a041167950fa93e48f65dcf106a576ed110e/foolscap/banana.py#L41 into zbase62 for your convenience.
I needed a signed integer, so I ended up going with:
import struct, base64
def b64encode_integer(i):
return base64.urlsafe_b64encode(struct.pack('i', i)).rstrip('=\n')
Example:
>>> b64encode_integer(1)
'AQAAAA'
>>> b64encode_integer(-1)
'_____w'
>>> b64encode_integer(256)
'AAEAAA'
I'm working on making a pip package for this.
I recommend you use my bases.py https://github.com/kamijoutouma/bases.py which was inspired by bases.js
from bases import Bases
bases = Bases()
bases.toBase16(200) // => 'c8'
bases.toBase(200, 16) // => 'c8'
bases.toBase62(99999) // => 'q0T'
bases.toBase(200, 62) // => 'q0T'
bases.toAlphabet(300, 'aAbBcC') // => 'Abba'
bases.fromBase16('c8') // => 200
bases.fromBase('c8', 16) // => 200
bases.fromBase62('q0T') // => 99999
bases.fromBase('q0T', 62) // => 99999
bases.fromAlphabet('Abba', 'aAbBcC') // => 300
refer to https://github.com/kamijoutouma/bases.py#known-basesalphabets
for what bases are usable
For your case
I recommend you use either base 32, 58 or 64
Base-64 warning: besides there being several different standards, padding isn't currently added and line lengths aren't tracked. Not recommended for use with APIs that expect formal base-64 strings!
Same goes for base 66 which is currently not supported by both bases.js and bases.py but it might in the future
I'd go the 'encode integer as binary string, then base64 encode that' method you suggest, and I'd do it using struct:
>>> import struct, base64
>>> base64.b64encode(struct.pack('l', 47))
'LwAAAA=='
>>> struct.unpack('l', base64.b64decode(_))
(47,)
Edit again:
To strip out the extra 0s on numbers that are too small to need full 32-bit precision, try this:
def pad(str, l=4):
while len(str) < l:
str = '\x00' + str
return str
>>> base64.b64encode(struct.pack('!l', 47).replace('\x00', ''))
'Lw=='
>>> struct.unpack('!l', pad(base64.b64decode('Lw==')))
(47,)
Pure python, no dependancies, no encoding of byte strings etc. , just turning a base 10 int into base 64 int with the correct RFC 4648 characters:
def tetrasexagesimal(number):
out=""
while number>=0:
if number == 0:
out = 'A' + out
break
digit = number % 64
out = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"[digit] + out
number /= 64 # //= 64 for py3 (thank spanishgum!)
if number == 0:
break
return out
tetrasexagesimal(1)
As it was mentioned here in comments you can encode a data using 73 characters that are not escaped in URL.
I found two places were this Base73 URL encoding is used:
https://git.nolog.cz/NoLog.cz/f.bain/src/branch/master/static/script.js JS based URL shortener
https://gist.github.com/LoneFry/3792021 in PHP
But in fact you may use more characters like /, [, ], :, ; and some others. Those characters are escaped only when you doing encodeURIComponent i.e. you need to pass data via get parameter.
So in fact you can use up to 82 characters. The full alphabet is !$&'()*+,-./0123456789:;=#ABCDEFGHIJKLMNOPQRSTUVWXYZ[]_abcdefghijklmnopqrstuvwxyz~. I sorted all the symbols by their code so when Base82URL numbers are sorted as plain strings they are keep the same order.
I tested in Chrome and Firefox and they are works fine but may be confusing for regular users. But I used such ids for an internal API calls where nobody sees them.
Unsigned integer 32 bit may have a maximum value of 2^32=4294967296
And after encoding to the Base82 it will take 6 chars: $0~]mx.
I don't have a code in Python but here is a JS code that generates a random id (int32 unsigned) and encodes it into the Base82URL:
/**
* Convert uint32 number to Base82 url safe
* #param {int} number
* #returns {string}
*/
function toBase82Url(number) {
// all chars that are not escaped in url
let keys = "!$&'()*+,-./0123456789:;=#ABCDEFGHIJKLMNOPQRSTUVWXYZ[]_abcdefghijklmnopqrstuvwxyz~"
let radix = keys.length
let encoded = []
do {
let index = number% radix
encoded.unshift(keys.charAt(index))
number = Math.trunc(number / radix)
} while (number !== 0)
return encoded .join("")
}
function generateToken() {
let buf = new Uint32Array(1);
window.crypto.getRandomValues(buf)
var randomInt = buf[0]
return toBase82Url(randomInt)
}