I am using caching for my site using using. This is giving the following error:
"Control/space characters not allowed (key="\xebw\x1b}\xae\xa3\xb8\x18\xc4\xb5\xce\x0c%\x13'\xed")".
The code which I am using is as follows:
def hash_key(key, key_prefix, version):
new_key = '%s :%s :%s' % (key_prefix, version, key)
if len(new_key) > 250:
m = hashlib.md5()
m.update(new_key)
new_key = m.digest()
return new_key
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1.11211',
'KEY_FUNCTION': hash_key,
}
}
Try using m.hexdigest() instead of m.digest(). The data in the error message is 16 bytes, the length of the binary hash data. It appears you want the 32-character ASCII representation, which is what hexdigest provides.
Docs, for Python 3
Related
Using docker python library,
UID = USER_ID = os.getuid()
GROUP_ID = os.getgid()
USER = getpass.getuser()
HOME = "~"
DIR = os.getcwd()
DOCKER_GID = execute_shell_command('"$(stat -c "%g" /var/run/docker.sock)"', log_path)
VERSION = "latest"
BASE_DOCKER = "ubuntu:20.04"
assert os.path.isfile(dockerfile_path)
build_args = {
"UID": UID,
"DOCKER_GID": DOCKER_GID,
"UNAME": USER,
"VER": VERSION,
"BASE_IMAGE": BASE_DOCKER,
}
client = client = docker.from_env()
client.images.build(fileobj=dockerfile, buildargs=build_args)
Getting
Bad Request ("error reading build args: json: cannot unmarshal number into Go value of type string")
This is not Googleable, so posting for future readers
Answer was found here
Convert numbers in build_args to strings to avoid this undocumented error:
build_args = {k: str(v) for k, v in {
"UID": UID,
"DOCKER_GID": DOCKER_GID,
"UNAME": USER,
"VER": VERSION,
"BASE_IMAGE": BASE_DOCKER,
}.items()}
"n": "rKZ-1zdz_CoLekSynOtyWv6cPSSkV28Kb9kZZHyYL-yhkKnH_bHl8OpWiGxQiKP0ulLRIaq1IhSMetkZ8FfXH-iptIDu4lPb8gt0HQYkjcy3HoaKRXBw2F8fJQO4jQ-ufR4l-E0HRqwLywzdtAImNWmju3A4kx8s0iSGHGSHyE4EUdh5WKt-NMtfUPfB5v9_2bC-w6wH7zAEsI5nscMXnvz1u8w7g2_agyhKSK0D9OkJ02w3I4xLMlrtKEv2naoBGerWckKcQ1kBYUh6WASPdvTqX4pcAJi7Tg6jwQXIP1aEq0JU8C0zE3d33kaMoCN3SenIxpRczRzUHpbZ-gk5PQ",
"e": "AQAB",
How can I generate a public key from these values? Preferable via python or Linux programs. I'm sorry if the question is nonsense because of invalid values.
The source is here.
In Python, you can use Python-JOSE
What you got there in your link is a JSON Web Key Set (JWKS), a JSON formated array of JSON Web Keys (JWK).
n and e are the modulus and exponent of a RSA public key.
The function jwk.construct can directly create a key from a JWK. The key can then be used e.g. to verify a JWT (key.verify).
You can also use jwt.decode like shown in the code below and pass the JWK directly, or in PEM format.
from jose import jwk
from jose.utils import base64url_decode
token = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IlctNjduZWt0WVRjOEpWWVBlV0g1c1dlN1JZVm5uMFN5NzQxZjhUT0pfQWMifQ.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWUsImlhdCI6MTUxNjIzOTAyMn0.hiKxeC66LIyVKOXjiOk7iScFPy_5-ATw7hEfqGij8sBZmwXAeTPT5BRFYHitFKSXomGqmy_63LLvg4zbhcTTmNf8XIeDAuLsC32soO5woSByisswWHVf8BgxMkI_FPW_oEtEQ8Xv3FL_1rF9j9Oy3jIjgjqhFhXUtsSQWAeuGYH-OQljFwiuO5Bqexcw-H71OEWvQLQof_6KJ0viJyte8QEwEVridyO834-ppHzeaoW2sTvZ22ZNfxPCew0Ul2V_TxHTtO7ZuJCZ81EmeIV6dYJ2GrYh3UN1x1PHy4-tEn-PL4otlaO3PYOcXfCHxHa6xtPsquzPZJnB1Vq8zULLfQ"
rsa_key = {
"kty": "RSA",
"kid": "W-67nektYTc8JVYPeWH5sWe7RYVnn0Sy741f8TOJ_Ac",
"use": "sig",
"alg": "RS256",
"n": "kFpGoVmBmmKepvBQiwq3hU9lIAuGsAPda4AVk712d3Z_QoS-5veGp4yltnyEFYyX867GOKDpbH7OF2uIjDg4-FPZwbuhiMscbkZzh25SQmfRtCT5ocUloQiopBcNAE-sd1p-ayUJWjhPrFoBrBLZHYxVEjY4JrWevQDj7kSeX7eJpud_VuZ77TNoIzj7d_iUuJUUlqF1ZF540igHKoVJJ6ujQLHh4ob8_izUuxX2iDq4h0VN3-uer59GsWw6OHgkOt85TsjMwYbeN9iw_7cNfLEYpSiH-sVHBCyKYQw7f8bKaChLxDRhUUTIEUUjGT9Ub_A3gOXq9TIi8BmbzrzVKQ",
"e": "AQAB"
}
key = jwk.construct(rsa_key)
message, encoded_sig = token.rsplit('.', 1)
decoded_sig = base64url_decode(encoded_sig + '=' * (4 - len(encoded_sig) % 4)) # looks weird, but without added padding I got errors
res = key.verify(bytes(message, "UTF-8"), decoded_sig)
# jwt.decode(token=token, key=key.to_pem().decode(), algorithms= 'RS256') # with PEM key
payload = jwt.decode(token=token, rsa_key, algorithms= 'RS256') # with JWK
print(res)
print(payload)
The result will be:
True
{'sub': '1234567890', 'name': 'John Doe', 'admin': True, 'iat': 1516239022}
which means the token could be verified with that key.
I have json file which has duplicate keys.
Example
{
"data":"abc",
"data":"xyz"
}
I want to make this as
{
"data1":"abc",
"data2":"xyz"
}
I tried using object_pairs_hook with json_loads, but it is not working. Could anyone one help me with Python solution for above problem
You can pass the load method a keyword parameter to handle pairing, there you can check for duplicates like this:
raw_text_data = """{
"data":"abc",
"data":"xyz",
"data":"xyz22"
}"""
def manage_duplicates(pairs):
d = {}
k_counter = Counter(defaultdict(int))
for k, v in pairs:
d[k+str(k_counter[k])] = v
k_counter[k] += 1
return d
print(json.loads(raw_text_data, object_pairs_hook=manage_duplicates))
I used Counter to count each key, if it already exists, I'm saving the key as k+str(k_counter[k) - so it will be added with a trailing number.
P.S
If you have control on the input, I would highly recommend to change your json structure to:
{"data": ["abc", "xyz"]}
The rfc 4627 for application/json media type recommends unique keys but it doesn't forbid them explicitly:
The names within an object SHOULD be unique.
A quick and dirty solution using re.
import re
s = '{ "data":"abc", "data":"xyz", "test":"one", "test":"two", "no":"numbering" }'
def find_dupes(s):
keys = re.findall(r'"(\w+)":', s)
return list(set(filter(lambda w: keys.count(w) > 1, keys)))
for key in find_dupes(s):
for i in range(1, len(re.findall(r'"{}":'.format(key), s)) + 1):
s = re.sub(r'"{}":'.format(key), r'"{}{}":'.format(key, i), s, count=1)
print(s)
Prints this string:
{
"data1":"abc",
"data2":"xyz",
"test1":"one",
"test2":"two",
"no":"numbering"
}
Having an odd time getting this to work;
I have a dictionary that contains the information for several machines. Based on a parameter, the machine is selected.
I would like to write the selected information to the log. But my attempts at recursion don't seem to get me where I need to go. I get the key but the value fails.
This is the dictionary
CSTU_CFG = {'A07': {
'password': 'CastAIP', # default cast password( too lazy to use LDAP)
'host':'JSIPVWCASTD01',
'port':'2280', # Ports are assumed to be 2280 but can be any
'location': 'C:Users/WDI/Documents/CSTU/DMPRST', # use os.path to convert
'gzips': 'GZIPS', # location for zip files ( Backup )
'schematype':{'local', 'central', 'mngt'},
'logintv': 30,
'version': '0.9'
},
'A01': {
'machine': 'A01',
'password': 'CastAIP', # default cast password( too lazy to use LDAP)
'host':'JSIPVWCASTD01',
'port':'2280', # Ports are assumed to be 2280 but can be any
'location': 'C:Users/WDI/Documents/CSTU/DMPRST', # use os.path to convert
'gzips': 'GZIPS', # location for zip files ( Backup )
'schematype':{'local', 'central', 'mngt'},
'logintv': 30,
'version': '0.9'
},
'A02': {
'machine': 'A02',
'password': 'CastAIP', # default cast password( too lazy to use LDAP)
'host':'JSIPVWCASTD01',
'port':'2280', # Ports are assumed to be 2280 but can be any
'location': 'C:Users/WDI/Documents/CSTU/DMPRST', # use os.path to convert
'gzips': 'GZIPS', # location for zip files ( Backup )
'schematype':{'local', 'central', 'mngt'},
'logintv': 30,
'version': '0.9'
}
}
logname = 'CSTU_'+timestr+'_'+ schemaname + '.CLOG'
logging.basicConfig(filename=logname,filemode='a',format='%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s', datefmt='%H:%M:%S',level=logging.DEBUG)
logging.debug("Starting CSTU_DUMP")
# print the CSTU_CFG file into the log
for key,value in CSTU_CFG:
logging.debug(key + " => " + value)
I'm obviously not getting the point on the logging. I've tried a few of the suggested fixes and I either get nothing, or various errors. I Can hard code it obviously but thats not the intent.
Thanks
You can iterate over dictionary to get key and that key's value:
for k, v in CSTU_CFG.iteritems():
logging.debug(k, v)
To iterate dictionary items you need to access the items first, not iterate the dictionary directly.
You need to change your code (for key,value in CSTU_CFG) as follows.
For Python 2.x:
for key, value in CSTU_CFG.iteritems():
logging.debug(key + " => " + str(value))
For Python 3.x:
for key, value in CSTU_CFG.items():
logging.debug(key + " => " + str(value))
By the way, you said you're getting errors. It may help to include the exact error trace next time in your question.
Here is my code:
def renren_get_sig(params):
cat_params = ''.join([u'%s=%s'%(unicode(k), unicode(params[k])) for k in sorted(params)])
sig = hashlib.md5(u"%s%s"%(unicode(cat_params), unicode(SEC_KEY))).hexdigest()
return sig
The exception message is:
Exception Type: UnicodeEncodeError
Exception Value: 'ascii' codec can't encode characters in position 138-141: ordinal not in range(128)
The dic params value is as the following:
params ={
'access_token':u'195036|6.3cf38700f.2592000.1347375600-462350295',
'action_link': u'http://wohenchun.xxx.com',
'action_name': u'\u6d4b\u8bd5\u4e00\u4e0b',
'api_key': u'8c0a2cded4f84bbba4328ccba22c3374',
'caption': u'\u7eaf\u6d01\u6307\u6570\u6d4b\u8bd5',
'description': u'\u4e16\u754c\u8fd9\u4e48\u4e71\uff0c\u88c5\u7eaf\u7ed9\u8c01\u770b\uff1f\u5230\u5e95\u4f60\u6709\u591a\u5355\u7eaf\uff0c\u8d85\u7ea7\u5185\u6db5\u7684\u4f60\uff0c\u6562\u4e0d\u6562\u6311\u6218\u8d85\u7ea7\u5185\u6db5\u7684\u9898\u76ee?\u4e0d\u7ba1\u4f60\u6d4b\u4e0d\u6d4b\uff0c\u53cd\u6b63\u6211\u662f\u6d4b\u4e86\uff01',
'format': u'JSON',
'image': u'http://hdn.xnimg.cn/photos/hdn21/20120809/1440/h0dd1376.jpg',
'message': u'\u5c3c\u?!! \u3010\u4f60\u96be\u9053\u6bd4\u6211\u66f4\u7eaf\u6d01\u4e48,\u6765\u6d4b\u8bd5\u4e00\u4e0b\u5427!\u4f20\u9001\u95e8 >> http://wohenchun.jiongceyan.com \u3011\r\n\t\t\t\t\t\t\t\t\t\t',
'method': u'feed.publishFeed',
'name': u'\u4eba\u4eba\u53f2\u4e0a\u6700\u706b\u7206\u6d4b\u8bd5\u4e4b\u5355\u7eaf\u6d4b\u8bd5',
'url': u'http://wohenchun.xxx.com',
'v': u'1.0'}
All the key-value pairs in params are Unicode objects. Why do I still get such an exception?
Thank you!
Unicode is the problem. Hashing algorithms are designed to be used with bytes, not unicode code points. So you must choose encoding and encode your unicode strings to byte strings before applying hashing algorithm:
from hashlib import md5
str_to_hash = unicode_str.encode('utf-8')
md5(str_to_hash).hexdigest()
There was an issue about this problem in Python tracker - investigate it for more information.
#Rostyslav has it right. Use byte strings with hashlib. May I also suggest using a source file encoding for readability? Check the message parameter. The original code had an error with \u?!! in the string. I left it out:
# coding: utf8
import hashlib
SEC_KEY = 'salt'
params = {
u'access_token' : u'195036|6.3cf38700f.2592000.1347375600-462350295',
u'action_link' : u'http://wohenchun.xxx.com',
u'action_name' : u'测试一下',
u'api_key' : u'8c0a2cded4f84bbba4328ccba22c3374',
u'caption' : u'纯洁指数测试',
u'description' : u'世界这么乱，装纯给谁看？到底你有多单纯，超级内涵的你，敢不敢挑战超级内涵的题目?不管你测不测，反正我是测了！',
u'format' : u'JSON',
u'image' : u'http://hdn.xnimg.cn/photos/hdn21/20120809/1440/h0dd1376.jpg',
u'message' : u'尼【你难道比我更纯洁么,来测试一下吧!传送门 >> http://wohenchun.jiongceyan.com 】\r\n\t\t\t\t\t\t\t\t\t\t',
u'method' : u'feed.publishFeed',
u'name' : u'人人史上最火爆测试之单纯测试',
u'url' : u'http://wohenchun.xxx.com',
u'v' : u'1.0'}
def renren_get_sig(params):
data = u''.join(u'{0}={1}'.format(k,v) for k,v in sorted(params.items()))
return hashlib.md5(data.encode('utf8') + SEC_KEY).hexdigest()
print renren_get_sig(params)
Output:
085b14d1384ba805d2d5d5e979913b27