I am trying to validate a dict which contains stringified integers as keys. These ints are arbitrary numerical IDs, and out of my control.
How could I declare this in a schema?
Maybe there's a way to declare a default schema which I could use as a catch-all?
Thanks.
http://marshmallow.readthedocs.org/en/latest/index.html
The way I prefer, use a map instead of if statement:
for key, value in dictionary.items():
dictionary[
{
"True": str(key),
"False": <build a random key>
}.get(str(isinstance(key, int)))
] = value
Note: works only on Python3+
Related
I am using the python library cerberus (http://docs.python-cerberus.org/en/stable/) and I want to check if a JSON field is a number (integer) or an empty string.
I tried using the condition:
{"empty": True, "type": "intenger"}
But when the field is an empty string, for example: (""), I get the following error.
'must be of integer type'
Is there a way of using the basic validation rules so it detects also an empty string in a numeric field?, I know it can be done by using extended validation functions but I want to avoid that solution for the moment.
Try something like this:
{"anyof":[
{"type":"string","allowed":[""]},
{"anyof_type":["float","integer"]}
]},
I would advise to not overcomplicate schemas. 1) Multiple types can be declared for the type rule. 2) The empty rule is only applied to sizable values, so it would ignore any given integer. Hence this is the simplest possible rules set for your constraints:
{'type': ('integer', 'string'),
'empty': True}
Mind that this doesn't enforce the value to be an empty string, but allows it to be, vulgo: a non-empty string would also pass. You may want to use the max_lengh rule w/ 0 as constraint instead.
I need to extract out a specific value from within a nested key in python
for example from the below I want to extract out the Start from the key Params
{'Key': 'Params', 'Value': `'{"Shut":false,"Remove":false,"SnapshotRequired":false,"Start":"Never","End":"Never"}'}
This is as far as I have got
for tag in i["Tags"]:
if 'Params' in tag['Key']:
then I can get the value but this is the whole string.
You can use dict.get(key) method to get a value of the key you specify in get(), where dict is variable which stores your dictionary.
Also you can use dict[key]. It gives the same result.
In your case, for example, dict['Key'] will return 'Params',
dict['Value'] will return nested dictionary.
Ok after fixing an issue in the string, an extra '`' you can simply make dict(string) to make it a valid python dict, which then accepts nested dicts as values.
also have in mine that "false" is not a valid python type, so you would have to make it a string or convert to python format which is False with the capital F
and finally do:
variable['Value']['Start']
test = dict({'Key': 'Params', 'Value': {"Shut":False,"Remove":False,"SnapshotRequired":False,"Start":"Never","End":"Never"}})
test['Value']['Start']
I have a c# background and I am trying to pass a dictionary in method and some some I want to check if a specific key exists or not. If that key exists in dictionary and then I want to perform few other things.
Let suppose if have following dictionary:
tel = {'jack': 4098, 'sape': 4139}
and I want to check whether key 'sape' exists or not? In method somehow, I do not know the type of argument. How can I cast to dictionary in order to retrieve keys and values.
In method I want to check:
keys = tel.keys () ;
keys.__contains__('sap')
do something
However, in method, I do not have information about the type of passed argument.
Few things to note about Python:
Python is not statically typed like C#. The function/method signature doesn't contain the data types of the parameters. So you do not need to cast the passed in value to a dictionary. Instead you assume that what is passed in is a dictionary and work on it. If you expect that something other than dictionary will be passed you can either use exception handling to catch any issues or you can use isinstance to check what is passed in is what you expect. Having said that casting does exist in python; for example you can convert an int 1 to a string by str(1).
Also note that if use the in keyword it will work for any iterable not just dictionary.
Example:
>>> 'jack' in 'lumberjack'
True
>>> tel = {'jack': 4098, 'sape': 4139}
>>> 'jack' in tel
True
>>>
If you really want to check if something is a dictionary you can do like this:
>>> isinstance(tel,dict)
True
I have found that when the following is run, python's json module (included since 2.6) converts int dictionary keys to strings.
>>> import json
>>> releases = {1: "foo-v0.1"}
>>> json.dumps(releases)
'{"1": "foo-v0.1"}'
Is there any easy way to preserve the key as an int, without needing to parse the string on dump and load.
I believe it would be possible using the hooks provided by the json module, but again this still requires parsing.
Is there possibly an argument I have overlooked?
Sub-question:
Thanks for the answers. Seeing as json works as I feared, is there an easy way to convey key type by maybe parsing the output of dumps?
Also I should note the code doing the dumping and the code downloading the json object from a server and loading it, are both written by me.
This is one of those subtle differences among various mapping collections that can bite you. JSON treats keys as strings; Python supports distinct keys differing only in type.
In Python (and apparently in Lua) the keys to a mapping (dictionary or table, respectively) are object references. In Python they must be immutable types, or they must be objects which implement a __hash__ method. (The Lua docs suggest that it automatically uses the object's ID as a hash/key even for mutable objects and relies on string interning to ensure that equivalent strings map to the same objects).
In Perl, Javascript, awk and many other languages the keys for hashes, associative arrays or whatever they're called for the given language, are strings (or "scalars" in Perl). In perl $foo{1}, $foo{1.0}, and $foo{"1"} are all references to the same mapping in %foo --- the key is evaluated as a scalar!
JSON started as a Javascript serialization technology. (JSON stands for JavaScript Object Notation.) Naturally it implements semantics for its mapping notation which are consistent with its mapping semantics.
If both ends of your serialization are going to be Python then you'd be better off using pickles. If you really need to convert these back from JSON into native Python objects I guess you have a couple of choices. First you could try (try: ... except: ...) to convert any key to a number in the event of a dictionary look-up failure. Alternatively, if you add code to the other end (the serializer or generator of this JSON data) then you could have it perform a JSON serialization on each of the key values --- providing those as a list of keys. (Then your Python code would first iterate over the list of keys, instantiating/deserializing them into native Python objects ... and then use those for access the values out of the mapping).
No, there is no such thing as a Number key in JavaScript. All object properties are converted to String.
var a= {1: 'a'};
for (k in a)
alert(typeof k); // 'string'
This can lead to some curious-seeming behaviours:
a[999999999999999999999]= 'a'; // this even works on Array
alert(a[1000000000000000000000]); // 'a'
alert(a['999999999999999999999']); // fail
alert(a['1e+21']); // 'a'
JavaScript Objects aren't really proper mappings as you'd understand it in languages like Python, and using keys that aren't String results in weirdness. This is why JSON always explicitly writes keys as strings, even where it doesn't look necessary.
Answering your subquestion:
It can be accomplished by using json.loads(jsonDict, object_hook=jsonKeys2int)
def jsonKeys2int(x):
if isinstance(x, dict):
return {int(k):v for k,v in x.items()}
return x
This function will also work for nested dicts and uses a dict comprehension.
If you want to to cast the values too, use:
def jsonKV2int(x):
if isinstance(x, dict):
return {int(k):(int(v) if isinstance(v, unicode) else v) for k,v in x.items()}
return x
Which tests the instance of the values and casts them only if they are strings objects (unicode to be exact).
Both functions assumes keys (and values) to be integers.
Thanks to:
How to use if/else in a dictionary comprehension?
Convert a string key to int in a Dictionary
Alternatively you can also try converting dictionary to a list of [(k1,v1),(k2,v2)] format while encoding it using json, and converting it back to dictionary after decoding it back.
>>>> import json
>>>> json.dumps(releases.items())
'[[1, "foo-v0.1"]]'
>>>> releases = {1: "foo-v0.1"}
>>>> releases == dict(json.loads(json.dumps(releases.items())))
True
I believe this will need some more work like having some sort of flag to identify what all parameters to be converted to dictionary after decoding it back from json.
I've gotten bitten by the same problem. As others have pointed out, in JSON, the mapping keys must be strings. You can do one of two things. You can use a less strict JSON library, like demjson, which allows integer strings. If no other programs (or no other in other languages) are going to read it, then you should be okay. Or you can use a different serialization language. I wouldn't suggest pickle. It's hard to read, and is not designed to be secure. Instead, I'd suggest YAML, which is (nearly) a superset of JSON, and does allow integer keys. (At least PyYAML does.)
Here is my solution! I used object_hook, it is useful when you have nested json
>>> import json
>>> json_data = '{"1": "one", "2": {"-3": "minus three", "4": "four"}}'
>>> py_dict = json.loads(json_data, object_hook=lambda d: {int(k) if k.lstrip('-').isdigit() else k: v for k, v in d.items()})
>>> py_dict
{1: 'one', 2: {-3: 'minus three', 4: 'four'}}
There is filter only for parsing json key to int. You can use int(v) if v.lstrip('-').isdigit() else v filter for json value too.
Convert the dictionary to be string by using str(dict) and then convert it back to dict by doing this:
import ast
ast.literal_eval(string)
I made a very simple extension of Murmel's answer which I think will work on a pretty arbitrary dictionary (including nested) assuming it can be dumped by JSON in the first place. Any keys which can be interpreted as integers will be cast to int. No doubt this is not very efficient, but it works for my purposes of storing to and loading from json strings.
def convert_keys_to_int(d: dict):
new_dict = {}
for k, v in d.items():
try:
new_key = int(k)
except ValueError:
new_key = k
if type(v) == dict:
v = _convert_keys_to_int(v)
new_dict[new_key] = v
return new_dict
Assuming that all keys in the original dict are integers if they can be cast to int, then this will return the original dictionary after storing as a json.
e.g.
>>>d = {1: 3, 2: 'a', 3: {1: 'a', 2: 10}, 4: {'a': 2, 'b': 10}}
>>>convert_keys_to_int(json.loads(json.dumps(d))) == d
True
[NSFW] You can write your json.dumps by yourself, here is a example from djson: encoder.py. You can use it like this:
assert dumps({1: "abc"}) == '{1: "abc"}'
In python I have a dictionary that maps tuples to a list of tuples. e.g.
{(1,2): [(2,3),(1,7)]}
I want to be able to encode this data use it with javascript, so I looked into json but it appears keys must be strings so my tuple does not work as a key.
Is the best way to handle this is encode it as "1,2" and then parse it into something I want on the javascript? Or is there a more clever way to handle this.
You might consider saying
{"[1,2]": [(2,3),(1,7)]}
and then when you need to get the value out, you can just parse the keys themselves as JSON objects, which all modern browsers can do with the built-in JSON.parse method (I'm using jQuery.each to iterate here but you could use anything):
var myjson = JSON.parse('{"[1,2]": [[2,3],[1,7]]}');
$.each(myjson, function(keystr,val){
var key = JSON.parse(keystr);
// do something with key and val
});
On the other hand, you might want to just structure your object differently, e.g.
{1: {2: [(2,3),(1,7)]}}
so that instead of saying
myjson[1,2] // doesn't work
which is invalid Javascript syntax, you could say
myjson[1][2] // returns [[2,3],[1,7]]
If your key tuples are truly integer pairs, then the easiest and probably most straightforward approach would be as you suggest.... encode them to a string. You can do this in a one-liner:
>>> simplejson.dumps(dict([("%d,%d" % k, v) for k, v in d.items()]))
'{"1,2": [[2, 3], [1, 7]]}'
This would get you a javascript data structure whose keys you could then split to get the points back again:
'1,2'.split(',')
My recommendation would be:
{ "1": [
{ "2": [[2,3],[1,7]] }
]
}
It's still parsing, but depending on how you use it, it may be easier in this form.
You can't use an array as a key in JSON. The best you can do is encode it. Sorry, but there's really no other sane way to do it.
Could it simply be a two dimensional array? Then you may use integers as keys