Develop Reference

Read and parse logic in dictionary format

I'm trying to store some logic in dictionary format, then have a Python parser that read and evaluate, for example, in my dictionary I have
rule_dict = {
'logic': 'equal',
'value': '1'
}
And my parser is defined and used like below
def test_data_against_rule_dict(rule_dict, data):
if rule_dict['logic'] == 'equal'
return rule_dict['value'] == data
elif ...
data = 1
result = test_data_against_rule_dict(rule_dict, data)
# result = True
I need to handle other logic like not equal, is in given list, not in given list etc. I can define the name of these logic myself and write the parser for each logic myself, but I wonder if there's existing standard or Python library I can use directly.
Adding another example to explain another one of my use cases, e.g. for the not in given list example:
rule_dict = {
'logic': 'exclude',
'value': '1'
}
def test_data_against_rule_dict(rule_dict, data):
if rule_dict['logic'] == 'equal'
return rule_dict['value'] == data
elif rule_dict['logic'] == 'exclude'
return rule_dict['value'] not in data
...
data = [1,2,3]
result = test_data_against_rule_dict(rule_dict, data)
# result = False

I don't know any library to solve this task. However, you could use the eval method for less code (but the execution will be slower):
Change your rule_dict to the code that should be executed, like this:
rule = '1 not in data'
Your check becomes simple then:
return eval(rule)

Dynamic filter in Django

In my app, I have a form. Depending of the form, one or many filters could be configured by the user.
In my view, I have for exemple :
query = Test.objects.filter(filter1 = request.post['filter1'], filter2 = request.post['filter2'], filter3 = request.post['filter3'])
So, sometimes filter1, filter2 or filter3 could not exist.
If any filters doesn't exist, I just want to ignore the filter.
I could do a script with many "IF" conditions but may be there is a smart solution ?
Thanks for your help !

You could do something along the lines of:
filters = {}
for key, value in request.post.items():
if key in ['filter1', 'filter2', 'filter3']:
filters[key] = value
Test.objects.filter(**filters)
Where the list is a list of keys that you are intending to filter by
Edit
As Thomas Junk suggested you can make it a lot cleaner with a comprehension:
filters = {
key: value
for key, value in request.post.items()
if key in ['filter1', 'filter2', 'filter3']
}
Test.objects.filter(**filters)

This code solved my problem:
if request.method == 'GET':
filters = {}
for key, value in request.GET.items():
if value != '':
filters[key] = value
filter_list=Pet.objects.filter(**filters)

You could use something like this:
import ast
def query_to_dict(query) -> dict:
filters = {}
for field, value in query.items(): # <- param and value like ?param1=True
filters[field] = ast.literal_eval(str(value)) # <- parse list/str/bool
return filters
def event_view(request):
events = Event.objects.filter(
Q(**query_to_dict(request.GET))
)
return Response(
EventsSerializer(events, many=True).data
)
In this example:
ast - module which will help to parse bool values and lists
request.GET - this is your query params ?param1=True&param2="[1, 2]". Make sure to pass params in way like ?title="hello" - using double quotes
query_to_dict - function which will convert query params to python dict. So you can pass this dict to your Event.objects.filter.
Benefit of using filtering this way, you can pass ?id__in="[1, 2]" something like this and it will work. I can't say for sure how it secure or not, but for some complex filtering or bool fields filtering it is perfect.

Dynamic dict value access with dot separated string

I'm using Python 3.5.1
So what I am trying to do is pass in a dict a dot separated string representing the path to a key and a default value. I want to check for the keys existence and if it's not there , provide the default value. The problem with this is that the key I want to access could be nested in other dicts and I will not know until run time. So what I want to do is something like this:
def replace_key(the_dict, dict_key, default_value):
if dict_key not in the_dict:
the_dict[dict_key] = default_value
return the_dict
some_dict = {'top_property': {'first_nested': {'second_nested': 'the value'}}}
key_to_replace = 'top_property.first_nested.second_nested'
default_value = 'replaced'
#this would return as {'top_property': {'first_nested': {'second_nested': 'replaced'}}}
replace_key(some_dict, key_to_replace, default_value)
What I'm looking for is a way to do this without having to do a split on '.' in the string and iterating over the possible keys as this could get messy. I would rather not have to use a third party library. I feel like there is clean built in Pythonic way to do this but I just can't find it. I've dug through the docs but to no avail. If anyone has any suggestion as to how I could do this it would be very much appreciated. Thanks!

You could use recursivity:
def replace_key(the_dict, dict_keys, default_value):
if dict_keys[0] in the_dict:
if len(dict_keys)==1:
the_dict[dict_keys[0]]=default_value
else:
replace_key(the_dict[dict_keys[0]], dict_keys[1:],default_value)
else:
raise Exception("wrong key")
some_dict = {'top_property': {'first_nested': {'second_nested': 'the value'}}}
key_to_replace = 'top_property.first_nested.second_nested'
default_value = 'replaced'
#this would return as {'top_property': {'first_nested': {'second_nested': 'replaced'}}}
replace_key(some_dict, key_to_replace.split("."), default_value)
But it still uses the split(). But maybe you consider it to be less messy?

the easyest way that I've found to do this, namely get value using a "key path" by "dotted string" is using replace and eval:
for key in pfields:
if key.find('.') > 0:
key = key.replace(".", "']['")
try:
data = str(eval(f"row['{key}']"))
except KeyError:
data = ''
And this is an example of the keys:
lfields = ['cpeid','metadata.LinkAccount','metadata.DeviceType','metadata.SoftwareVersion','mode_props.vfo.CR07.VIKPresence','mode_props.vfo.CR13.VIBHardVersion']
With this raw solution You don't need install other library

Cleaner way to represent Rules (if-else) in Python

I am trying to find a design pattern (or maybe an algorithm) which will help me write these rules in a cleaner way. Any suggestions?
def get_rules(user, value):
if 500 <= value < 5000 and not user.address:
return [REQUEST_ADDRESS]
if value >= 5000:
if not user.address and not user.phone:
return [REQUEST_ADDRESS, REQUEST_PHONE]
if user.address and not user.phone:
return [REQUEST_PHONE]
if not user.address and user.phone:
return [REQUEST_ADDRESS]
# Potentially ~20 more conditions here based on various attributes of user
return [STATES.REQUEST_NONE]
Note: I am not looking for a rules engine since I don't want to complicate my code by adding "business friendly" DSL in python. Python itself is a simple language to write these rules.
Interesting read: http://martinfowler.com/bliki/RulesEngine.html (but I am still trying to stay away from a "framework" to do this for me).

You're checking lots of different combinations with your "if a and not b else check not a and b else check not a and not b" strategy to figure out what combination of requests you need to send.
Instead, only check what you're missing:
missing = []
if not user.phone:
missing.append(REQUEST_PHONE)
if not user.address:
missing.append(REQUEST_ADDRESS)
return missing or [REQUEST_NONE]

You can use a dict in this case:
resdict = {(False, False): [REQUEST_ADDRESS, REQUEST_PHONE],
(True, False): [REQUEST_PHONE],
(False, True): [REQUEST_ADDRESS]}
return resdict[(user.address, user.phone)]
You can also use a list comprehension:
return [req for req, haveit in zip([REQUEST_ADDRESS, REQUEST_PHONE], [user.address, user.phone]) if not haveit]
Or a simpler list append:
res = []
if not user.address:
res.append(REQUEST_ADDRESS)
if not user.phone:
res.append(REQUEST_PHONE)

If I understood the question right, you have a list of attributes for the user. If one is false a REQUEST value schould be added to the list. Then this could help:
# define all your combinations here:
mapping = {'address': REQUEST_ADDRESS, 'phone': REQUEST_PHONE, …)
return [value for key, value in mapping.items()
if not getattr(user, key, None)]

Looks like your "rules" boil down to this: Request values for fields that are not present as attributes in the object user. I will assume that the mapping of attributes to requests can be arbitrary; you can represent it as a dictionary mapping, e.g. like this:
rulemap = {
"address": REQUEST_ADDRESS,
"phone": REQUEST_PHONE,
# etc.
}
You can then get a list of the requests to issue by checking which of the keys in rulemap are not present as attributes in the object user:
return [ rulemap[fld] for fld in rulemap.keys() if fld not in user.__dict__ ]

Accessing dictionary of dictionary in python

Hi in my code there is a dictionary of dictionary.
nrec={'bridge': 'xapi1', 'current_operations': {}, 'uuid': '9ae5ca7d-e7d6-7a81-f619-d0ea33efb534', 'tags': [], 'other_config': {'is_guest_installer_network': 'true', 'netmask': '255.255.255.0', 'ip_end': '192.168.128.254', 'ip_begin': '192.168.128.1'}, 'name_label': 'Guest installer network', 'VIFs': ['OpaqueRef:dff106aa-1a94-8384-1c86-862b47c87fcf'], 'allowed_operations': [], 'PIFs': [], 'name_description': 'Network on which guests will get assigned a private local IP address', 'MTU': '1500', 'blobs': {}}
Here you can see inside this dictionary one more dictionary 'other_config': {'is_guest_installer_network': 'true', 'netmask': '255.255.255.0', 'ip_end': '192.168.128.254', 'ip_begin': '192.168.128.1'} is there.
I want to check is_guest_installer_network=="true"
I have done nrec["other_config"]["is_guest_installer_network"]== "true" but the problem is some attribute have this other_config property with either empty value or different value. Then in this case my solution will throw exception. So i want to do it in a efficient way like If is_guest_installer_network is consists in the dictionary and the value (string) is true or not.

If this is a config item, you shouldn't need to access it very often (thus your efficiency requirement would be questionable). Configure once and forget about it (e.g. set self.is_guest_installer_network = True).
If you can't forget about it, it would depend on the likelihood of the entry being present in your dictionary. If it's more likely that the item is missing it would probably be better if you do something like the following. You get some shortcut behavior if an item misses, the other config dict is looked up only once (for the existance check and for the value following lookup.
def check_guest_installer_network(nrec):
other_config = nrec.get("other_config", None)
return other_config is not None and other_config.get('is_guest_installer_network', False)
If it's more likely that the item is there, the lazy try/except approach could be better suited. As the saved check performance, would outweigh the additional performance cost when the exception actually needs to be handled.
def check_guest_installer_network(nrec):
try:
return nrec["other_config"]['is_guest_installer_network'] == "true"
except KeyError:
return False
After all, if this check indeed has a significant impact on the overall performance, you should put this variable somewhere it is better accessible, than in a nested dictionary, e.g. put it into a global/class member variable once, and enjoy the cheap and easy checks afterwards.
You should have a look at the cprofile module to verify that this lookup is indeed the bottleneck of your software, that is worth the optimization effort. And You should look at the timeit module to choose the most performant solution for your problem.

Try this:
nrec["other_config"].get('is_guest_installer_network')
It'll return its value if 'is_guest_installer_network' exists in nrec['other_config']

Just check
'other_config' in nrec and 'is_guest_installer_network' in nrec['other_config'] and nrec['other_config']['is_guest_installer_network'] == 'true'

To check if a key exists in a dictionary use:
if 'key' in dictionary:
# do sth
So your code will be:
if 'other_config' in nrec and 'is_guest_installer_network' in nrec['other_config'] and nrec['other_config']['is_guest_installer_network'] == 'true':
# do sth
Additionally if you want default value if the key is not present in the dict use get(key, default) method:
nrec.get('other_config', default_value_here)

One possible solution is
>>> try:
if nrec["other_config1"]["is_guest_installer_network"] == 'true':
print 'Do Something'
except (KeyError,TypeError):
None #Or do something logical

Your best bet to avoid exception is either try .. except, or use dictionary built-in methods.
my_dict = {'one': {'two': 'hello world!'}, 'four': 'Dummy!'}
try:
my_name = my_dict['one']['two']
except:
pass
// instead of pass, you can also return something or do something else
try:
my_dict['one']['two']
except Exception as e:
my_name = 'default'
return my_name, e // returns a tuple which contains 'default' and error message
#or use has_key()
# this is for one-level nested dictionary
for id, items in my_dict.iteritems():
if items.has_key('two'):
//do something
# or simply this --> eliminates dummy for loop
if my_dict['one'].has_key('two'): // has_key returns True / False
// do something
# or use in operator (replace has_key)
if 'two' in my_dict['one'].keys():
// do something
# or get method
my_dict['one'].get('two', 'Default')
Get is nice if that's all you need to avoid exception.

You've got the answer
nrec["other_config"]["is_guest_installer_network"]== "true"
can be written like
if nrec.has_key("other_config") and type(nrec["other_config"]) == type({}) and nrec["other_config"].has_key("....") and nrec["other_config"]["is_guest_installer_network"]== "true":
But this is sort of ugly.
Or, as noted in the comments
nrec.get("other_config",{}).get("is_guest_installer_network",{}) == "true"
But this doesn't handle the type checking.
Maybe best to do something like this
def traverse(_dict, keys):
val = _dict
for key in keys[:-1]:
if key in val and type(val[key]) is dict :
val = val[key]
else:
return None
return val.get(keys[-1],None)