I am trying to make arguments easier to manage in a script I am putting together, I figured I'd wrap a bunch of related items into a dictionary, and pass the dictionary out of the func. pulling out the objects as I need them.
One of these items is a regEx, and I'm struggling to figure out how to structure things properly so I can make it work.
In my initial code (no dictionary). I am 'hard' coding the regex into the parser:
def TopnTail(self,line):
topRegEx = re.compile(r'(<!--make_database header end-->)')
tailRegEx = re.compile(r'(<!--make_database footer start-->)')
searchdumpTopOfPage = topRegEx.search(line)
searchdumpBottomOfPage = tailRegEx.search(line)
if searchdumpTopOfPage:
self.__useLine=1
if searchdumpBottomOfPage:
self.__useLine=0
if self.__useLine == 1:
self.trimmedLines = self.trimmedLines + line + "\n"
return (self.trimmedLines)
in the 'dictionaried' version I want to set the variables in a setter:
def siteDetails():
baseDict = {'topRegex':'''re.compile(r'(<!--make_database header end-->)')''', 'tailRegex':'''re.compile(r'(<!--make_database footer start-->)')'''}
return baseDict
and get to the compiled regex:
def TopnTail(self,line):
topRegEx = baseDict['topRegex']
tailRegEx = baseDict['tailRegex']
searchdumpTopOfPage = topRegEx.search(line)
searchdumpBottomOfPage = tailRegEx.search(line)
if searchdumpTopOfPage:
self.__useLine=1
if searchdumpBottomOfPage:
self.__useLine=0
if self.__useLine == 1:
self.trimmedLines = self.trimmedLines + line + "\n"
return (self.trimmedLines)
but this throws an error:
line 35, in TopnTail
searchdumpTopOfPage = topRegEx.search(line)
AttributeError: 'str' object has no attribute 'search'
Which I am guessing means that its not actually made the regex object, but is still passing the string.
I appreciate that I am probably breaking about 3 cardinal rules here... but any suggestions about how to make it work would be fantastic... (also, first time playing with both classes and dictionaries... so please go easy if I've really messed up!)
How about this?
baseDict = {
'topRegex': r'(<!--make_database header end-->)'
}
And in your TopnTail method
topRegEx = re.compile(baseDict['topRegex'])
The problem with what you have, is that you're assigning a string to topRegEx that contains '''re.compile(r'(<!--make_database header end-->)')'''. Since str has no method search, you get an error.
It makes sense to still compile your regex, and use the returned object. Splitting the contents of the regex into a dict will help if you ever need to changes the regex pattern, or you want to define it dynamically.
Related
I am reading data from nested json with this code:
data = json.loads(json_file.json)
for nodesUni in data["data"]["queryUnits"]['nodes']:
try:
tm = (nodesUni['sql']['busData'][0]['engine']['engType'])
except:
tm = ''
try:
to = (nodesUni['sql']['carData'][0]['engineData']['producer']['engName'])
except:
to = ''
json_output_for_one_GU_owner = {
"EngineType": tm,
"EngineName": to,
}
I am having an issue with None type error (eg. this one doesn't exists at all nodesUni['sql']['busData'][0]['engine']['engType'] cause there are no data, so I am using try/except. But my code is more complex and having a try/except for every value is crazy. Is there any other option how to deal with this?
Error: "TypeError: 'NoneType' object is not subscriptable"
This is non-trivial as your requirement is to traverse the dictionaries without errors, and get an empty string value in the end, all that in a very simple expression like cascading the [] operators.
First method
My approach is to add a hook when loading the json file, so it creates default dictionaries in an infinite way
import collections,json
def superdefaultdict():
return collections.defaultdict(superdefaultdict)
def hook(s):
c = superdefaultdict()
c.update(s)
return(c)
data = json.loads('{"foo":"bar"}',object_hook=hook)
print(data["x"][0]["zzz"]) # doesn't exist
print(data["foo"]) # exists
prints:
defaultdict(<function superdefaultdict at 0x000001ECEFA47160>, {})
bar
when accessing some combination of keys that don't exist (at any level), superdefaultdict recursively creates a defaultdict of itself (this is a nice pattern, you can read more about it in Is there a standard class for an infinitely nested defaultdict?), allowing any number of non-existing key levels.
Now the only drawback is that it returns a defaultdict(<function superdefaultdict at 0x000001ECEFA47160>, {}) which is ugly. So
print(data["x"][0]["zzz"] or "")
prints empty string if the dictionary is empty. That should suffice for your purpose.
Use like that in your context:
def superdefaultdict():
return collections.defaultdict(superdefaultdict)
def hook(s):
c = superdefaultdict()
c.update(s)
return(c)
data = json.loads(json_file.json,object_hook=hook)
for nodesUni in data["data"]["queryUnits"]['nodes']:
tm = nodesUni['sql']['busData'][0]['engine']['engType'] or ""
to = nodesUni['sql']['carData'][0]['engineData']['producer']['engName'] or ""
Drawbacks:
It creates a lot of empty dictionaries in your data object. Shouldn't be a problem (except if you're very low in memory) as the object isn't dumped to a file afterwards (where the non-existent values would appear)
If a value already exists, trying to access it as a dictionary crashes the program
Also if some value is 0 or an empty list, the or operator will pick "". This can be workarounded with another wrapper that tests if the object is an empty superdefaultdict instead. Less elegant but doable.
Second method
Convert the access of your successive dictionaries as a string (for instance just double quote your expression like "['sql']['busData'][0]['engine']['engType']", parse it, and loop on the keys to get the data. If there's an exception, stop and return an empty string.
import json,re,operator
def get(key,data):
key_parts = [x.strip("'") if x.startswith("'") else int(x) for x in re.findall(r"\[([^\]]*)\]",key)]
try:
for k in key_parts:
data = data[k]
return data
except (KeyError,IndexError,TypeError):
return ""
testing with some simple data:
data = json.loads('{"foo":"bar","hello":{"a":12}}')
print(get("['sql']['busData'][0]['engine']['engType']",data))
print(get("['hello']['a']",data))
print(get("['hello']['a']['e']",data))
we get, empty string (some keys are missing), 12 (the path is valid), empty string (we tried to traverse a non-dict existing value).
The syntax could be simplified (ex: "sql"."busData".O."engine"."engType") but would still have to retain a way to differentiate keys (strings) from indices (integers)
The second approach is probably the most flexible one.
I've followed a tutorial to write a Flask REST API and have a special request about a Python code.
The offered code is following:
# data list is where my objects are stored
def put_one(name):
list_by_id = [list for list in data_list if list['name'] == name]
list_by_id[0]['name'] = [new_name]
print({'list_by_id' : list_by_id[0]})
It works, which is nice, and even though I understand what line 2 is doing, I would like to rewrite it in a way that it's clear how the function iterates over the different lists. I already have an approach but it returns Key Error: 0
def put(name):
list_by_id = []
list = []
for list in data_list:
if(list['name'] == name):
list_by_id = list
list_by_id[0]['name'] = request.json['name']
return jsonify({'list_by_id' : list_by_id[0]})
My goal with this is also to be able to put other elements, that don't necessarily have the type 'name'. If I get to rewrite the function in an other way I'll be more likely to adapt it to my needs.
I've looked for tools to convert one way of coding into the other and answers in forums before coming here and couldn't find it.
It may not be beatiful code, but it gets the job done:
def put(value):
for i in range(len(data_list)):
key_list = list(data_list[i].keys())
if data_list[i][key_list[0]] == value:
print(f"old value: {key_list[0], data_list[i][key_list[0]]}")
data_list[i][key_list[0]] = request.json[test_key]
print(f"new value: {key_list[0], data_list[i][key_list[0]]}")
break
Now it doesn't matter what the key value is, with this iteration the method will only change the value when it finds in the data_list. Before the code breaked at every iteration cause the keys were different and they played a role.
this is a dummy version of what a function returns. I would like to know how to extract
'the.email#addresi.want' and 'Nextstringiwant from:
{'blah': {'blah1': 'the.email#addresi.want', 'blah2': 'Nextstringiwant'}, 'blah3': {'-note-': 'blah4', 'blah5': 'blah6', 'blah7': 'blah#bleble.blah', 'blah8': 'blah9'}}
I honestly don't understand the purpose for {} brackets very well, or how to work with it. I cannot change the function that returns this. Please help me, i'm lost. My gut tells me that I should convert this into a normal list and just get the desired position within that list, but it returns this error.
My code:
brackets = function().split(sep=':')
brackets.to_list()
email=brackets[2]
string=brackets[3]
The error:
brackets = creds.split(sep=':')
AttributeError: 'dict' object has no attribute 'split'
Note:
This is exactly how the function returns the {} list, I only changed the values for simplicity sake.
I would really appreciate
As the error message indicates, split is an attribute/method for a string, not for a dictionary.
Your function returns a Python dictionary.
Given your function is called function, you can access the values like this:
result = function()
email_address = result["blah"]["blah1"] # this will be 'the.email#addresi.want'
next_string = result["blah"]["blah2"] # this will be 'Nextstringiwant'
You can get further information on Python dictionaries on this site:
https://realpython.com/python-dicts/
mydict = {
'blah': {'blah1': 'the.email#addresi.want',
'blah2': 'Nextstringiwant'},
'blah3': {'-note-': 'blah4',
'blah5': 'blah6',
'blah7':
'blah#bleble.blah',
'blah8': 'blah9'}
}
[k_ for k_ in mydict.get("blah", dict()).values()]
Output:
['the.email#addresi.want', 'Nextstringiwant']
{} mean json object in Python.
if that function return is string, you should use Json module of python to convert it to json object and access its properties. For example:
import json
obj = json.loads(str_above)
print (obj.blah.blah1)
print (obj.blah.blah2)
I am using a new script (a) to extract information from an old script (b) to create a new file (c). I am looking for an equal sign in the old script (b) and want to modify the modification script (a) to make it automated.
The string is
lev1tolev2 'from=e119-b3331l1 mappars="simp:180" targ=enceladus.bi.def.3 km=0.6 lat=(-71.5,90) lon=(220,360)'
It is written in python 3.
The current output is fixed at
cam2map from=e119-b3331l1 to=rsmap-x map=enc.Ink.map pixres=mpp defaultrange=MAP res=300 minlat=-71.5 maxlat=90 minlon=220 maxlon=360
Currently, I have the code able to export a string of 0.6 for all of the iterations of lev1tolev2, but each one of these is going to be different.
cam2map = Call("cam2map")
cam2map.kwargs["from"] = old_lev1tolev2.kwargs["from"]
cam2map.kwargs["to"] = "rsmap-x"
cam2map.kwargs["map"] = "enc.Ink.map"
cam2map.kwargs["pixres"] = "mpp"
cam2map.kwargs["defaultrange"] = "MAP"
**cam2map.kwargs["res"] = float((old_lev1tolev2.kwargs["km"]))**
cam2map.kwargs["minlat"] = lat[0]
cam2map.kwargs["maxlat"] = lat[1]
cam2map.kwargs["minlon"] = lon[0]
cam2map.kwargs["maxlon"] = lon[1]
I have two questions, why is this not converting the string to a float? And, why is this not iterating over all of the lev1tolev2 commands as everything else in the code does?
The full code is available here.
https://codeshare.io/G6drmk
The problem occurred at a different location in the code.
def escape_kw_value(value):
if not isinstance(value, str):
return value
elif (value.startswith(('"', "'")) and value.endswith(('"', "'"))):
return value
# TODO escape the quote with \" or \'
#if value.startswith(('"', "'")) or value.endswith(('"', "'")):
# return value
if " " in value:
value = '"{}"'.format(value)
return value
it doesn't seem to clear to me, but from you syntax here :
**cam2map.kwargs["res"] = float((old_lev1tolev2.kwargs["km"]))**
I'd bet that cam2map.kwargs["res"] is a dict, and you thought that it would convert every values in the dict, using the ** syntax. The float built-in should then be called in a loop over the elements of the dict, or possible a list-comprehension as here :
cam2map.kwargs["res"] = dict()
for key, value in old_lev1tolev2.kwars["res"].items():
cam2map.kwargs["res"][key] = float(value)
Edit :
Ok so, it seems you took the string 'from=e119-b3331l1 mappars="simp:180" targ=enceladus.bi.def.3 km=0.6 lat=(-71.5,90) lon=(220,360)'
And then thought that calling youstring.kwargs would give you a dict, but it won't, you can probably parse it to a dict first, using some lib, or, you use mystring.split('=') and then work your way to a dict first, like that:
output = dict()
for one_bit in lev_1_lev2.split(' '):
key, value = one_bit.split('=')
output[key] = value
http://www.learnpython.org/Serialization_using_JSON_and_pickle
Here are the instructions:
The aim of this exercise is to print out the JSON string with key-value pair "Me" : 800 added to it.
And below is the starting code, which we should modify.
#Exercise fix this function, so it adds the given name and salary pair to the json it returns
def add_employee(jsonSalaries, name, salary):
# Add your code here
return jsonSalaries
#Test code - shouldn't need to be modified
originalJsonSalaries = '{"Alfred" : 300, "Jane" : 301 }'
newJsonSalaries = add_employee(originalJsonSalaries, "Me", 800)
print(newJsonSalaries)
I'm completely lost. The JSON lesson was brief, at best. The issue I seem to be running in to here is that orginalJsonSalaries is defined as a string (containing all sort of unnecessary symbols like brackets. In fact, I think if the single quotes surrounding its definition were removed, originalJsonSalaries would be a dictionary and this would be a lot easier. But as it stands, how can I append "Me" and 800 to the string and still maintain the dictionary-like formatting?
And yes, I'm very very new to coding. The only other language I know is tcl.
EDIT:
OK, thanks to the answers, I figured out I was being dense and I wrote this code:
import json
#Exercise fix this function, so it adds the given name and salary pair to the json it returns
def add_employee(jsonSalaries, name, salary):
# Add your code here
jsonSalaries = json.loads(jsonSalaries)
jsonSalaries["Me"] = 800
return jsonSalaries
#Test code - shouldn't need to be modified
originalJsonSalaries = '{"Alfred" : 300, "Jane" : 301 }'
newJsonSalaries = add_employee(originalJsonSalaries, "Me", 800)
print(newJsonSalaries)
This does not work. For whatever reason, the original dictionary keys are formatted as unicode (I don't know where that happened), so when I print out the dictionary, the "u" flag is shown:
{u'Jane': 301, 'Me': 800, u'Alfred': 300}
I have tried using dict.pop() to replace the key ( dict("Jane") = dict.pop(u"Jane") ) but that just brings up SyntaxError: can't assign to function call
Is my original solution incorrect, or is this some annoying formatting issue and how to resolve it?
The page you linked to says exactly how to do this:
In order to use the json module, it must first be imported:
import json
[...]
To load JSON back to a data structure, use the "loads" method. This method takes a string and turns it back into the json object datastructure:
print json.loads(json_string)
They gave you a string (jsonSalaries). Use json.loads to turn it into a dictionary.
Your last question is a new question, but... When you print a dictionary like that you are just using the fact that python is nice enough to show you the contents of its variables in a meaningful way. To print the dictionary in your own format, you would want to iterate through the keys and print the key and value:
for k in newJsonSalaries:
print("Employee {0} makes {1}".format(k, newJsonSalaries[k]))
There are other problems in your code....
It is weird to load the JSON inside the add employee function. That should be separate...
Also, in your add_employee() function you are hardwired always to add the same values of Me and 800 instead of using the name and salary variables that are passed in, so that line should be:
jsonSalaries[name] = salary
Use this:
import json
def add_employee(jsonSalaries, name, salary):
# Add your code here
jsonSalaries = json.loads(jsonSalaries)
jsonSalaries[name] = salary
jsonSalaries = json.dumps(jsonSalaries)
return jsonSalaries
#Test code - shouldn't need to be modified
originalJsonSalaries = '{"Alfred" : 300, "Jane" : 301 }'
newJsonSalaries = add_employee(originalJsonSalaries, "Me", 800)
print(newJsonSalaries)
Add this before return jsonSalaries:
jsonSalaries = json.dumps(jsonSalaries)