Develop Reference

Handle nested fields with conversion types in string with string.Formatter

Update 2
Alright, my answer to this question is not a complete solution to what I originally wanted but it's ok for simpler things like filename templating (what I originally intended to use this for). I have yet to come up with a solution for recursive templating. It might not matter to me though as I have reevaluated what I really need. Though it's possible I'll need bigger guns in the future, but then I'll probably just choose another more advanced templating engine instead of reinventing the tire.
Update
Ok I realize now string.Template probably is the better way to do this. I'll answer my own question when I have a working example.
I want to accomplish formatting strings by grouping keys and arbitrary text together in a nesting manner, like so
# conversions (!):
# u = upper case
# l = lower case
# c = capital case
# t = title case
fmt = RecursiveNamespaceFormatter(globals())
greeting = 'hello'
person = 'foreName surName'
world = 'WORLD'
sample = 'WELL {greeting!u} {super {person!t}, {tHiS iS tHe {world!t}!l}!c}!'
print(fmt.format(sample))
# output: WELL HELLO Super Forename Surname, this is the World!
I've subclassed string.Formatter to populate the nested fields which I retrieve with regex, and it works fine, except for the fields with a conversion type which doesn't get converted.
import re
from string import Formatter
class RecursiveNamespaceFormatter(Formatter):
def __init__(self, namespace={}):
Formatter.__init__(self)
self.namespace = namespace
def vformat(self, format_string, *args, **kwargs):
def func(i):
i = i.group().strip('{}')
return self.get_value(i,(),{})
format_string = re.sub('\{(?:[^}{]*)\}', func, format_string)
try:
return super().vformat(format_string, args, kwargs)
except ValueError:
return self.vformat(format_string)
def get_value(self, key, args, kwds):
if isinstance(key, str):
try:
# Check explicitly passed arguments first
return kwds[key]
except KeyError:
return self.namespace.get(key, key) # return key if not found (e.g. key == "this is the World")
else:
super().get_value(key, args, kwds)
def convert_field(self, value, conversion):
if conversion == "u":
return str(value).upper()
elif conversion == "l":
return str(value).lower()
elif conversion == "c":
return str(value).capitalize()
elif conversion == "t":
return str(value).title()
# Do the default conversion or raise error if no matching conversion found
return super().convert_field(value, conversion)
# output: WELL hello!u super foreName surName!t, tHiS iS tHe WORLD!t!l!c!
What am I missing? Is there a better way to do this?

Recursion is a complicated thing with this, especially with the limitations of python's re module. Before I tackled on with string.Template, I experimented with looping through the string and stacking all relevant indexes, to order each nested field in hierarchy. Maybe a combination of the two could work, I'm not sure.
Here's however a working, non-recursive example:
from string import Template, _sentinel_dict
class MyTemplate(Template):
delimiter = '$'
pattern = '\$(?:(?P<escaped>\$)|\{(?P<braced>[\w]+)(?:\.(?P<braced_func>\w+)\(\))*\}|(?P<named>(?:[\w]+))(?:\.(?P<named_func>\w+)\(\))*|(?P<invalid>))'
def substitute(self, mapping=_sentinel_dict, **kws):
if mapping is _sentinel_dict:
mapping = kws
elif kws:
mapping = _ChainMap(kws, mapping)
def convert(mo):
named = mapping.get(mo.group('named'), mapping.get(mo.group('braced')))
func = mo.group('named_func') or mo.group('braced_func') # i.e. $var.func() or ${var.func()}
if named is not None:
if func is not None:
# if named doesn't contain func, convert it to str and try again.
callable_named = getattr(named, func, getattr(str(named), func, None))
if callable_named:
return str(callable_named())
return str(named)
if mo.group('escaped') is not None:
return self.delimiter
if mo.group('invalid') is not None:
self._invalid(mo)
if named is not None:
raise ValueError('Unrecognized named group in pattern',
self.pattern)
return self.pattern.sub(convert, self.template)
sample1 = 'WELL $greeting.upper() super$person.title(), tHiS iS tHe $world.title().lower().capitalize()!'
S = MyTemplate(sample1)
print(S.substitute(**{'greeting': 'hello', 'person': 'foreName surName', 'world': 'world'}))
# output: WELL HELLO super Forename Surname, tHiS iS tHe World!
sample2 = 'testing${äää.capitalize()}.upper()ing $NOT_DECLARED.upper() $greeting '
sample2 += '$NOT_DECLARED_EITHER ASDF$world.upper().lower()ASDF'
S = MyTemplate(sample2)
print(S.substitute(**{
'some_var': 'some_value',
'äää': 'TEST',
'greeting': 'talofa',
'person': 'foreName surName',
'world': 'världen'
}))
# output: testingTest.upper()ing talofa ASDFvärldenASDF
sample3 = 'a=$a.upper() b=$b.bit_length() c=$c.bit_length() d=$d.upper()'
S = MyTemplate(sample3)
print(S.substitute(**{'a':1, 'b':'two', 'c': 3, 'd': 'four'}))
# output: a=1 b=two c=2 d=FOUR
As you can see, $var and ${var} works as expected, but the fields can also handle type methods. If the method is not found, it converts the value to str and checks again.
The methods can't take any arguments though. It also only catches the last method so chaining doesn't work either, which I believe is because re do not allow multiple groups to use the same name (the regex module does however).
With some tweaking of the regex pattern and some extra logic in convert both these things should be easily fixed.
MyTemplate.substitute works like MyTemplate.safe_substitute by not throwing exceptions on missing keys or fields.

How can I accept one variable or multple when using a function?

I'm looking to create a search function similar to my rent_book fuction that allows me to search by first name, second name or title or any combination of the three. So I could maybe search for first name "George" and title "Animal Farm" or just title "Animal Farm" and receive the same result.
Books are stored in a list of dictionaries this is the dict struct and the rent_book function. I could do a convoluted nest of ifs but I'm sure there's a better way.
book = {
"fname": fname,
"sname": sname,
"title": title,
"avail": True
}
def rent_book(self, fname, sname, title):
# if is_return is False:
for x in self.lstBooks:
if x['fname'] == fname and x['sname'] == sname and x['title'] == title and x['avail'] is True:
x['avail'] = False
return True
return False
Thanks

Since your arguments are the same as the dictionary keys you're matching, you could just use **kwargs and iterate over the kwargs:
def rent_book(self, **kwargs):
# if is_return:
# return False
if not kwargs:
raise KeyError("Must search on at least one of fname, sname, or title.")
for x in self.lstBooks:
if not (x['avail'] and all(x[k] == v for k, v in kwargs.items())):
continue
x['avail'] = False
return True
return False
Note that the function will implicitly raise KeyError if it's called with any invalid keys (the x[k] will raise it), and there's an explicit raise KeyError to guard against the caller accidentally not providing any kwargs at all, since otherwise it would just return the first book in lstBooks.
(Yes, pedants, they can still call it with avail=True.)

I am not sure If I understood Your question correctly,
But If I am correct, You want the function to work appropriately handling the following cases:
if all "title","fname","sname" are given.
if only one among the 3 is given.
If any pair of 2 among the 3 is given.
If that is the case You can use default Parameters in the function for all the 3.
def rent_book(self, fname="", sname="", title=""):
# if is_return is False:
for x in self.lstBooks:
if (len(fname) and x['fname'] == fname) and (len(sname) and x['sname'] == sname) and (len(title) and x['title'] == title) and x['avail'] is True:
x['avail'] = False
return True
return False
The line
(len(fname) and x['fname'] == fname)
makes sure that You only check the condition if fname is explicitly provided in as parameter.
So, In all the above scenarios mentioned above,
You can call the same function with only optional parameters
(Dont forget to specify the parameter name while calling else it will always consider it the value for the first parameter by default!!)

The first thing to do here is think about having a Book class. Initially it needs to support sname, fname, title and avail (a flag to indicate whether or not the book is available to rent).
We could then construct a Book instance with any combination of these attributes or even none.
So let's start with this:
class Book:
klist = ['sname', 'fname', 'title']
def __init__(self, **kwargs):
self.vars = kwargs
for k in Book.klist:
self.vars.setdefault(k, None)
self.vars.setdefault('avail', True)
#property
def avail(self):
return self['avail']
#avail.setter
def avail(self, v):
self.vars['avail'] = v
def __getitem__(self, item):
return self.vars.get(item, None)
def __repr__(self):
return f'sname={self["sname"]} fname={self["fname"]} title={self["title"]} avail={self.avail}'
Defining getitem keeps the code neater and safer. The repr is optional - it just helps to see what's going on should we ever want to print an instance of this class.
Let's create a couple of Book instances and put them in a list.
booklist = [Book(sname='Orwell', fname='George', title='Animal Farm'), Book(sname='Fleming', fname='Ian', title='Casino Royale')]
Now we want to rent a book. We can provide any one or all of the main attributes. If there's a match for all of the attributes that have been passed and if that book is available, we'll mark it as unavailable and return True. If nothing matches or the book isn't available we return False.
def rent_book(**kwargs):
for book in booklist:
if book.avail:
for k, v in kwargs.items():
if book[k] is not None and v != book[k]:
break
else:
book.avail = False
return True
return False
Now let's see what happens:
print(rent_book(title='Casino Royale'))
print(rent_book(title='Casino Royale'))
print(rent_book(sname='Orwell', title='Animal Farm'))
Output:
True # the book title matches and it's available
False # the book title matches but it's no longer available
True # both the surname and title match so it's available

Accessing Attributes in Python Class?

I have the following class:
class convert_to_obj(object):
def __init__(self, d):
for llist in d:
for a, b in llist.items():
if isinstance(b, (list, tuple)):
setattr(self, a, [obj(x) if isinstance(x, dict) else x for x in b])
else:
setattr(self, a, obj(b) if isinstance(b, dict) else b)
def is_authenticated(self):
username = self.username
password = self.password
if username and password:
return True
I am converting a dict to obj and then trying to access the is_authenticated method, when I do the following below:
new_array = [{'username': u'rr', 'password': u'something', }]
user = convert_to_obj(new_array)
user.is_authenticated()
it returns an error saying:
'convert_to_obj' object has no attribute 'is_authenticated'
I don't know why it's doing this. Hopefully, some other eyes might be able to point out what I am doing wrong. Thanks

#user2357112 is right (and good catch, because I wouldn't have seen it):
DO NOT USE TABS IN PYTHON ­— EVER!
That said, I have a few comments about your code.
First:
class convert_to_obj(object):
is a very bad name for a class. It'd be a good name for a function, though. You should better call it, for example:
class DictObject(object):
That being said, I'd advice you to use existing tools for doing such a thing. There's a powerful one called namedtuple, in the collections module. To do your thing, you could do:
from collections import namedtuple
# Create a class that declares the interface for a given behaviour
# which will expect a set of members to be accessible
class AuthenticationMixin():
def is_authenticated(self):
username = self.username
password = self.password
# useless use of if, here you can simply do:
# return username and password
if username and password:
return True
# but if you don't, don't forget to return False below
# to keep a proper boolean interface for the method
return False
def convert_to_object(d): # here that'd be a good name:
# there you create an object with all the needed members
DictObjectBase = namedtuple('DictObjectBase', d.keys())
# then you create a class where you mix the interface with the
# freshly created class that will contain the members
class DictObject(DictObjectBase, AuthenticationMixin):
pass
# finally you build an instance with the dict, and return it
return DictObject(**d)
which would give:
>>> new_array = [{'username': u'rr', 'password': u'something', }]
>>> # yes here I access the first element of the array, because you want
>>> # to keep the convert_to_object() function simple.
>>> o = convert_to_object(new_array[0])
>>> o
DictObject(password='something', username='rr')
>>> o.is_authenticated()
True
all that being more readable and easy to use.
N.B.: for a list of dicts to convert, just make:
>>> objdict_list = [convert_to_object(d) for d in new_array]
>>> objdict_list
[DictObject(password='something', username='rr')]
And if you're working with a list of pairs instead of a dict:
>>> tup_array = [('username', u'rr'), ('password', u'something')]
>>> {t[0]:t[1] for t in tup_array}
{'password': 'something', 'username': 'rr'}
So you don't need the extra leg work in the __init__().
HTH

You've mixed tabs and spaces, so the is_authenticated definition is erroneously nested inside the definition of __init__. Turn on "show whitespace" in your editor to see the problem, and run Python with the -tt flag to make it tell you when you do something like this. To fix the problem, convert the tabs to spaces; your editor most likely has a function to do this automatically.

How do I search for attributes in a list of objects?

I am creating a phone register in python where the user gets the choice to add a new person to the register. I need to get the code to find if that user is already in the register. My objects have the attributes surname, firstname, phonenumber and address.
My code looks like this:
def personInRegister(surname, firstname, phonenumber, address):
matchInRegister = False
x = 0
while x in range(len(people)):
if (people[x].surname.lower == surname.lower and
people[x].firstname.lower == firstname.lower and
people[x].phonenumber == phonenumber and
people[x].address.lower == address.lower):
matchInRegister = True
break
else:
x = x+1
return matchInRegister
where people is my list of people.
I and't get it to work, and I don't know what I'm doing wrong. Please help!

str.lower is a function. str.lower() calls the function and returns a lower case version of the string.
Python has a useful function for checking whether any value in a sequence is True. So we can build a sequence of boolean values like so:
def personInRegister(surname, firstname, phonenumber, address):
return any(person.surname.lower() == surname.lower() and
person.firstname.lower() == firstname.lower() and
person.phonenumber == phonenumber and
person.address.lower() == address.lower()
for person in people)
To avoid calling lower() for the query every time through the loop you could create the query as a tuple and compare that instead:
def personInRegister(surname, firstname, phonenumber, address):
query = (surname.lower(),
firstname.lower(),
phonenumber,
address.lower())
return any((person.surname.lower(),
person.firstname.lower(),
person.phonenumber,
person.address.lower()) == query
for person in people)
If you want to return the person you can use next:
def personInRegister(surname, firstname, phonenumber, address):
query = (surname.lower(),
firstname.lower(),
phonenumber,
address.lower())
return next((person for person in people
if query == (person.surname.lower(),
person.firstname.lower(),
person.phonenumber,
person.address.lower())),
None)
This will return None if it doesn't find the person. You have to use a generator expression if you don't use the default return. So this tells you the existence of a person, and gives you their record.

The Pythonic way of validating a long chain of conditions in Python

So I have a long chain of conditions that should be validated to be true. Instead of chaining a long if condition, I tried to be "innovative" and did it this way, which I reckon is more readable. But my question is, is this the optimal way of doing it?
Or is there a pythonic way of doing it? PS: Please respond with an alternative instead of answering "No", thanks!
Here's the code chunk:
def site_exists(site):
"""
returns the sitebean if it exists,
else returns false
"""
vpadmin_service = _get_vpadmin_service(site)
all_sites = VpAdminServiceUtil.getSites(vpadmin_service)
for site_listing in all_sites:
if site.getId():
#condition check
try:
assert site.getId() == site_listing.getId()
assert site.getName() == site_listing.getName()
assert site.getCustomer().getId() == site_listing.getCustomer().getId()
except AssertionError:
continue
#pass conditions
return site_listing
#no id, so just check for name and customer
else:
#condition check
try:
assert site.getName() == site_listing.getName()
assert site.getCustomer().getId() == site_listing.getCustomer().getId()
except AssertionError:
continue
#pass conditions
site.setId(site_listing.getId())
return site_listing
return False

A simpler approach is to build a tuple of the conditions and compare the tuples:
def site_info(s):
return s.getId(), s.getName(), s.getCustomer().getId()
if site_info(site) == site_info(site_listing):
return site_listing
else:
continue
If you have a lot of conditions, or the conditions are expensive, you can instead create a generator for the conditions, and compare with any or all:
import itertools
def iter_site_info(s):
yield s.getId()
yield s.getName()
yield s.getCustomer().getId()
if all(x==y for (x, y) in itertools.izip(iter_site_info(site), iter_site_info(site_listing)):
return site_listing
else:
continue
I'm not sure whether Jython has any and all, but they're trivial functions to write.
EDIT - any and all appeared in Python 2.5, so Jython should have them.

Using exception for control flow is bad! And it will also kill your performance. Assuming the condition will be true in one out of 10 cases? So 9 exceptions to handle in the worst case - this comes with a huge performance cost.
If you want readability, try:
if (
condition1 and \
condition2 and \
condition3
):
# do something
For the revised version, this should be equivalent:
for site_listing in all_sites:
if site.getName() == site_listing.getName() and site.getCustomer().getId() == site_listing.getCustomer().getId():
if not site.getId():
site.setId(site_listing.getId())
if site.getId() == site_listing.getId():
return site_listing

I would implement an is_same method on site and call it in the for loops.
all_sites = VpAdminServiceUtil.getSites(vpadmin_service)
for site_listing in all_sites:
if site_listing.is_same(site, set_id=True):
return site_listing
As for its implementation (that is your real question), I suggest something like:
...
if self.getName() != site.getName(): return False
if self.getCustomer() != site.getCustomer(): return False
...
return True

Use the __eq__ method! If you wrote the class of site yourself, no problem. If it is from a module outside of your control, see this solution, or create a somewhat less elegant wrapping class. For the wrap variant, this would be the class:
class WrappedSite(object):
def __init__(self, site):
self.site = site
def __eq__(self, other):
if site.getId():
if self.site.getId() != other.site.getId():
return False
if self.site.getName() != other.site.getName():
return False
if self.site.getCustomer().getId() != other.site.getCustomer().getId():
return False
return True
Then your big ugly function is reduced to this:
def site_exists(site):
"""
returns the sitebean if it exists,
else returns false
"""
vpadmin_service = _get_vpadmin_service(site)
all_sites = VpAdminServiceUtil.getSites(vpadmin_service)
wsite = WrappedSite(site)
for site_listing in all_sites:
if wsite == WrappedSite(site_listing):
return site_listing
return False
EDIT Fixed site_exists to return sitebean instead of True.

Remove some code duplication:
def site_exists(site):
"""
returns the sitebean if it exists,
else returns None
"""
vpadmin_service = _get_vpadmin_service(site)
all_sites = VpAdminServiceUtil.getSites(vpadmin_service)
for site_listing in all_sites:
if (site.getName() == site_listing.getName() and
site.getCustomer().getId() == site_listing.getCustomer().getId()):
if site.getId(): # if id is set; it should be the same
if site.getId() != site_listing.getId(): continue
else: # no id; consider it the same site
site.setId(site_listing.getId()) #XXX side-effect
return site_listing
Note: it is unexpected that site_exists() might modify its argument (via .setId()). Consider to refactor it:
def same_site(site, other):
if site.getId() and site.getId() != other.getId():
# if id is set; it should be the same
return False
return (site.getName() == other.getName() and
site.getCustomer().getId() == other.getCustomer().getId())
def get_site_listing(site):
"""
returns the sitebean corresponding to `site`,
returns None if there is none
"""
vpadmin_service = _get_vpadmin_service(site)
all_sites = VpAdminServiceUtil.getSites(vpadmin_service)
return next((s for s in all_sites if same_site(site, s)), None)
Note: the code doesn't modify the site object. Use the return value from get_site_listing() instead.
If next() is unavailable then use:
for site_listing in all_sites:
if same_site(site, site_listing):
return site_listing
return None
btw, jython should provide property wrappers for you; so you could write:
def same_site(site, other):
if site.id and site.id != other.id:
# if id is set; it should be the same
return False
return site.name == other.name and site.customer.id == other.customer.id
and site.id = id instead of site.setId(id).

Using an exception handler as a code target is basically a "goto". I don't see anything wrong with it really (despite the ancient furor surrounding "goto considered harmful"), but knowing that it's a goto might give you a different perspective.

Using exception handling for control flow is pythonic in the sense that "it's easier to ask for forgiveness than for permission". However, that doesn't extend to making rules to break just so that you can ask for forgiveness.
Others have provided excellent alternatives. Just wanted to address the principle aspect.