Develop Reference

I want a more efficient answer with these methods only

I have been self teaching myself python for the past 2 weeks. Today, I came across a problem and I have a very annoying solution to it(I feel bad for whoever has to read it). So firstly, I will introduce the problem and my solution to it.
Problem:Complete the getHost() function, which takes a single string argument representing a URL and returns a string that cor-
responds to the next-to-last section of the hostname. For example, given the URL "http://www.example.com/", the function
would return the string "example". Given the URL "ftp://this.is.a.long.name.net/path/to/some/file.php", the function would
return the string "name". While the path and filename sections of the URL are optional, you may assume that the full
hostname is always followed by a single forward slash ("/").
My solution:
def getHost(x):
newstring = ""
listofx = []
for i in range(len(x)):
listofx.append(x[i])
for j in range(2):
a = listofx.index("/")
listofx.reverse()
for k in range(a+1):
listofx.pop()
listofx.reverse()
b = listofx.index("/")
for g in range(len(listofx)-b):
listofx.pop()
for t in range(listofx.count(".")-1):
for o in range(listofx.index(".")+1):
listofx.reverse()
listofx.pop()
listofx.reverse()
for f in range(len(listofx)-listofx.index(".")):
listofx.pop()
for h in range(len(listofx)):
newstring = newstring + listofx[h]
print (newstring)
I HATE my solution because look at how many for loops I used. I felt like I had no choice since strings are immutable. I would appreciate someone can showing me a solution using while loops and the find()/rfind() methods. I do not want to keep converting strings to lists to solve these type of problems.

Using find and rfind:
def getHost(x):
index1 = x.find('//')
index2 = x.find('/', index1+2)
index3 = x.rfind('.',index1+2, index2)
return(x[:index3].split('.')[-1])

Yeah, there is better (pythonic) way
def extract(data):
print(data.split('/')[2].split('.')[-2])
extract("http://www.example.com/")
extract("ftp://this.is.a.long.name.net/path/to/some/file.php")
Output (obviously)
example
name

Assuming that your URL always has the double forward slash you could use something like the following;
url = "http://www.example.com/"
url = url.split("/")
url = url[2].split(".")
getHost = url[-2]
print(getHost)

Actually, a simpler version that doesn't need rfind:
def getHost(x):
index1 = x.find('//')
index2 = x.find('/', index1+2)
return(x[:index2].split('.')[-2])
print(getHost("ftp://this.is.a.long.name.net/path/to/some/file.php"))
print(getHost("http://www.example.com/"))

A better way to rewrite multiple appended replace methods using an input array of strings in python?

I have a really ugly command where I use many appended "replace()" methods to replace/substitute/scrub many different strings from an original string. For example:
newString = originalString.replace(' ', '').replace("\n", '').replace('()', '').replace('(Deployed)', '').replace('(BeingAssembled)', '').replace('ilo_', '').replace('ip_', '').replace('_ilop', '').replace('_ip', '').replace('backupnetwork', '').replace('_ilo', '').replace('prod-', '').replace('ilo-','').replace('(EndofLife)', '').replace('lctcvp0033-dup,', '').replace('newx-', '').replace('-ilo', '').replace('-prod', '').replace('na,', '')
As you can see, it's a very ugly statement and makes it very difficult to know what strings are in the long command. It also makes it hard to reuse.
What I'd like to do is define an input array of of many replacement pairs, where a replacement pair looks like [<ORIGINAL_SUBSTRING>, <NEW_SUBSTRING>]; where the greater array looks something like:
replacementArray = [
[<ORIGINAL_SUBSTRING>, <NEW_SUBSTRING>],
[<ORIGINAL_SUBSTRING>, <NEW_SUBSTRING>],
[<ORIGINAL_SUBSTRING>, <NEW_SUBSTRING>],
[<ORIGINAL_SUBSTRING>, <NEW_SUBSTRING>]
]
AND, I'd like to pass that replacementArray, along with the original string that needs to be scrubbed to a function that has a structure something like:
def replaceAllSubStrings(originalString, replacementArray):
newString = ''
for each pair in replacementArray:
perform the substitution
return newString
MY QUESTION IS: What is the right way to write the function's code block to apply each pair in the replacementArray? Should I be using the "replace()" method? The "sub()" method? I'm confused as to how to restructure the original code into a nice clean function.
Thanks, in advance, for any help you can offer.

You have the right idea. Use sequence unpacking to iterate each pair of values:
def replaceAllSubStrings(originalString, replacementArray):
for in_rep, out_rep in replacementArray:
originalString = originalString.replace(in_rep, out_rep)
return originalString

How about using re?
import re
def make_xlat(*args, **kwds):
adict = dict(*args, **kwds)
rx = re.compile('|'.join(map(re.escape, adict)))
def one_xlat(match):
return adict[match.group(0)]
def xlat(text):
return rx.sub(one_xlat, text)
return xlat
replaces = {
"a": "b",
"well": "hello"
}
replacer = make_xlat(replaces)
replacer("a well?")
# b hello?
You can add as many items in replaces as you want.

Iterating a conversion of a string to a float in a scripting file when parsing an old file

I am using a new script (a) to extract information from an old script (b) to create a new file (c). I am looking for an equal sign in the old script (b) and want to modify the modification script (a) to make it automated.
The string is
lev1tolev2 'from=e119-b3331l1 mappars="simp:180" targ=enceladus.bi.def.3 km=0.6 lat=(-71.5,90) lon=(220,360)'
It is written in python 3.
The current output is fixed at
cam2map from=e119-b3331l1 to=rsmap-x map=enc.Ink.map pixres=mpp defaultrange=MAP res=300 minlat=-71.5 maxlat=90 minlon=220 maxlon=360
Currently, I have the code able to export a string of 0.6 for all of the iterations of lev1tolev2, but each one of these is going to be different.
cam2map = Call("cam2map")
cam2map.kwargs["from"] = old_lev1tolev2.kwargs["from"]
cam2map.kwargs["to"] = "rsmap-x"
cam2map.kwargs["map"] = "enc.Ink.map"
cam2map.kwargs["pixres"] = "mpp"
cam2map.kwargs["defaultrange"] = "MAP"
**cam2map.kwargs["res"] = float((old_lev1tolev2.kwargs["km"]))**
cam2map.kwargs["minlat"] = lat[0]
cam2map.kwargs["maxlat"] = lat[1]
cam2map.kwargs["minlon"] = lon[0]
cam2map.kwargs["maxlon"] = lon[1]
I have two questions, why is this not converting the string to a float? And, why is this not iterating over all of the lev1tolev2 commands as everything else in the code does?
The full code is available here.
https://codeshare.io/G6drmk

The problem occurred at a different location in the code.
def escape_kw_value(value):
if not isinstance(value, str):
return value
elif (value.startswith(('"', "'")) and value.endswith(('"', "'"))):
return value
# TODO escape the quote with \" or \'
#if value.startswith(('"', "'")) or value.endswith(('"', "'")):
# return value
if " " in value:
value = '"{}"'.format(value)
return value

it doesn't seem to clear to me, but from you syntax here :
**cam2map.kwargs["res"] = float((old_lev1tolev2.kwargs["km"]))**
I'd bet that cam2map.kwargs["res"] is a dict, and you thought that it would convert every values in the dict, using the ** syntax. The float built-in should then be called in a loop over the elements of the dict, or possible a list-comprehension as here :
cam2map.kwargs["res"] = dict()
for key, value in old_lev1tolev2.kwars["res"].items():
cam2map.kwargs["res"][key] = float(value)
Edit :
Ok so, it seems you took the string 'from=e119-b3331l1 mappars="simp:180" targ=enceladus.bi.def.3 km=0.6 lat=(-71.5,90) lon=(220,360)'
And then thought that calling youstring.kwargs would give you a dict, but it won't, you can probably parse it to a dict first, using some lib, or, you use mystring.split('=') and then work your way to a dict first, like that:
output = dict()
for one_bit in lev_1_lev2.split(' '):
key, value = one_bit.split('=')
output[key] = value

More pythonic way to replace keywords in a string?

I am attempting to wrap an API with the following function. The API has end points that look similar to this:
/users/{ids}
/users/{ids}/permissions
The idea is that I'll be able to pass a dictionary to my function that contains a list of ids and those will be formatted as the API expects:
users = {'ids': [1, 2, 3, 5]}
call_api('/users/{ids}/permissions', users)
Then in call_api, I currently do something like this
def call_api(url, data):
for k, value in data.items():
if "{" + key + "}" in url:
url = url.replace("{"+k+"}", ';'.join(str(x) for x in value))
data.pop(k, None)
This works, but I can't imagine that if statement is efficient.
How can I improve it and have it work in both Python 2.7 and Python 3.5?
I've also been told that changing the dictionary while iterating is bad, but in my tests I've never had an issue. I am poping the value, because I later check if there are unexpected parameters (ie. anything left in data). Is what I'm doing now the right way?

Instead of modifying a dictionary as you iterate over it, creating another object to hold the unused keys is probably the way to go. In Python 3.4+, at least, removing keys during iteration will raise a
RuntimeError: dictionary changed size during iteration.
def call_api(url, data):
unused_keys = set()
for k, value in data.items():
key_pattern = "{" + k + "}"
if key_pattern in url:
formatted_value = ';'.join(map(str, value))
url = url.replace(key_pattern, formatted_value)
else:
unused_keys.add(k)
Also, if you think that you're more likely to run into an unused key, reversing the conditions might be the way to go.

Here is the way to do it. First, the string is parsed for the keys. It then remembers all keys not used in the url and saves it in the side. Lastly, it formats the url with the given parameters of the dict. The function returns the unused variables and the formatted url. If you wish you can remove the unused variables from the dict by iterating over them and deleting from the dict.
Here's some documentation with examples regarding the format syntax.
import string
users = {'ids': [1, 2, 3, 5]}
def call_api(url, data):
data_set = set(data)
formatter = string.Formatter()
used_set = {f[1] for f in formatter.parse(url) if f[1] is not None}
unused_set = data_set - used_set
formatted = url.format(**{k: ";".join(str(x) for x in v)
for k, v in data.items()})
return unused_set, formatted
print(call_api('/users/{ids}/permissions', users))

You could use re.subn which returns the number of replacements made:
import re
def call_api(url, data):
for k, value in list(data.items()):
url, n = re.subn(r'\{%s\}' % k, ';'.join(str(x) for x in value), url)
if n:
del data[k]
Note that for compatibilty with both python2 and python3, it is also necessary to create a copy of the list of items when destructively iterating over the dict.
EDIT:
It seems the main bottleneck is checking that the key is in the url. The in operator is easily the most efficient way to do this, and is much faster than a regex for the simple pattern that is being used here. Recording the unused keys separately is also more efficient than destructive iteration, but it doesn't make as much difference (relatively speaking).
So: there's not much wrong with the original solution, but the one given by #wegry is the most efficient.

The formatting keys can be found with a RegEx and then compared to the keys in the dictionary. Your string is already setup to use str.format, so you apply a transformation to the values in data, and then apply that transformation.
import re
from toolz import valmap
def call_api(url, data):
unused = set(data) - set(re.findall('\{(\w+)\}', url))
url = url.format_map(valmap(lambda v: ';'.join(map(str, v)), data))
return url, unused
The usage looks like:
users = {'ids': [1, 2, 3, 5], 'unused_key': 'value'}
print(call_api('/users/{ids}/permissions', users))
# ('/users/1;2;3;5/permissions', {'unused_key'})
This isn't going to time that well, but it's concise. As noted in one of the comments, it seems unlikely that this method is be a bottleneck.

Cleaner way to represent Rules (if-else) in Python

I am trying to find a design pattern (or maybe an algorithm) which will help me write these rules in a cleaner way. Any suggestions?
def get_rules(user, value):
if 500 <= value < 5000 and not user.address:
return [REQUEST_ADDRESS]
if value >= 5000:
if not user.address and not user.phone:
return [REQUEST_ADDRESS, REQUEST_PHONE]
if user.address and not user.phone:
return [REQUEST_PHONE]
if not user.address and user.phone:
return [REQUEST_ADDRESS]
# Potentially ~20 more conditions here based on various attributes of user
return [STATES.REQUEST_NONE]
Note: I am not looking for a rules engine since I don't want to complicate my code by adding "business friendly" DSL in python. Python itself is a simple language to write these rules.
Interesting read: http://martinfowler.com/bliki/RulesEngine.html (but I am still trying to stay away from a "framework" to do this for me).

You're checking lots of different combinations with your "if a and not b else check not a and b else check not a and not b" strategy to figure out what combination of requests you need to send.
Instead, only check what you're missing:
missing = []
if not user.phone:
missing.append(REQUEST_PHONE)
if not user.address:
missing.append(REQUEST_ADDRESS)
return missing or [REQUEST_NONE]

You can use a dict in this case:
resdict = {(False, False): [REQUEST_ADDRESS, REQUEST_PHONE],
(True, False): [REQUEST_PHONE],
(False, True): [REQUEST_ADDRESS]}
return resdict[(user.address, user.phone)]
You can also use a list comprehension:
return [req for req, haveit in zip([REQUEST_ADDRESS, REQUEST_PHONE], [user.address, user.phone]) if not haveit]
Or a simpler list append:
res = []
if not user.address:
res.append(REQUEST_ADDRESS)
if not user.phone:
res.append(REQUEST_PHONE)

If I understood the question right, you have a list of attributes for the user. If one is false a REQUEST value schould be added to the list. Then this could help:
# define all your combinations here:
mapping = {'address': REQUEST_ADDRESS, 'phone': REQUEST_PHONE, …)
return [value for key, value in mapping.items()
if not getattr(user, key, None)]

Looks like your "rules" boil down to this: Request values for fields that are not present as attributes in the object user. I will assume that the mapping of attributes to requests can be arbitrary; you can represent it as a dictionary mapping, e.g. like this:
rulemap = {
"address": REQUEST_ADDRESS,
"phone": REQUEST_PHONE,
# etc.
}
You can then get a list of the requests to issue by checking which of the keys in rulemap are not present as attributes in the object user:
return [ rulemap[fld] for fld in rulemap.keys() if fld not in user.__dict__ ]