Develop Reference

For loops and conditionals in Python

I am new to Python and I was wondering if there was a way I could shorten/optimise the below loops:
for breakdown in data_breakdown:
for data_source in data_source_ids:
for camera in camera_ids:
if (camera.get("id") == data_source.get("parent_id")) and (data_source.get("id") == breakdown.get('parent_id')):
for res in result:
if res.get("camera_id") == camera.get("id"):
res.get('data').update({breakdown.get('name'): breakdown.get('total')})
I tried this oneliner, but it doesn't seem to work:
res.get('data').update({breakdown.get('name'): breakdown.get('total')}) for camera in camera_ids if (camera.get("id") == data_source.get("parent_id")) and (data_source.get("id") == breakdown.get('parent_id'))

You can use itertools.product to handle the nested loops for you, and I think (although I'm not sure because I can't see your data) you can skip all the .get and .update and just use the [] operator:
from itertools import product
for b, d, c in product(data_breakdown, data_source_ids, camera_ids):
if c["id"] != d["parent_id"] or d["id"] != b["parent_id"]:
continue
for res in result:
if res["camera_id"] == c["id"]:
res['data'][b['name']] = b['total']

If anything, to optimize the performance of those loops, you should make them longer and more nested, with the data_source.get("id") == breakdown.get('parent_id') happening outside of the camera loop.
But there is perhaps an alternative, where you could change the structure of your data so that you don't need to loop nearly as much to find matching ID values. Convert each of your current lists (of dicts) into a single dict with its keys equal to the 'id' value you'll be trying to match in that loop, and the value being whole dict.
sources_dict = {source.get("id"): source for source in data_source_ids}
cameras_dict = {camera.get("id"): camera for camera in camera_ids}
results_dict = {res.get("camera_id"): res for res in result}
Now the whole loop only needs one level:
for breakdown in data_breakdown:
source = sources_dict[breakdown["parent_id"]]
camera = cameras_dict[source["parent_id"]]
res = results_dict[camera["id"]]
res.data[breakdown["name"]] = breakdown["total"]
This code assumes that all the lookups with get in your current code were going to succeed in getting a value. You weren't actually checking if any of the values you were getting from a get call was None, so there probably wasn't much benefit to it.
I'd further note that it's not clear if the camera loop in your original code was at all necessary. You might have been able to skip it and just directly compare data_source['parent_id'] against res['camera_id'] without comparing them both to a camera['id'] in between. In my updated version, that would translate to leaving out the creation of the cameras_dict and just directly indexing results_dict with source["parent_id"] rather than indexing to find camera first.

Python pandas if statement based off of boolean qualifier

I am try to do an IF statement where it keeps my currency pairs in alphabetic ordering (i.e. USD/EUR would flip to EUR/USD because E alphabetically comes before U, however CHF/JPY would stay the same because C comes alphabetically before J.) Initially I was going to write code specific to that, but realized there were other fields I'd need to flip (mainly changing a sign for positive to negative or vice versa.)
So what I did was write a function to create a new column and make a boolean identifier as to whether or not the field needs action (True) or not (False).
def flipFx(ccypair):
first = ccypair[:3]
last = ccypair[-3:]
if(first > last):
return True
else:
return False
brsPosFwd['Flip?'] = brsPosFwd['Currency Pair'].apply(flipFx)
This works great and does what I want it to.
Then I try and write an IF statement to use that field to create two new columns:
if brsPosFwd['Flip?'] is True:
brsPosFwd['CurrencyFlip'] = brsPosFwd['Sec Desc'].apply(lambda x:
x.str[-3:]+"/"+x.str[:3])
brsPosFwd['NotionalFlip'] = -brsPosFwd['Current Face']
else:
brsPosFwd['CurrencyFlip'] = brsPosFwd['Sec Desc']
brsPosFwd['NotionalFlip'] = brsPosFwd['Current Face']
However, this is not working properly. It's creating the two new fields, CurrencyFlip and NotionalFlip but treating every record like it is False and just pasting what came before it.
Does anyone have any ideas?

Pandas uses vectorised functions. You are performing operations on entire series objects as if they were single elements.
You can use numpy.where to vectorise your calculations:
import numpy as np
brsPosFwd['CurrencyFlip'] = np.where(brsPosFwd['Flip?'],
brsPosFwd['Sec Desc'].str[-3:]+'/'+brsPosFwd['Sec Desc'].str[:3]),
brsPosFwd['Sec Desc'])
brsPosFwd['NotionalFlip'] = np.where(brsPosFwd['Flip?'],
-brsPosFwd['Current Face'],
brsPosFwd['Current Face'])
Note also that pd.Series.apply should be used as a last resort; since it is a thinly veiled inefficient loop. Here you can simply use the .str accessor.

ArcMap Field Calculator Program to create Unique ID's

I'm using the Field Calculator in ArcMap and
I need to create a unique ID for every storm drain in my county.
An ID Should look something like this: 16-I-003
The first number is the municipal number which is in the column/field titled "Munic"
The letter is using the letter in the column/field titled "Point"
The last number is simply just 1 to however many drains there are in a municipality.
So far I have:
rec=0
def autoIncrement()
pStart=1
pInterval=1
if(rec==0):
rec=pStart
else:
rec=rec+pInterval
return "16-I-" '{0:03}'.format(rec)
So you can see that I have manually been typing in the municipal number, the letter, and the hyphens. But I would like to use the fields: Munic and Point so I don't have to manually type them in each time it changes.
I'm a beginner when it comes to python and ArcMap, so please dumb things down a little.

I'm not familiar with the ArcMap, so can't directly help you, but you might just change your function to a generator as such:
def StormDrainIDGenerator():
rec = 0
while (rec < 99):
rec += 1
yield "16-I-" '{0:03}'.format(rec)
If you are ok with that, then parameterize the generator to accept the Munic and Point values and use them in your formatting string. You probably should also parameterize the ending value as well.
Use of a generator will allow you to drop it into any later expression that accepts an iterable, so you could create a list of such simply by saying list(StormDrainIDGenerator()).
Is your question on how to get Munic and Point values into the string ID? using .format()?

I think you can use following code to do that.
def autoIncrement(a,b):
global rec
pStart=1
pInterval=1
if(rec==0):
rec=pStart
else:
rec=rec+pInterval
r = "{1}-{2}-{0:03}".format(a,b,rec)
return r
and call
autoIncrement( !Munic! , !Point! )
The r = "{1}-{2}-{0:03}".format(a,b,rec) just replaces the {}s with values of variables a,b which are actually the values of Munic and Point passed to the function.

shortening python "if" statements

I've been constantly coding in Python using this inefficient style
checkbox = self.request.get(u'checkbox') # get data from a web form
if checkbox == u'yes':
someclass.enabled = True
else:
someclass.enabled = False
how do I shorten this?

someclass.enabled = self.request.get(u'checkbox') == u'yes'

You can do this without an if statement:
someclass.enabled = (checkbox == u'yes')

You can just set the value to the outcome of the statement:
checkbox = self.request.get(u'checkbox') # get data from a web form
someclass.enabled = checkbox == u'yes'

As checkbox == u'yes' returns a boolean value you can simply assign this result to the variable directly.
someclass.enabled = (checkbox == u'yes')

Perhaps you could split it into a different function:
def getCheckboxValue(name):
return (self.request.get(name) == u'yes')

Python eval the statement and return the output to the statement. So you can use the assign variable in right side.
like
variable = eval_statment
so your example will be
someclass.enabled = self.request.get(u'checkbox') == u'yes'

It's a little unclear if you used booleans in your example because they were inherent to your problem or because they were a convenient example. If you want to assign to variables more complicated types than booleans, you may also want to check out Python's ternary operator (if you're using version 2.5 or greater):
someclass.int_val = 1 if checkbox == u'yes' else 2
which translates to
if checkbox == u'yes':
someclass.int_val = 1
else
someclass.int_val = 2
For boolean variables, I'd recommend using Yuushi's solution, but for completeness, this is what it would look like:
someclass.enabled = True if checkbox == u'yes' else False
It's about the same amount of typing, but saves some vertical space, which can be useful.

If you ever need more than a boolean value, you should consider using the dispatch pattern:
targets = {
'yes': do_yes,
'no': do_no,
'maybe': do_maybe,
}
targets[self.request.get(u'tricheckbox')]()
# where do_yes, do_no, and do_maybe are the functions to call for each state.

As pointed-out in another answer you could use dispatch table to do different things based on a value. However using a dictionary's get() method rather than \performing a direct lookup would allow you to also easily handle cases where nothing matches. Since the mapping won't be used again it can be temporary and anonymous.
This approach is very generic and can expanded as necessary, but usually requires extra functions to be written. Because of the latter, for very simple cases like your example, one of the other answers would probably require the least effort.
def do_yes(): print 'do_yes'
def do_no(): print 'do_no'
def do_maybe(): print 'do_maybe'
def no_match(): print 'no_match'
{
u'yes': do_yes,
u'no': do_no,
u'maybe': do_maybe,
}.get(self.request.get(u'checkbox'), no_match) ()

How to combine initialization and assignment of dictionary in Python?

I would like to figure out if any deal is selected twice or more.
The following example is stripped down for sake of readability. But in essence I thought the best solution would be using a dictionary, and whenever any deal-container (e.g. deal_pot_1) contains the same deal twice or more, I would capture it as an error.
The following code served me well, however by itself it throws an exception...
if deal_pot_1:
duplicates[deal_pot_1.pk] += 1
if deal_pot_2:
duplicates[deal_pot_2.pk] += 1
if deal_pot_3:
duplicates[deal_pot_3.pk] += 1
...if I didn't initialize this before hand like the following.
if deal_pot_1:
duplicates[deal_pot_1.pk] = 0
if deal_pot_2:
duplicates[deal_pot_2.pk] = 0
if deal_pot_3:
duplicates[deal_pot_3.pk] = 0
Is there anyway to simplify/combine this?

There are basically two options:
Use a collections.defaultdict(int). Upon access of an unknown key, it will initialise the correposnding value to 0.
For a dictionary d, you can do
d[x] = d.get(x, 0) + 1
to initialise and increment in a single statement.
Edit: A third option is collections.Counter, as pointed out by Mark Byers.

It looks like you want collections.Counter.

Look at collections.defaultdict. It looks like you want defaultdict(int).

So you only want to know if there are duplicated values? Then you could use a set:
duplicates = set()
for value in values:
if value in duplicates():
raise Exception('Duplicate!')
duplicates.add(value)
If you would like to find all duplicated:
maybe_duplicates = set()
confirmed_duplicates = set()
for value in values:
if value in maybe_duplicates():
confirmed_duplicates.add(value)
else:
maybe_duplicates.add(value)
if confirmed_duplicates:
raise Exception('Duplicates: ' + ', '.join(map(str, confirmed_duplicates)))

A set is probably the way to go here - collections.defaultdict is probably more than you need.
Don't forget to come up with a canonical order for your hands - like sort the cards from least to greatest, by suit and face value. Otherwise you might not detect some duplicates.