Am trying to write an if condition in which I test on the standard deviation of several lists that are in a dictionary, The if statement that I did didn't work for me as you can see in the code :
a= {}
a[0] = [0.9907568097114563, 0.9913344979286194, 0.9907568097114563, 0.9913344979286194,
0.9907568097114563]
a[1] = [0.8538417220115662, 0.8526862859725952, 0.8411322832107544, 0.8659734129905701,
0.851530909538269]
a[n] = ...
if np.std(a[i] for i in range(len(a)) < 0.01 :
print("Early stopping here !")
I am not sure what you're trying to do, but you are calling an np.std on a generator "a[i] for i in range(len(a))", you have a syntax error (missing ')') and are using a dictionary like a list (which although might work, I don't recommend it). Consider using "any" for the if statement, you're code would look something like this:
import numpy as np
a= {}
a[0] = [0.9907568097114563, 0.9913344979286194, 0.9907568097114563, 0.9913344979286194,
0.9907568097114563]
a[1] = [0.8538417220115662, 0.8526862859725952, 0.8411322832107544, 0.8659734129905701,
0.851530909538269]
# a[n] = ...
if all([np.std(a[i]) < 0.01 for i in a.keys()]):
print("Early stopping here !")
if tests for… well, a condition—truthfulness of one expression.
You can get what you want in a few different ways—you can create a condition that checks multiple things (for which any() may or may not be useful), or maybe a loop with the single checks inside would be more helpful.
Related
I am new to Python and I was wondering if there was a way I could shorten/optimise the below loops:
for breakdown in data_breakdown:
for data_source in data_source_ids:
for camera in camera_ids:
if (camera.get("id") == data_source.get("parent_id")) and (data_source.get("id") == breakdown.get('parent_id')):
for res in result:
if res.get("camera_id") == camera.get("id"):
res.get('data').update({breakdown.get('name'): breakdown.get('total')})
I tried this oneliner, but it doesn't seem to work:
res.get('data').update({breakdown.get('name'): breakdown.get('total')}) for camera in camera_ids if (camera.get("id") == data_source.get("parent_id")) and (data_source.get("id") == breakdown.get('parent_id'))
You can use itertools.product to handle the nested loops for you, and I think (although I'm not sure because I can't see your data) you can skip all the .get and .update and just use the [] operator:
from itertools import product
for b, d, c in product(data_breakdown, data_source_ids, camera_ids):
if c["id"] != d["parent_id"] or d["id"] != b["parent_id"]:
continue
for res in result:
if res["camera_id"] == c["id"]:
res['data'][b['name']] = b['total']
If anything, to optimize the performance of those loops, you should make them longer and more nested, with the data_source.get("id") == breakdown.get('parent_id') happening outside of the camera loop.
But there is perhaps an alternative, where you could change the structure of your data so that you don't need to loop nearly as much to find matching ID values. Convert each of your current lists (of dicts) into a single dict with its keys equal to the 'id' value you'll be trying to match in that loop, and the value being whole dict.
sources_dict = {source.get("id"): source for source in data_source_ids}
cameras_dict = {camera.get("id"): camera for camera in camera_ids}
results_dict = {res.get("camera_id"): res for res in result}
Now the whole loop only needs one level:
for breakdown in data_breakdown:
source = sources_dict[breakdown["parent_id"]]
camera = cameras_dict[source["parent_id"]]
res = results_dict[camera["id"]]
res.data[breakdown["name"]] = breakdown["total"]
This code assumes that all the lookups with get in your current code were going to succeed in getting a value. You weren't actually checking if any of the values you were getting from a get call was None, so there probably wasn't much benefit to it.
I'd further note that it's not clear if the camera loop in your original code was at all necessary. You might have been able to skip it and just directly compare data_source['parent_id'] against res['camera_id'] without comparing them both to a camera['id'] in between. In my updated version, that would translate to leaving out the creation of the cameras_dict and just directly indexing results_dict with source["parent_id"] rather than indexing to find camera first.
i am still learning python and i am struggling to find a way to get this to work the way i want.
#First Method
staticPaths = [
"[N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C02-N2][vlan-480]",
"[N4-OLDCLOUD-IPSTORAGE/epg-N4-NFS-NETAPP-8040-C01]/[vlan-481]]",
"['N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C02-N2][vlan-484]",
"['N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C02-N2][vlan-485]",
"['N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C01-N2][vlan-480]"
]
for path in staticPaths:
filter_object = filter(lambda a: 'vlan-480' in a, staticPaths)
print(list(filter_object))
So what i am trying to do here is filter out anything that matches ‘vlan-480’ and return the entire line, so for example, if i run that code, i receive the correct output. which would be -
[N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C01-N2][vlan-480]
['N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C02-N2][vlan-480]
However where is states ‘vlan-480’ in the lambda function i actually want to pass it a LIST but because i am using the “in” statement, it only allows me to pass a single string.
Again i want to check multiples, so for example, give me the output for ‘vlan-480’ AND ‘vlan-484’ and it should return the lines for me from the staticPaths
I cannot think of way of getting this done, might just be me been stupid but for some reason i cannot solve it.
Also tried an if statement but i have the same problem, with the single string option.
#Second Method
path_matches = []
for path_match in staticPaths:
if 'vlan-480' in path_match:
path_matches.append(path_match)
print(path_matches)
Can anyone think of a way of doing this, its probably really easy but for some reason i cannot think of it. I did try and use List Comprehension but struggled to get the output i needed.
much appericated
Try this:
substrings = ['vlan-480', 'vlan-484']
filter_object = filter(lambda a: any(x in a for x in substrings), staticPaths)
print(list(filter_object))
The list substrings contains substrings to search for
The output I get for your dataset is:
['[N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C02-N2][vlan-480]', "['N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C02-N2][vlan-484]", "['N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C01-N2][vlan-480]"]
You can also use list comprehension
substrings = ['vlan-480', 'vlan-484']
path_matches = [x for x in staticPaths if substrings[0] in x or substrings[1] in x]
that results in :
path_matches
['[N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C02-N2][vlan-480]',
"['N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C02-N2][vlan-484]",
"['N4-NFS-NETAPP-8040-C01]/[N4-NHT-LEAF-VPC-FAS-C01-N2][vlan-480]"]
Your problem is with indentation i guess check this
path_matches = []
for path_match in staticPaths:
if 'vlan-480' in path_match:
path_matches.append(path_match)
Try to use this:
filter_object = filter(lambda a: 'vlan-480' in a or 'vlan-484' in a, staticPaths)
print(list(filter_object))
To do multi selection you can use operator OR/AND, so it will be such this 'vlan-480' in a or 'vlan-484' in a
So I have been liking to have my code more clean and I have been stuck into something that might be very easy to do.
Basically what I have done is currently:
for raw_product in r.json().get('data'):
if raw_product.get('countdown') is False:
print(raw_product.get('url'))
and I have been trying to figure out how to make it to one liner. So far I have only come to
test = ['{}'.format(raw_product.get('url')) for raw_product in r.json().get('data')]
however inside the one liner, there is missing the if statement and I wonder if it possible to apply the if statement inside the ['{}'.format(raw_product.get('url')) for raw_product in r.json().get('data')] ?
Try this generator :
gen = (repr(x.get('url')) for x in r.json().get('data') if not x.get('countdown'))
Or list :
li = [repr(x.get('url')) for x in r.json().get('data') if not x.get('countdown')]
What makes thise code not clean, is not the comprehension, but this
'get' methods. Compare that to :
li = [x.url for x in r.json_data if not x.countdown]. Is kind of clearer.
I'm not sure it's cleaner, but you can use a generator expression with an if filter to get this kind of pattern in one line:
[ print(rp.get('url')) for rp in r.json().get('data') if rp.get('countdown') is False ]
Note that this has the inefficiency of creating and remembering a whole list of None values (the result of every print call).
Set-up
I'm scraping apartment ads using Scrapy. For certain housing characteristics, I loop over the elements of a list BC I obtain per ad. If the characteristic is in the list I assign a 'yes' and if not a 'no'. E.g.
for x in BC:
if 'Terraza' in x:
terrace = 'yes'
break
else:
terrace = 'no'
For each 'yes-no' characteristic I have a copy of the above loop.
Problem
Besides looping over the elements of the list, I'd like to loop over the characteristics themselves. I.e. I'd like to 'merge' all the loops per characteristic into one loop.
I've tried the following (my actual bcl does contain multiple elements):
found = False
bcl = ['Terraza']
for x in l: # l is a list of strings containing housing characteristics
for y in bcl:
if y in x:
y = 'yes'
found = True
break
else:
y = 'no'
if found:
break
terrace = Terrazza
but this loop does not create the variable Terrazza. I'm not sure I can solve this with globals.
How do I make this loop work?
Depending on the wanted result, you may take a different approach. In a case like this, I tend to use a more functional style of coding. I am not sure if this is what you intend, but I think you could do it this way:
list_characteristics = ['Terraza', "Swimming pool"] # sample ad text - list of strings
bcl = ["test1", "test2", "Terraza", "test3"] # sample checklist - list of strings
def check_characteristics(checklist, adlist):
list_of_found_items = []
for characteristic in list_characteristics:
print("characteristic:", characteristic)
for item in bcl:
print("item:", item)
if item in characteristic:
list_of_found_items.append(item)
return list_of_found_items
result_list = check_characteristics(bcl, list_characteristics)
print("This ad has the following characteristics:", result_list)
Using the code above, you have a function that takes two lists of strings and lists all the items found. In case you one want to know if there is at least one of those, you can use this other function as a faster, short-circuit, way:
list_characteristics = ['Terraza', "Swimming pool"] # ad text - list of strings
bcl = ["test1", "test2", "Terraza", "test3"] # checklist - list of strings
def has_any_characteristic(checklist, adlist):
for characteristic in list_characteristics:
for item in bcl:
if item in characteristic:
return True
return False
result = has_any_characteristic(bcl, list_characteristics)
print("This ad has at least one of the wanted characteristics?", result)
Seems like a lot of code, but you code it once and then you use it whenever you need it, in a clean and easy to read way, IMHO. The definition of these two functions can even be placed in a separate module that you import where needed. So, in the main code, you will only need to use one line to call the function. Each function allows to answer a simple question in a way that is easy to understand, as presented in the print()statements in both code samples above.
Your problem isn't in merging loops but probably in breaking from outer loop. You can break out of a top loop by raising custom exception and then trying to catch it. Take a look at this peace of code:
bcl = ['Terraza']
class FoundCharacteristicException(Exception):
pass
for x in li:
try:
for y in bcl:
if y in x:
raise FoundCharacteristicException
except FoundCharacteristicException:
break
I would like to figure out if any deal is selected twice or more.
The following example is stripped down for sake of readability. But in essence I thought the best solution would be using a dictionary, and whenever any deal-container (e.g. deal_pot_1) contains the same deal twice or more, I would capture it as an error.
The following code served me well, however by itself it throws an exception...
if deal_pot_1:
duplicates[deal_pot_1.pk] += 1
if deal_pot_2:
duplicates[deal_pot_2.pk] += 1
if deal_pot_3:
duplicates[deal_pot_3.pk] += 1
...if I didn't initialize this before hand like the following.
if deal_pot_1:
duplicates[deal_pot_1.pk] = 0
if deal_pot_2:
duplicates[deal_pot_2.pk] = 0
if deal_pot_3:
duplicates[deal_pot_3.pk] = 0
Is there anyway to simplify/combine this?
There are basically two options:
Use a collections.defaultdict(int). Upon access of an unknown key, it will initialise the correposnding value to 0.
For a dictionary d, you can do
d[x] = d.get(x, 0) + 1
to initialise and increment in a single statement.
Edit: A third option is collections.Counter, as pointed out by Mark Byers.
It looks like you want collections.Counter.
Look at collections.defaultdict. It looks like you want defaultdict(int).
So you only want to know if there are duplicated values? Then you could use a set:
duplicates = set()
for value in values:
if value in duplicates():
raise Exception('Duplicate!')
duplicates.add(value)
If you would like to find all duplicated:
maybe_duplicates = set()
confirmed_duplicates = set()
for value in values:
if value in maybe_duplicates():
confirmed_duplicates.add(value)
else:
maybe_duplicates.add(value)
if confirmed_duplicates:
raise Exception('Duplicates: ' + ', '.join(map(str, confirmed_duplicates)))
A set is probably the way to go here - collections.defaultdict is probably more than you need.
Don't forget to come up with a canonical order for your hands - like sort the cards from least to greatest, by suit and face value. Otherwise you might not detect some duplicates.