Develop Reference

How can I print variables from the following for loop?

import numpy as np
def initialize_parameters(n, L):
for i in range(0, L):
globals()['W%s' % i] = np.random.randn(n[i], n[i-1]) * 0.01
n = [2, 3, 1]
L = 3
initialize_parameters(n, L)
print(W0, '\n', W1, '\n', W2)
Instead of writing W0, W1, etc. in the last print function, I want a print function to print all Ws

Since you asked how to turn this into a list I will go through it step by step.
In python a list is a set of objects surrounded with square brackets (e.g. n in your code is a list). Knowing this we can simply change your function like this to create a list and then add new values to it with the append function:
def initialize_parameters(n, L):
a_list = []
for i in range(0, L):
a_list.append(np.random.randn(n[i], n[i-1]) * 0.01)
However as we are no longer using global variables there are a couple of other things we have to do to make this list accessible.
Firstly we need to tell the function to return the list:
def initialize_parameters(n, L):
a_list = []
for i in range(0, L):
a_list.append(np.random.randn(n[i], n[i-1]) * 0.01)
return a_list
Secondly when we call this function we have to assign it to a new variable:
list_of_Ws = initialize_parameters(n, L)
Now all you have to do is print(list_of_Ws) and you will see all of the values or print(list_of_Ws[0]) for the first value, etc. for the rest of the values.

You could theoretically iterate over globals() and output the value:
for g in list(globals()):
if g.startswith("W"):
print(globals()[g])
But as #MisterMiyagi wrote in the comment, I would store the values in a list, not in globals().

Remove similar items in a list in Python

How do you remove similar items in a list in Python but only for a given item. Example,
l = list('need')
If 'e' is the given item then
l = list('nd')
The set() function will not do the trick since it will remove all duplicates.
count() and remove() is not efficient.

use filter
assuming you write function that decide on the items that you want to keep in the list.
for your example
def pred(x):
return x!="e"
l=list("need")
l=list(filter(pred,l))

Assuming given = 'e' and l= list('need').
for i in range(l.count(given)):
l.remove(given)

If you just want to replace 'e' from the list of words in a list, you can use regex re.sub(). If you also want a count of how many occurrences of e were removed from each word, then you can use re.subn(). The first one will provide you strings in a list. The second will provide you a tuple (string, n) where n is the number of occurrences.
import re
lst = list(('need','feed','seed','deed','made','weed','said'))
j = [re.sub('e','',i) for i in lst]
k = [re.subn('e','',i) for i in lst]
The output for j and k are :
j = ['nd', 'fd', 'sd', 'dd', 'mad', 'wd', 'said']
k = [('nd', 2), ('fd', 2), ('sd', 2), ('dd', 2), ('mad', 1), ('wd', 2), ('said', 0)]
If you want to count the total changes made, just iterate thru k and sum it. There are other simpler ways too. You can simply use regEx
re.subn('e','',''.join(lst))[1]
This will give you total number of items replaced in the list.

List comprehension Method. Not sure if the size/complexity is less than that of count and remove.
def scrub(l, given):
return [i for i in l if i not in given]
Filter method, again i'm not sure
def filter_by(l, given):
return list(filter(lambda x: x not in given, l))
Bruteforce with recursion but there are a lot of potential downfalls. Still an option. Again I don't know the size/comp
def bruteforce(l, given):
try:
l.remove(given[0])
return bruteforce(l, given)
except ValueError:
return bruteforce(l, given[1:])
except IndexError:
return l
return l
For those of you curious as to the actual time associated with the above methods, i've taken the liberty to test them below!
Below is the method I've chosen to use.
def timer(func, name):
print("-------{}-------".format(name))
try:
start = datetime.datetime.now()
x = func()
end = datetime.datetime.now()
print((end-start).microseconds)
except Exception, e:
print("Failed: {}".format(e))
print("\r")
The dataset we are testing against. Where l is our original list and q is the items we want to remove, and r is our expected result.
l = list("need"*50000)
q = list("ne")
r = list("d"*50000)
For posterity I've added the count / remove method the OP was against. (For good reason!)
def count_remove(l, given):
for i in given:
for x in range(l.count(i)):
l.remove(i)
return l
All that's left to do is test!
timer(lambda: scrub(l, q), "List Comp")
assert(scrub(l,q) == r)
timer(lambda: filter_by(l, q), "Filter")
assert(filter_by(l,q) == r)
timer(lambda : count_remove(l, q), "Count/Remove")
assert(count_remove(l,q) == r)
timer(lambda: bruteforce(l, q), "Bruteforce")
assert(bruteforce(l,q) == r)
And our results
-------List Comp-------
10000
-------Filter-------
28000
-------Count/Remove-------
199000
-------Bruteforce-------
Failed: maximum recursion depth exceeded
Process finished with exit code 0
The Recursion method failed with a larger dataset, but we expected this. I tested on smaller datasets, and Recursion is marginally slower. I thought it would be faster.

Call a function with 3 parameters to run through all combinations

I have a function multirun that I want to run. It has 3 parameters. I want to feed it a list of numbers for each parameter, and i want it to run through all combinations.
ie: for function multirun(a,b,c) i have
a = [1,2]
b=[3,4]
c=[5,6]
and I want it to run all (1,3,5) , (1,3,6) (1,4,5) ,(1,4,6) (2,3,5) etc...
Below I have my actual code:
CO2 = [0.00007, 0.00008, 0.00009]
H2O = [0.00003, 0.000035, 0.00004]
FO2 = [-2,-1,0,1,2]
for i in CO2:
for j in H2O:
for k in FO2:
multirun( WTCO2_START = [i], WTH2O_START = [j], FO2_buffer_START= [k])
This doesn't seem to do it. What do i have to change?

itertools.product should do exactly what you want. I think this will work for you:
import itertools
for i, j, k in itertools.product(CO2, H20, FO2):
multirun(WTCO2_START = [i], WTH2O_START = [j], FO2_buffer_START= [k])
You may have to play around with the repeat parameter, but that should be easier to determine.

try it like this:
for i in CO2:
for j in H2O:
for k in FO2:
multirun(i, j, k)

multiple self-contained filters applied during a single iteration

Lets say I have a data structure that's very expensive to iterate through and I need to collect it's elements into lists depending on certain criteria.
#fake data. pretend it's hard to access
import random
slowStruct = range(30)
random.shuffle(slowStruct)
def check1(x):
return x < 3
def check2(x):
return x > 15
def check3(x):
return x < 25 and x > 20
The easiest way would be to use list comprehensions. But this requires 3 iterations through the structure:
res1 = [node for node in slowStruct if check1(node)]
res2 = [node for node in slowStruct if check2(node)]
res3 = [node for node in slowStruct if check3(node)]
A faster way would be to use a loop and append to the result lists:
res1 = []
res2 = []
res3 = []
for node in slowStruct:
if check1(node):
res1.append(node)
if check2(node):
res2.append(node)
if check3(node):
res3.append(node)
Is there a functional-programming construct or idiom that can perform multiple filters, while only using a single iteration?
I can imagine it would look something like:
res1, res2, res3 = multiFilter(preds=[check1, check2, check3], iterable=slowStruct)

There is not a clean way of doing so with comprehensions. If you want a loop, use a loop. A list comprehension should only make one list.
If you're willing to use a loop, you are allowed to encapsulate it:
def multi_filter(predicates, iterable):
bins = [[] for _ in predicates]
for item in iterable:
for predicate, bin in zip(predicates, bins):
if predicate(item):
bin.append(item)
return bins

Is there a functional-programming construct or idiom that can perform
multiple filters, while only using a single iteration?
Yes, you can do this purely functional (although you will hit the max-recursion depth as python doesn't permit last call elimination).
The following code looks horrible, is purely functional, iterates only once over the iterable (but iterates over all conditions for each iteration, which is unavoidable):
#! /usr/bin/python3
slowStruct = [1,2,3,4,5,6,7,8,9,10]
conditions = [lambda x: x < 5,
lambda x: x % 2,
lambda x: x % 3]
multiFilter = lambda preds, iterable: (
lambda f, preds, iterable: f (
f,
iterable,
preds,
[ [] for _ in preds]
)
) (
lambda f, l, cs, accs: f (
f,
l [1:],
cs,
[acc + (l [:1] if c (l [0] ) else [] )
for acc, c in zip (accs, cs) ]
) if l else accs,
preds,
iterable
)
print (multiFilter (conditions, slowStruct) )
Nota bene: Python wasn't made for functional programming (see LCO). Also PEP8 doesn't help for formatting functional code.
To all commenters: I do not advocate this coding style, I am just trying to give a functional implementation (as requested by OP) that uses only one iteration.
Edit: Might not be purely functional if you consider the destructive assignment inside the list comprehension.

Better looping, for string manipulation (python)

If i have this code
s = 'abcdefghi'
for grp in (s[:3],s[3:6],s[6:]):
print "'%s'"%(grp)
total = calc_total(grp)
if (grp==s[:3]):
# more code than this
p = total + random_value
x1 = my_function(p)
if (grp==s[3:6]):
# more code than this
p = total + x1
x2 = my_function(p)
if (grp==s[6:]):
# more code than this
p = total + x2
x3 = my_function(p)
If the group is the first group, perform code for this group, if the group is the second group, perform code using the a value generated from code performed for the first group, the same applies for the third group, using a generated value from code for the second group:
How can i tidy this up to use better looping?
Thanks

I may have misunderstood what you're doing, but it appears that you want to do something to s[:3] on the first iteration, something different to s[3:6] on the second, and something else again to s[6:] on the third. In other words, that isn't a loop at all! Just write those three blocks of code out one after another, with s[:3] and so on in place of grp.

I must say I agree with Peter in that the loop is redundant. If you are afraid of duplicating code, then just move the repeating code into a function and call it multiple times:
s = 'abcdefghi'
def foo(grp):
# Anything more you would like to happen over and over again
print "'%s'"%(grp)
return calc_total(grp)
def bar(grp, value):
total = foo(grp)
# more code than this
return my_function(total + value)
x1 = bar(s[:3], random_value)
x2 = bar(s[3:6], x1)
x3 = bar(s[6:], x2)
If
# more code than this
contains non-duplicate code, then you must of course move that out of "bar" (which together with "foo" should be given a more descriptive name).

I'd code something like this as follows:
for i, grp in enumerate((s[:3],s[3:6],s[6:])):
print "'%s'"%(grp)
total = calc_total(grp)
# more code that needs to happen every time
if i == 0:
# code that needs to happen only the first time
elif i == 1:
# code that needs to happen only the second time
etc. The == checks can be misleading if one of the groups "just happens" to be the same as another one, while the enumerate approach runs no such risk.

x = reduce(lambda x, grp: my_function(calc_total(list(grp)) + x),
map(None, *[iter(s)] * 3), random_value)
At the end, you'll have the last x.
Or, if you want to keep the intermediary results around,
x = []
for grp in map(None, *[iter(s)] * 3):
x.append(my_function(calc_total(list(grp)) + (x or [random_value])[-1]))
Then you have x[0], x[1], x[2].

Get your data into the list you want, then try the following:
output = 0
seed = get_random_number()
for group in input_list:
total = get_total(group)
p = total + seed
seed = my_function(p)
input_list will need to look like ['abc', 'def', 'ghi']. But if you want to extend it to ['abc','def','ghi','jkl','mno','pqr'], this should still work.