Develop Reference

Return main function in sub function

I create multiple sub functions to process data and want to return the main function, once any nested function meets some condition. Then, the multiprocessing can continue the processes.
Here's a simple example:
import multiprocessing
from multiprocessing import Pool
def program1(num):
num += 1
return num
def program2(num):
num += 1
return num
def program3(num):
num += 1
return num
def main(num):
num = program1(num)
if num %2 != 0:
return
else:
num = program2(num)
if num %3 != 0:
return
else:
num = program3(num)
print(num)
return num
pool = Pool(4)
nums = range(8)
pool.map(main, nums)
Because there're multiple programs, it will make the code looks quite long ...
Is it possible to wrap the if condition in each nested function?
Then I can just use
def main(num):
num = program1(num)
num = program2(num)
num = program3(num)
Note that please don't care or simplify the data process hah. This is just an example to show the processes.

Python: Failing to break out of while loop with "SyntaxError: 'break' outside loop"

I am developing a breadth-first-search algorithm for a factorization problem and am running into an interesting/confusing bug when attempting to break out of a while loop. If you run the code below, it will fail inside the "construct_path" method, stating :
File "main.py", line 96
break
SyntaxError: 'break' outside loop
but I am inside of a while loop! If anyone could give me some advice on this issue, I would really appreciate it. Thanks in advance.
from numpy import random
import itertools
import Queue
#Finding multiples, BFS problem
#Given input of list with unique integers 0 - 9 and n = range(0,1000000), calculate smallest multiple of n and unique combination of values in the list
#Example : Input : list = {0,1,2} , n = 3,
# output = 12
# Input : list = {0,1,2} , n = 50
# Output = 200
class Problem:
def __init__(self):
self.n = random.randint(0,10000000)
listSize = random.randint(1,9)
mainSet = set()
self.mainList = []
while True:
toAdd = random.randint(0,9)
if(toAdd not in self.mainList):
self.mainList.append(toAdd)
if(len(self.mainList) == listSize):
break
def get_start_state(self):
s = ''.join(map(str, self.mainList))
return int(s)
def is_goal(self, state):
return True
def get_sucessors(self):
print "Getting successors"
def breadth_first_search(problem):
# a FIFO open_set
open_set = Queue.Queue()
# an empty set to maintain visited nodes
closed_set = set()
# a dictionary to maintain meta information (used for path formation)
meta = dict() # key -> (parent state, action to reach child)
# initialize
start = problem.get_start_state()
meta[start] = (None, None)
open_set.put(start)
while not open_set.empty():
parent_state = open_set.get()
print "{} {}".format("parent_state is ", parent_state)
if problem.is_goal(parent_state):
return construct_path(parent_state, meta)
for (child_state, action) in problem.get_successors(parent_state):
if child_state in closed_set:
continue
if child_state not in open_set:
meta[child_state] = (parent_state, action)
open_set.put(child_state)
closed_set.add(parent_state)
#collect path to desired answer
def construct_path(state, meta):
action_list = list()
while True:
row = meta[state]
if (len(row) == 2):
state = row[0]
action = row[1]
action_list.append(action)
else:
break
return action_list.reverse()
x = Problem()
breadth_first_search(x)

Could be that you have a mix of tabs and spaces so that the break in line 96 looks like it is indented to be below action_list.append(action) but effectively it is below the while. That would explain the error at least.
It is just a guess. But it could be like this, using a visible tabwidth of 4 in the editor:
→ while True:
→ → row = meta[state]
if (len(row) == 2):
state = row[0]
action = row[1]
action_list.append(action)
else:
break
To the Python interpreter this looks like this (because it assumes a tabwidth of 8):
→ while True:
→ → row = meta[state]
if (len(row) == 2):
state = row[0]
action = row[1]
action_list.append(action)
else:
break
This is still valid but obviously means a different thing and would put your break outside of the while loop.

My quicksort hangs for list of size 2^13

I've written a quicksort function in python as follows.
def quickSort1(A,p,q):
if p < q:
pivotIndex = partition(A,p,q)
quickSort1(A,p,pivotIndex-1)
quickSort1(A,pivotIndex+1,q)
def partition(A,first, last):
pivot = A[first]
tooBig = first+1
tooSmall = last
while True:
while tooBig<=last and A[tooBig]<pivot:
tooBig+=1
while tooSmall>first and A[tooSmall]>pivot:
tooSmall-=1
if tooBig < tooSmall:
temp = A[tooBig]
A[tooBig] = A[tooSmall]
A[tooSmall] = temp
else:
break
A[first] = A[tooSmall]
A[tooSmall] = pivot
return tooSmall
I'm testing the algorithm's execution time with varying sizes of lists (from 2 to 2^16). For example,
n = 2**13
s = [random.randint(-65536,65536) for i in range(n)]
#Begin Regular Quick Sort test with unsorted list
setup = '''
gc.enable()
from __main__ import quickSort1
from __main__ import n
from __main__ import s
'''
average = sum(timeit.Timer('A=s[:]; quickSort1(A,0,len(A)-1)',setup = setup).repeat(1,10))/10
I've verified that the algorithm works correctly for smaller sizes, but once I reach 2^13, the program hangs. I've tried doing sys.setrecursionlimit(2**30), but this doesn't change anything. Is there something wrong with my algorithm?

Yes, there's an error in your logic. If partition receives a sub-list in which A[tooBig] == A[tooSmall], then both of your while loop conditions are False from the start, and the if swaps equal values. Nothing changes, and you're stuck in an infinite loop.
I've managed to reproduce this with 212 often enough: the problem arises when your **n gets large enough that matching endpoints is a likely occurrence.
In case it helps, here's the code I used to trace the problem -- it's all yours with a few print insertions and indent to help trace teh call depth.
indent = ""
def quickSort1(A,p,q):
global indent
print indent, "ENTER", p, q
indent += " "
if p < q:
pivotIndex = partition(A,p,q)
quickSort1(A,p,pivotIndex-1)
quickSort1(A,pivotIndex+1,q)
indent = indent[2:]
print indent, "LEAVE"
def partition(A,first, last):
print indent, " PART", first, last
pivot = A[first]
tooBig = first+1
tooSmall = last
while True:
if abs(tooSmall-tooBig) < 10:
print indent, tooSmall, tooBig, A[tooBig:tooSmall+1]
while tooBig<=last and A[tooBig]<pivot:
tooBig+=1
while tooSmall>first and A[tooSmall]>pivot:
tooSmall-=1
if tooBig < tooSmall:
temp = A[tooBig]
A[tooBig] = A[tooSmall]
A[tooSmall] = temp
else:
break
A[first] = A[tooSmall]
A[tooSmall] = pivot
print indent, " PART", tooSmall
return tooSmall
The most common loop is when the list gets down to two equal values. Here's a longer one:
2030 2022 [-32421, -32303, -32723, -32402, -32705, -32269, -32422, -32834, -32421]

Restart current iteration in Python

I want to restart the current iteration of the first for loop when testing12(bad_order, order) == True. I have tried to use continue but it skips over the iteration which is not what I want.
bad_order = []
order = []
for iteration in range(0, 10):
args = []
print("\n def test%d(self):" % (iteration))
for input in range(num_arguments):
args.append(pick_type())
order = args
if testing12(bad_order, order) == True:
continue
try:
result = target(*args)
code = test_to_string(target, args, result)
except Exception as error:
bad_order = args
code = test_to_string_exc(target, args, error)

You can add an inner while loop which will in effect repeat the outer loop iteration until it exits. If you can put the repeat condition in the while test, then you're done:
for iteration in range(0, 10):
while some_condition:
...
If not, you can use a while True loop, put an unconditional break at the bottom, and use a continue to repeat:
for iteration in range(0, 10):
while True:
...
if continue_condition:
continue
...
break

you can also change the:
for iteration in range(0,10):
to
iteration =0
while iteration < 10 :
and increment iteration only when no repeat is needed

You just need to add an infinite while loop that breaks at the end of the iteration.
Then you can restart the iteration as often as needed.
bad_order = []
order = []
for iteration in range(0, 10):
while True: #
args = []
print("\n def test%d(self):" % (iteration))
for input in range(num_arguments):
args.append(pick_type())
order = args
if testing12(bad_order, order) == True:
continue
try:
result = target(*args)
code = test_to_string(target, args, result)
except Exception as error:
bad_order = args
code = test_to_string_exc(target, args, error)
break #

Closures in Python - an example [duplicate]

This question already has answers here:
Read/Write Python Closures
(8 answers)
Closed 9 years ago.
I'm performing an action many times in a loop and want to know how far along I am. I'm trying to make a progress report function that should act something like this:
def make_progress_report(n):
i = 0
def progress_report():
i = i + 1
if i % n == 0:
print i
return progress_report
pr = make_progress_report(2)
pr()
pr() # 2
pr()
pr() # 4
This code does not work. Specifically, I get an UnboundLocalError for i. How should I modify it so that it works?

Here are 3 options:
use a list for your counter:
def make_progress_report(n):
i = [0]
def progress_report():
i[0] = i[0] + 1
if i[0] % n == 0:
print i[0]
return progress_report
use itertools.count to track your counter:
from itertools import count
def make_progress_report(n):
i = count(1)
def progress_report():
cur = i.next()
if cur % n == 0:
print cur
return progress_report
Use nonlocal for your counter (Python 3+ only!):
def make_progress_report(n):
i = 0
def progress_report():
nonlocal i
i = i + 1
if i % n == 0:
print i
return progress_report

You could consider using a generator:
def progress_report(n):
i = 0
while 1:
i = i+1
if i % n == 0:
print i
yield # continue from here next time
pr = progress_report(2)
next(pr)
next(pr)
next(pr)
next(pr)

So the progress_report isn't closed over the variable i. You can check it like so...
>>> def make_progress_report(n):
... i=0
... def progress_report():
... i += 1
... if i % n == 0:
... print i
... return progress_report
...
>>> pr = make_progress_report(2)
>>> pr.__closure__
(<cell at 0x1004a5be8: int object at 0x100311ae0>,)
>>> pr.__closure__[0].cell_contents
2
You'll notice there is only one item in the closure of pr. That is the value you passed in originally for n. The i value isn't part of the closure, so outside of the function definition, i is no longer in scope.
Here is a nice discussion on closures in Python: http://www.shutupandship.com/2012/01/python-closures-explained.html

Look again at how you're defining your closure. The n should be passed in when you define the closure... take the following example:
#!/usr/env python
def progressReportGenerator(n):
def returnProgress(x):
if x%n == 0:
print "progress: %i" % x
return returnProgress
complete = False
i = 0 # counter
n = 2 # we want a progress report every 2 steps
getProgress = progressReportGenerator(n)
while not complete:
i+=1 # increment step counter
# your task code goes here..
getProgress(i) # how are we going?
if i == 20: # stop at some arbtirary point...
complete = True
print "task completed"