im playing a little bit with scoop and i want to know if i can distribute simple problems like a backtracking in a finite state machine to get all states.
For example:
But i want to print all solutions.
solutions = []
def backtraking(state)
for new_state in state.get_new_states():
if new_state.is_terminal():
solutions.append(new_state)
else:
futures.submit(backtraking,new_state)
def main():
task = futures.submit(backtracking,state)
if __name__ == "__main__":
main()
Now in solutions i will have all the solutions for the backtracking computing, but in a distributed system.
This code is not working, does anyone have some experience with Python and Scoop to solve this?
From the Scoope group
The statement "futures.map(backtraking(new_state))" will call backtraking() with new_state as it's argument, and then call futures.map with the result of the previous call as argument.
I doubt that is what you want to do.
The simplest way to parallelize your program using SCOOP would be by replacing your recursive call to a futures.submit() to backtracking.
Something along the lines of:
def backtraking(state)
for new_state in state.get_new_states():
if new_state.is_terminal():
print "A solution"
valid_list.append(new_state)
else:
futures.submit(backtraking, new_state)
This will create a Future (basically a task that can be executed concurrently) for every node. Your tree traversal is then performed in parallel, provided you have multiple cores assigned to the program.
If you are seeking maximum performance, you can improve it by only performing a submit on the firsts depth levels, such as (untested!):
def backtraking(state, depth=0)
for new_state in state.get_new_states():
if new_state.is_terminal():
print "A solution"
valid_list.append(new_state)
else:
if depth < 3:
futures.submit(backtraking, new_state, depth + 1)
else:
backtraking(new_state, depth + 1)
Hope it clarified things up.
Related
Sorry for the vague title, but since I have no clue what the reason for my problem might be I don't know how to describe it better. Anyway, I have a strange problem in connection with multiprocessing and hope some of you can help me.
I'm currently dealing with convex optimization, especially with parallelizing the tasks. My goal is to utilize as many cores as possible on a multi-core-machine (on which I get access only temporarily).
Therefore, I took a look on the CVXPY's Tutorial page and tried to implement an easy example to get into the topic (scroll down, it's the last example on the page). I shortened the example to the parts which are necessary for my question, so my code looks as follows:
import cvxpy as cp
import numpy
from multiprocessing import Pool
# Assign a value to gamma and find the optimal x.
def get_x(gamma_value):
print("E")
gamma.value = gamma_value
result = prob.solve()
print("F")
return x.value
# Problem data.
n = 15
m = 10
numpy.random.seed(1)
A = numpy.random.randn(n, m)
b = numpy.random.randn(n)
# gamma must be nonnegative due to DCP rules.
gamma = cp.Parameter(nonneg=True)
# Construct the problem.
x = cp.Variable(m)
error = cp.sum_squares(A # x - b)
obj = cp.Minimize(error + gamma*cp.norm(x, 1))
prob = cp.Problem(obj)
# Construct a trade-off curve of ||Ax-b||^2 vs. ||x||_1
sq_penalty = []
l1_penalty = []
x_values = []
print("A")
gamma_vals = numpy.logspace(-4,6, num = 6400)
print("B")
for val in gamma_vals:
gamma.value = val
print("C")
# Parallel computation (set to 1 process here).
pool = Pool(processes = 1)
print("D")
x_values = pool.map(get_x, gamma_vals)
print(x_values[-1])
As you might have observed, I added some prints with capital letters, they served to find out where exactly the issue occurs, so I can refer to them in my problem description;
When I run the code the code processes and the letters "A" to "D" are displayed on the screen, so everything's fine until passing "D". But then the program kind of gets stuck. The CPU load is still high, so there definitely is going on something, but the code never reaches capital "E", which would be after successfully starting
x_values = pool.map(get_x, gamma_vals).
In my eyes it looks a bit like being stuck in an infinite loop.
Therefore, I guess someting with this pool.map function must be fishy. First, I thought that calling this function might be time-consuming and therefore it takes long to process (but that's rubbish, the optimization starts within the get_x function, so no reasoning).
Nevertheless, I tried to run my program on the multi-core-machine (with multiple cores, as well as with only a single core) and - surprise - it successfully passed this line nearly instantaneous and started with the actual optimization problem (and finally solved it).
So my issue is, that I don't know what's going wrong on my computer and how to fix it.
I can't access the machine at any time so - of course - I want to try the code on my computer first before uploading it, which isn't possible if even this easy toy-example doesn't work.
I' be grateful for any ideas/help!
Thank you in advance!
FOOTNOTE: I am using WIN10, the multi-core machine uses Linux
I am wondering if there is a standard way or a better way to control against infinite recursion than in my code below? I want my recursive function to give up after max attempts. The code below does it by introducing attempt method parameter and incrementing it during the recursive invocation. Is there a better way?
def Rec(attempt=0):
if attempt==10:
return()
else:
print(attempt)
Rec(attempt=attempt+1)
Rec()
There is also this way but is not recommended for what you want to do - I only posted it for reference and is good to use in other cases...
#!/usr/bin/env python
import sys
sys.setrecursionlimit(5)
def Rec(attempt=0):
print attempt
Rec(attempt=attempt+1)
try:
Rec()
except RuntimeError:
print 'maximum recursion depth exceeded'
From sys.setrecursionlimit(limit)
To be even more clear, as said in python docs what sys.setrecursionlimit(limit) does is:
Set the maximum depth of the Python interpreter stack to limit. This
limit prevents infinite recursion from causing an overflow of the C
stack and crashing Python.
The highest possible limit is platform-dependent. A user may need to
set the limit higher when she has a program that requires deep
recursion and a platform that supports a higher limit. This should be
done with care, because a too-high limit can lead to a crash.
So in my opinion is not good to mess with the Python interpreter stack unless you know very well what you are doing.
You could make a decorator, and then you can write your proper recursive function, with its usual exit conditions, but also impose a recursion limit:
def limit_recursion(limit):
def inner(func):
func.count = 0
def wrapper(*args, **kwargs):
func.count += 1
if func.count < limit:
result = func(*args, **kwargs)
else:
result = None
func.count -= 1
return result
return wrapper
return inner
Your code would be (with a limit of 3):
#limit_recursion(limit=3)
def Rec():
print('hi')
Rec()
Running:
>>> Rec()
hi
hi
hi
Yours is already good. That's the way to go. It is good because it is light weight. You basically need an int and a condition branch - that's it.
Alternatively you can try to guarantee to break the loop in the condition without a counter (but that usually is dependent from case to case).
I'm creating a script using this library: https://github.com/oczkers/fut14
However, after some time, I get the error "maximum recursion depth exceeded." Afterwards I did some research and saw that I could increase this, however I would like to know a better way of avoiding this error and making my script more efficient.
def search():
items = fut.searchAuctions('player', level='gold', max_buy=250, start=0, page_size=15)
print (items)
if items == 1:
print (buy_now_price)
print (trade_id)
fut.bid(items[0]['trade_id'], 'buy_now_price')
elif items == 0:
search()
Most python installation only allow you to recurse 1000 levels (Python's self-imposed artificial limit).
A new "recursion level" happens every time your function search calls itself and involves saving some information of each function call on top of the stack. Once this stack has "grown" tall enough, you receive a "maximum recursion depth exceeded." also known as a stack overflow :).
You can modify your recursive algorithm to an iterative one instead to avoid this.
See this question here for an algorithmic way to convert from a recursive function to an iterative one. It involves using a list to simulate a stack (a list can grow much larger than Python's stack).
Recursive to iterative example
Although using the algorithmic way to convert a recursive to iterative function works, it's not the best approach since you are still using a growing list to represent your stack. Sometimes you can analyze your algorithm and find a way rewrite your algorithm without simulating any stack.
In your example, it seems that you simply want a function that will run forever until it finds something. You can rewrite it iteratively as follows
def search():
while True:
items = fut.searchAuctions('player', level='gold', max_buy=250, start=0, page_size=15
print (items)
if items == 1:
break
print (buy_now_price)
print (trade_id)
fut.bid(items[0]['trade_id'], 'buy_now_price')
I saw the thread about how to keep count in a recursive function but I didn't quite follow the answer and it also didn't seem to apply to what I am looking for (at least from what I could tell so sorry if this is a repetitive question!). I am working on a piece of code to decipher text that has been Ceasar shifted in several different spots. I have constructed a recursive function that works well to shift the text until until it has found the correct shift and to return the shifted text but I am unable to get the function to return the number of times it iterated.
text_to_shift = apply_coder(text[start:], build_decoder(1))
Ltext = text_to_shift.split()
for w in Ltext:
if is_word(wordlist, w) == True:
text = ' '.join(Ltext)
return text
else:
text = ' '.join(Ltext)
return find_best_shifts_rec(wordlist, text, start)
I could write this as a while loop but I like the elegance of what I have written. I am leaning towards a global variable (which I'm going to try after this) but I feel there is a better solution. Thank you in advance for either an answer or a better explanation of the one in the thread i referred to.
def recursive_thing(calls=1):
# "calls" argument keeps track of recursion depth
if keep_recursing():
# pass a higher count to the recursive call
recursive_thing(calls + 1)
else:
print calls
return
Give your function an argument that keeps track of the recursion depth, or put the recursion in a helper function with such an argument.
In the minmax algorithm,How to determine when your function reaches the end of the tree and break the recursive calls.
I have made a max function in which I am calling the min function. In the min function , what shud I do?? For max function, I am just returning the bestscore.
def maxAgent(gameState, depth):
if (gameState.isWin()):
return gameState.getScore()
actions = gameState.getLegalActions(0);
bestScore = -99999
bestAction = Directions.STOP
for action in actions:
if (action != Directions.STOP):
score = minAgent(gameState.generateSuccessor(0, action), depth, 1)
if (score > bestScore):
bestScore = score
bestAction = action
return bestScore
def minvalue(gameState,depth,agentIndex):
if (gameState.isLose()):
return gameState.getScore()
else:
finalstage = False
number = gameState.getNumAgents()
if (agentIndex == number-1):
finalstage = True
bestScore = 9999
for action in gameState.getLegalActions(agentIndex):
if(action != Directions.STOP
I could not understand how to proceed now?I not allowed to set the limit of depth of tree. it has to be arbitrary.
Usually you want to search until a certain recursion depth (e.g. n moves in advance, when playing chess). Therefore, you should pass the current recursion depth as a parameter. You may abort earlier when your results do not improve, if you can determine that with little effort.
In the minmax algorithm,How to
determine when your function reaches
the end of the tree and break the
recursive calls.
Basically, you're asking when you've reached a leaf node.
A leaf node occurs when you've reached the maximum depth for the search, or a terminal node (i.e. a position that ends the game).
I completely agree with relet. But you might want to consider this also:
There are times when you might also think that the current branch that you are exploring is worth exploring a little deeper. This will call for some further heuristics to be applied. I don't know how advanced the solution for your problem is required to be. This is because I don't know where the problem is coming from.
As suggested by Amber, try posting some code so we know how complex you want the solution to be. Further, if you explained how much you know/can_do, then we might be able to suggest more useful options - things that you might be able to realistically implement, rather than just amazing ideas that you may not know how to or have the time to implement
This site has post of minimax, even though it is based on IronPython:
http://blogs.microsoft.co.il/blogs/dhelper/archive/2009/07/13/getting-started-with-ironpython-part-4-minimax-algorithm.aspx
It says:
The Minimax algorithm is recursive by nature, and as such it requires a stop condition, in our case either the game has ended (no more moves) or the desired depth has been reached (lines 3-8).
You can avoid having two different functions by using Negamax http://en.wikipedia.org/wiki/Negamax