I have written an instance method which uses recursion to find a certain solution. It works perfectly fine except the time when I'm exiting the if-elif block. I call the function itself inside IF block. Also, I have only one return statement. The output from the method is weird for me to understand. Here is the code and the output:
def create_schedule(self):
"""
Creates the day scedule for the crew based on the crew_dict passed.
"""
sched_output = ScheduleOutput()
assigned_assignements = []
for i in self.crew_list:
assigned_assignements.extend(i.list_of_patients)
rest_of_items = []
for item in self.job.list_of_patients:
if item not in assigned_assignements:
rest_of_items.append(item)
print("Rest of the items are:", len(rest_of_items))
if len(rest_of_items) != 0:
assignment = sorted(rest_of_items, key=lambda x: x.window_open)[0]
# print("\nNext assignment to be taken ", assignment)
output = self.next_task_eligibility(assignment, self.crew_list)
if len(output) != 0:
output_sorted = sorted(output, key=itemgetter(2))
crew_to_assign = output_sorted[0][1]
assignment.eta = output_sorted[0][4]
assignment.etd = int(assignment.eta) + int(assignment.care_duration)
crew = next((x for x in self.crew_list if x.crew_number == crew_to_assign), None)
self.crew_list.remove(crew)
crew.list_of_patients.append(assignment)
crew.time_spent = assignment.etd
self.crew_list.append(crew)
self.create_schedule()
else:
print("*" * 80, "\n", "*" * 80, "\nWe were not able to assign a task so stopped.\n", "*" * 80, "\n", "*" * 80)
sched_output.crew_output = self.crew_list
sched_output.patients_left = len(rest_of_items)
elif not rest_of_items:
print("Fully solved.")
sched_output.crew_output = self.crew_list
sched_output.patients_left = 0
print("After completely solving coming here.")
return sched_output
This was the output:
Rest of the items are: 10
Rest of the items are: 9
Rest of the items are: 8
Rest of the items are: 7
Rest of the items are: 6
Rest of the items are: 5
Rest of the items are: 4
Rest of the items are: 3
Rest of the items are: 2
Rest of the items are: 1
Rest of the items are: 0
Fully solved.
After completely solving coming here.
After completely solving coming here.
After completely solving coming here.
After completely solving coming here.
After completely solving coming here.
After completely solving coming here.
After completely solving coming here.
After completely solving coming here.
After completely solving coming here.
After completely solving coming here.
After completely solving coming here.
What I don't understand is that as soon as my list rest_of_items is empty, I assign data to sched_output and return it. However, print statement is being executed for the same number of time as recursion was done. How can I avoid this?
My output is perfectly fine. All I want to do is understand the cause of this behaviour and how to avoid it.
The reason it's printing out 11 times is that you always call print at the end of the function, and you're calling the function 11 times. (It's really the same reason you get Rest of the items are: … 11 times, which should be a lot more obvious.)
Often, the best solution is to redesign things so instead of doing "side effects" like print inside the function, you just return a value, and the caller can then do whatever side effects it wants with the result. In that case, it doesn't matter that you're calling print 11 times; the print will only happen once, in the caller.
If that isn't possible, you can change this so that you only print something when you're at the top of the stack. But in many recursive functions, there's no obvious way to figure that out without passing down more information:
def create_schedule(self, depth=0):
# etc.
self.create_schedule(depth+1)
# etc.
if not depth:
print('After completely solving come here.')
returns sched_output
The last resort is to just wrap the recursive function, like this:
def _create_schedule(self):
# etc.
self._create_schedule()
# etc.
# don't call print
return sched_output
def create_schedule(self):
result = self._create_schedule()
print('After completely solving come here.')
return result
That's usually only necessary when you need to do some one-time setup for the recursive process, but here you want to do some one-time post-processing instead, which is basically the same problem, so it can be solved the same way.
(Of course this is really just the first solution in disguise, but it's hidden inside the implementation of create_schedule, so you don't need to change the interface that the callers see.)
As you call your create_schedule function within itself before the function finishes, once it has gotten to the end and doesn't need to call itself again, each function ends, and hits the "After completely solving coming here.", at the end of the function.
This means that each function, after calling itself, is still running - just stuck at the line where it calls itself - until they have all completed, which is when the paused functions can finish their task, printing out your statement.
You have print("After completely solving coming here.") at the end of your recursive function. That line will be executed once for each recursion.
Consider this simple example, which recreates your issue:
def foo(x):
print("x = {x}".format(x=x))
if x > 1:
foo(x-1)
print("Done.")
Now call the function:
>>> foo(5)
x = 5
x = 4
x = 3
x = 2
x = 1
Done.
Done.
Done.
Done.
Done.
As you can see, on the final call to foo(x=0), it will print "Done.". At that point, the function will return to the previous call, which will also print "Done." and so on.
This program is going through a web-based development environment and is actually my attempt to get what I need from itertools that I can't actually import in this case. I've changed the function somewhat to meet the naming requirements, but it should be basically the same. My current problem is that reversed() is not implemented, so I have to work around that. I tried creating a list with a for loop that iterated down, both inside and outside of this function. I also tried the current version without the ()s around it. Now I get an error that I'm out of range of the index when I run this. What are my alternatives to reversed?
def combinations_of_options(options, opt_len):
''' yields all combinations of option in specific length
'''
pool = tuple(options)
pool_len = len(pool)
if opt_len > pool_len:
return
indices = range(opt_len)
yield tuple(pool[index] for index in indices)
while True:
for index in (range(opt_len,-1,-1)): #this is the line that uses reversed
if indices[index] != index + pool_len - opt_len:
break
else:
return
indices[index] += 1
for dummy_index in range(index+1, opt_len):
indices[dummy_index] = indices[dummy_index-1] + 1
yield tuple(pool[index] for index in indices)
I'm creating objects derived from a rather large txt file. My code is working properly but takes a long time to run. This is because the elements I'm looking for in the first place are not ordered and not (necessarily) unique. For example I am looking for a digit-code that might be used twice in the file but could be in the first and the last row. My idea was to check how often a certain code is used...
counter=collections.Counter([l[3] for l in self.body])
...and then loop through the counter. Advance: if a code is only used once you don't have to iterate over the whole file. However You are stuck with a lot of iterations which makes the process really slow.
So my question really is: how can I improve my code? Another idea of course is to oder the data first. But that could take quite long as well.
The crucial part is this method:
def get_pc(self):
counter=collections.Counter([l[3] for l in self.body])
# This returns something like this {'187':'2', '199':'1',...}
pcode = []
#loop through entries of counter
for k,v in counter.iteritems():
i = 0
#find post code in body
for l in self.body:
if i == v:
break
# find fist appearence of key
if l[3] == k:
#first encounter...
if i == 0:
#...so create object
self.pc = CodeCana(k,l[2])
pcode.append(self.pc)
i += 1
# make attributes
self.pc.attr((l[0],l[1]),l[4])
if v <= 1:
break
return pcode
I hope the code explains the problem sufficiently. If not, let me know and I will expand the provided information.
You are looping over body way too many times. Collapse this into one loop, and track the CodeCana items in a dictionary instead:
def get_pc(self):
pcs = dict()
pcode = []
for l in self.body:
pc = pcs.get(l[3])
if pc is None:
pc = pcs[l[3]] = CodeCana(l[3], l[2])
pcode.append(pc)
pc.attr((l[0],l[1]),l[4])
return pcode
Counting all items first then trying to limit looping over body by that many times while still looping over all the different types of items defeats the purpose somewhat...
You may want to consider giving the various indices in l names. You can use tuple unpacking:
for foo, bar, baz, egg, ham in self.body:
pc = pcs.get(egg)
if pc is None:
pc = pcs[egg] = CodeCana(egg, baz)
pcode.append(pc)
pc.attr((foo, bar), ham)
but building body out of a namedtuple-based class would help in code documentation and debugging even more.
The following code executes in a work thread, and just happily spins away, receiving pause/report commands, etc. I assume they're soft faults since I'm not having RAM use issues and my hard drive didn't melt over the weekend, and that it's got something to do with how long I've let results accumulate, as I'm only getting ~50 page faults when I started it up a few days ago.
The "counter" attribute is currently at 22,496,115, and "results" has 1,418,641 elements.
The slice of "results" is taken because I was feeling contrary and started the list at 1.
def run(self):
while self.keep_running:
self.lock.acquire()
is_prime = True
self.counter += 1
cutoff_val = pow(self.counter,.5)
for number in self.results[1:]:
if number > cutoff_val:
break
if self.counter % number == 0:
is_prime = False
break
if is_prime:
self.results.append(self.counter)
self.lock.release()
Note: I know that I could use the Sieve of Eratosthenes to optimize the algorithm and probably cut down on the page faults, but that's not the point: I'm trying to pinpoint the exact reason - or at least the worst offender - behind the page faults so I can avoid doing the same kinds of things in the future. This algorithm is used simply for testing UI responsiveness when I need a "stupidly expensive, simple work thread."
Additional setup pieces as requested:
def __init__(self):
self.counter = 0
self.keep_running = False;
self.lock = threading.Lock()
self.results = list()
def __call__(self, *args):
if not self.keep_running:
self.keep_running = True
self.run()
I think that #John Gaines Jr. has pointed out something you need to change. If your list is really big, you don't want to be making copies of it like that.
Here is a good way to loop over the same values as self.results[1:] but without making a copy:
res = iter(self.results) # get an iterator for the list values
next(res) # iterate once to throw away first value
for number in res:
# same code you already have goes here
...
EDIT: The above code is correct and simple, but doesn't scale well. I thought about it and figured there must be something in itertools for this, and sure enough there is:
import itertools as it
res = it.islice(self.results, 1, None)
for number in res:
# same code you already have goes here
...
EDIT: Thank you to #John Gaines Jr. for pointing out that you can use None instead of len(self.results) in the call to it.islice().
From the Python tutorial Lists section:
All slice operations return a new list containing the requested elements. This means that the following slice returns a shallow copy of the list a:
>>> a[:]
['spam', 'eggs', 100, 1234]
So in your for loop, the bit self.results[1:] results in a copy of the results list. If this routine is being called over and over, it could quite easily cause memory thrashing.
Maybe the problem can be solved by deleting all those functions, can't it?
However, i really don't know what to do to get the source work.
By the way, it just simulates a horse in a chesstable, going around and around, randomly, trying to visit each square once; and I get a recursion depth exceeded error.
import random
def main():
global tries,moves
tries,moves=0,0
restart()
def restart():
global a,indexes,x,y
a=[[0 for y in range(8)] for x in range(8)] #Costrutto chic
indexes=[x for x in range(8)]
#Random part
x=random.randint(0,7)
y=random.randint(0,7)
a[x][y]=1
start()
def start():
global i,indexes,moves,tries
i=0
random.shuffle(indexes) #List filled with random numbers that i'll use as indexes
while i<=7:
if indexes[i]==0:
move(-2,-1)
elif indexes[i]==1:
move(-2,1)
elif indexes[i]==2:
move(-1,-2)
elif indexes[i]==3:
move(-1,2)
elif indexes[i]==4:
move(1,-2)
elif indexes[i]==5:
move(1,2)
elif indexes[i]==6:
move(2,-1)
elif indexes[i]==7:
move(2,1)
i+=1
for _ in a:
if 0 in _:
print "Wasted moves: %d"%(moves)
tries+=1
moves=0
restart()
print "Success obtained in %d tries"%(tries)
def move(column,row):
global x,y,a,moves
try: b=a[x+row][y+column]
except IndexError: return 0
if b==0 and 0<=x+row<=7 and 0<=y+column<=7:
x=x+row
y=y+column
a[x][y]=1
moves+=1
start()
else: return 0
try :main()
#except: print "I couldn't handle it" <-Row added to prevent python from returning a huge amount of errors
EDIT: This is the modified version, which still does not works, but at least it's an improvement:
import random
def move((column,row),x,y):
try: cell=squares_visited[x+row][y+column]
except IndexError: return x,y ## NONE TYPE OBJECT
if cell==0 and 0<=x+row<=7 and 0<=y+column<=7:
x+=row
y+=column
squares_visited[x][y]=1
return x,y ## NONE TYPE OBJECT
squares_visited=[[0] * 8 for _ in range(8)]
x=random.randint(0,7)
y=random.randint(0,7)
squares_visited[x][y]=1
moves=[(-2,-1),(-2,1),(-1,-2),(-1,2),(1,-2),(1,2),(2,-1),(2,1)]
indexes=list(range(8))
tries=0
print "The horse starts in position %d,%d"%(x,y)
while True:
random.shuffle(indexes)
for _ in indexes:
cells=move(moves[indexes[_]],x,y) ##Passing as arguments x and y looks weird
x=cells[0]
y=cells[1]
#If you out this for cicle, there are no legal moves anymore(due to full completion with 1, or to lack of legit moves)
for _ in squares_visited:
if 0 in _:
squares_visited=[[0] * 8 for _ in range(8)]
tries+=1
else:
print "You managed to do it in %d tries."%(tries)
This code has a lot of problems -- enough that it's worth going over in full:
import random
def main():
global tries,moves
The first of many examples of over-use of global variables. When possible, pass parameters; or create a class. This is a general strategy that will help you construct more comprehensible (and thus more debuggable) algorithms; and in a general sense, this is part of why your code fails -- not because of any particular bug, but because the complexity of your code makes it hard to find bugs.
tries,moves=0,0
restart()
def restart():
global a,indexes,x,y
Why do you name your board a? That's a terrible name! Use something descriptive like squares_visited.
a=[[0 for y in range(8)] for x in range(8)] #Costrutto chic
indexes=[x for x in range(8)]
Minor: in python 2, [x for x in range(8)] == range(8) -- they do exactly the same thing, so the list comprehension is unnecessary. In 3, it works a little differently, but if you want a list (rather than a range object) just pass it to list as in (list(range(8))).
#Random part
x=random.randint(0,7)
y=random.randint(0,7)
a[x][y]=1
start()
So my understanding of the code so far is that a is the board, x and y are the starting coordinates, and you've marked the first spot visited with a 1. So far so good. But then things start to get hairy, because you call start at the end of restart instead of calling it from a top-level control function. That's theoretically OK, but it makes the recursion more complicated than necessary; this is another part of your problem.
def start():
global i,indexes,moves,tries
Argh more globals...
i=0
random.shuffle(indexes) #List filled with random numbers that i'll use as indexes
while i<=7:
if indexes[i]==0:
move(-2,-1)
elif indexes[i]==1:
move(-2,1)
elif indexes[i]==2:
move(-1,-2)
elif indexes[i]==3:
move(-1,2)
elif indexes[i]==4:
move(1,-2)
elif indexes[i]==5:
move(1,2)
elif indexes[i]==6:
move(2,-1)
elif indexes[i]==7:
move(2,1)
i+=1
Ok, so what you're trying to do is go through each index in indexes in sequence. Why are you using while though? And why is i global?? I don't see it being used anywhere else. This is way overcomplicated. Just use a for loop to iterate over indexes directly, as in
for index in indexes:
if index==0:
...
Ok, now for the specific problems...
for _ in a:
if 0 in _:
print "Wasted moves: %d"%(moves)
tries+=1
moves=0
restart()
print "Success obtained in %d tries"%(tries)
I don't understand what you're trying to do here. It seems like you're calling restart every time you find a 0 (i.e. an unvisited spot) on your board. But restart resets all board values to 0, so unless there's some other way to fill the board with 1s before hitting this point, this will result in an infinite recursion. In fact, the mutual recursion between move and start might be able to achieve that in principle, but as it is, it's way too complex! The problem is that there's no clear recursion termination condition.
def move(column,row):
global x,y,a,moves
try: b=a[x+row][y+column]
except IndexError: return 0
if b==0 and 0<=x+row<=7 and 0<=y+column<=7:
x=x+row
y=y+column
a[x][y]=1
moves+=1
start()
else: return 0
Here, in principle, the idea seems to be that if your move hits a 1 or goes off the board, then the current branch of the recursion terminates. But because i and indexes are global above in start, when start is re-called, it re-shuffles indexes and resets i to 0! The result is sheer chaos! I can't even begin to comprehend how that will effect the recursion; it seems likely that because i gets reset at the beginning of start every time, and because every successful call of move results in a call of start, the while loop in start will never terminate!
I suppose it's possible that eventually this process will manage to visit every square, at which point things might work as expected, but as it is, this is too complex even to predict.
try :main()
#except: print "I couldn't handle it" <-Row added to prevent python from returning a huge amount of errors
Not sure what you mean by that last line, but it doesn't sound like a good sign -- you're papering over an error instead of finding the root cause.
I'm going to play with this code a bit and see if I can get it to behave marginally better by de-globalizing some of its state... will report back soon.
Update:
Ok I de-globalized indexes as described above. I then replaced the start/restart recursion with an infinite loop in restart, a return statement in start where the call to restart used to be, and a sys.exit() at the end of start (to break out of the infinite loop on success). The result behaves more as expected. This is still poor design but it works now, in the sense that it recursively tries a bunch of random paths until every local position has been visited.
Of course it still doesn't ever succeed; it just keeps looping. Actually finding a solution will probably require a lot more rethinking of this algorithm! But following my above suggestions should help some, at least.
start() and move() call each other, making it an indirect recursive call BUT the move() return startment get out of move() and notout of the recursion.
You see, when you are calling a function, that calls a function that calls a functions, it all goes in a stack that reference each calls. When you get a your final result, you are supposed to go backward, and unstack these function calls.
Here, you don't, you call move(), that calls start(), and it it returns something then you just keep going deeper in the stack.
Try to make an iterative version of your code. Recursion is hard, start with something easier.
If you do want to persist in the recursive version, make move() call itself, and then go backward in the stack from it self once it reach the out condition. It will be clearer than dealing with recursive calls from two functions.
BTW:
Avoid using global variables. Either pass the data as arguments, or use a class. I would use argument passing, it will force you to come up with a better algo that this one.
the while loop is not necessary. Replace it with a for loop on indexes
the huge if/elif statement is not necessary, replace it with a dictionary
You should end up with somthing like this:
for i in indexes:
move(*MOVES[i])
MOVES, being a dict of values of i associated with params for move().
you may want to use generators instead of your list comprehensions, but that would require some algo changes. It could be better for your memory footprint. At the very least, make this shorter:
[x for x in range(8)] can be written range(8)
[[0 for y in range(8)] for x in range(8)] can be written [[0] * 8 for x in range(8)]
range() can be replaced by xrange()