can I make it parallelized? - python

it an answer for Euler project #4.
A palindromic number reads the same both ways. The largest palindrome made from the product of two 2-digit numbers is 9009 = 91 99.
Find the largest palindrome made from the product of two 3-digit numbers.
Answer:
906609
code is:
from multiprocessing import Pool
from itertools import product
def sym(lst):
rst=[]
for x,y in lst:
tmp=x*y
if rec(tmp):
rst.append(tmp)
return rst
def rec(num):
num=str(num)
if num == "".join(reversed(num)): return True
else: return False
if __name__ == "__main__":
pool=Pool(processes=8)
lst=product(xrange(100,1000),repeat=2)
rst=pool.map(sym,lst)
#rst=sym(lst)
print max(rst)
when I run this:
# TypeError:'int' object is not iterable
but I can't understand it...isn't list iterable? or is there an error in my code?

Your problem is with the function sym.
sym is being passed the first element of your product iterable. (e.g. lst = (100,100)). When you get to the for loop, you're iterating over lst and then trying to unpack it into two numbers -- equivalent to:
for x,y in (100,100):
...
Which fails for obvious reasons.
I think you probably want to get rid of the for loop all-together -- which was probably an artifact of your serial version.
def sym(lst):
x,y=lst
tmp=x*y
if rec(tmp):
return tmp
else:
return None #max will ignore None values since None > x is always False.
The traceback was somewhat cryptic -- Apparently the traceback gets returned to the Pool which then gets re-raised ... But the way it is done makes it a little difficult to track.
Sometimes, when debugging these things it is helpful to replace Pool.map() with the regular version of map. Then, any exceptions which get raised are raised on your main "thread" and the tracebacks can be a little easier to follow.

Related

Nothing is output in python recursive function

List item
I'm working on a code that calculates the 'distance' between two configurations of a Flip Cube, The distance between two configurations x and y is the minimum number of steps required to go from x to y, or conversely.
To make that I've created a simpler version that makes something different, this code takes two integer numbers ci and cf. with ci returns an iterable called main_level through the generator called multi, then, it iterates through it searching for the parameter cf, whenever cf is not in main_level the variable steps is increased by 1 and for each element in main_level we repeat the same process done for ci. Finally, when cii==cf the program ends and returns the steps variable, which counts the number of "levels" that we have to go down to find the given parameter cf. This code doesn't have any practical purpose is just a base for the problem I mentioned above.
If I call the distance(ci, cf) function with ci=5, the first two levels are:
{0,3,6,9,12} <-- first level (steps is initialized with 1)
if cf is any of the numbers in the set, the program should end and return steps=1,
if cf is not in that set, the programs form the second level:
{15,18,21,24,27,30,33} and search cf, if cf is there, the program ends and should return steps=2, if not, it forms the third level, and so on. But there is a problem, actually, when I call the distance function with ci=5 and cf= any natural number, and print its value, anything is output, only for cf=0, it outputs step=1. I don't really know what's going on. I would appreciate your help.
Here is the code:
#Base solution to FlipCube problem
def multi(par):
for i in range(par):
yield i*3
steps=1
def distance(ci,cf):
main_level =set(multi(ci))
global steps
def check_main_level(cf):
global steps
nonlocal main_level
def lower_level(config_list):
sett=set()
for i in config_list:
sett.update(q for q in multi(i) if q not in config_list)
nonlocal main_level
main_level=sett
check_main_level(cf)
for i in main_level:
if i==cf:
break
else:
steps+=1
lower_level(main_level)
check_main_level(cf)
return steps
#testing
e= distance(5,0)
print(e)# prints 1, very good
e2= distance(5,9)
print(e2)# should print 1, but doesn't print anything :(
e3= distance(5,27)
print(e3)# should print 2, but doesn't print anything :(
The program does not terminate recursion under all circumstances. The culprit seems to be the for loop in check_main_level. Change the code after your definition of lower_level to:
# code portion of check_main_level
if cf > max(main_level):
steps+=1
lower_level(main_level)
# end of code portion check_main_level (replacing for-loop)
you have an infinity loop, that's why nothing is printed.
You can see it easyly by adding a print :
for i in config_list:
print(i)
sett=set()
sett.update(q for q in list(multi(i)) if q not in config_list)

function returns none instead of a tuple

I am using python 2.7. I wrote a code to generate passwords. For doing this, I used the random module to generate how many characters of different types(uppercase, lowercase, special and numbers) to be used to generate a password of a given length. When a wrote a function for this, it was supposed to return a tuple, but it returns None. Why is it happening?
I tried the casual debugging method of putting print statements in between and figuring out where it went wrong. Everything works just fine, except it returns None.
def passlen(a):
"""Gives length of different characters to be put into passwords"""
uplen=random.randrange(0, a)
lwlen=random.randrange(0, a)
speclen=random.randrange(0, a)
nmbrlen=random.randrange(0, a)
if uplen+lwlen+speclen+nmbrlen==a:
print (uplen, lwlen, speclen, nmbrlen)
return(uplen, lwlen, speclen, nmbrlen)
else:
passlen(a)
x=input("how many characters in the password?")
print(passlen(x))
Expected results are 4-tuples, but it gives None instead.
So you want four random numbers that add to a? Of course you can try choosing four random numbers until you find a set that adds up to a, but that might take a while for large values of a (and you definitely don't want to do this recursively).
Much better to choose three split points between 0 and a:
def passlen(a):
splits = sorted([random.randrange(0,a) for _ in range(3)])
uplen = splits[0]
lwlen = splits[1] - uplen
speclen = splits[2] - uplen - lwlen
nmbrlen = a - uplen - lwlen - speclen
return uplen, lwlen, speclen, nmbrlen
Thanks to Kamiccolo for trying to help out.
The function should look like this:
def passlen(a):
"""Gives length of different characters to be put into passwords"""
uplen=int(random.randrange(0, a))
lwlen=int(random.randrange(0, a))
speclen=int(random.randrange(0, a))
nmbrlen=int(random.randrange(0, a))
bab=(uplen, lwlen, speclen, nmbrlen)
if uplen+lwlen+speclen+nmbrlen==a:
return bab
else:
return passlen(a)
A duplicate thread also helped me in this.

How to stop recursion in my bubble sort code when the list is sorted?

Problem: Implement bubble sort in python. Don’t use Python’s built in sort or sorted. Assume your inputs will be sufficient for the memory you have.
Code :
def check_if_sorted(details):
"""compares two consecutive integers in a list and returns the list if sorted else get the list sorted by calling another function"""
i = 0
while i <= len(details) - 2:
if details[i] < details[i+1]:
status = True
i = i+1
print details
else:
bubble_sort(details)
if status :
return details
def bubble_sort(details):
"""compares two consecutive integers in a list and shifts the smaller one to left """
for i in range(len(details)-1):
if details[i] > details[i+1]:
temp = details[i]
details[i]= details[i+1]
details[i+1] = temp
return check_if_sorted(details)
sort_me = [11,127,56,2,1,5,7,9,11,65,12,24,76,87,123,65,8,32,86,123,67,1,67,92,72,39,49,12, 98,52,45,19,37,22,1,66,943,415,21,785,12,698,26,36,18,97,0,63,25,85,24,94,1501]
print sort_me
print bubble_sort(sort_me)
I have written the following code but it keeps on running even after sorting the list and then prints messsage "RuntimeError: maximum recursion depth exceeded while calling a Python object". How can I stop recursion after checking the list is sorted?
you don't need check_if_sorted(details) function. use try/except in your sort function to check IndexError and keep calling the bubble_sort() function.
def bubble_sort(details):
"""compares two consecutive integers in a list and shifts the smaller one to left """
for i in range(len(details)-1):
try:
if details[i] > details[i+1]:
temp = details[i]
details[i]= details[i+1]
details[i+1] = temp
bubble_sort(details)
except IndexError:
return
return details
sort_me = [11,127,56,2,1,5,7,9,11,65,12,24,76,87,123,65,8,32,86,123,67,1,67,92,72,39,49,12, 98,52,45,19,37,22,1,66,943,415,21,785,12,698,26,36,18,97,0,63,25,85,24,94,1501]
print(sort_me)
print(bubble_sort(sort_me))
In check_if_sorted you do not check for equal. This causes the duplicates in your list to trigger another (unneeded) calling to bubble_sort causing an infinite loop. To fix it change the comparison line in check_if_sorted to:
if details[i] <= details[i+1]:
Edit:
This solves the infinite loop, However your algorithm is very inefficient. To improve your algorithm I suggest a google search for 'bubble sort algorithm', and/or a discussion with your instructor

'float' object can't be interpreted as int, but converting to int yields no output

So I'm trying to test if something is a palindrome. Here is my code:
This function returns a string of the first half of a larger string. ("TEST" returns "TE", "HELLO" returns "HE")
def takeStart(s):
start = ""
# The following determines the final index of the first half
if len(s)%2==0:
a = (len(s)/2)-1
else:
a = ((len(s)-1)/2)-1
for i in range(a):
start+=s[i]
return start
This function returns a string of the second half of a larger string. ("TEST" returns "ST", "HELLO" returns "LO")
def takeEnd(s):
end = ""
# The following determines the beginning index of the second half
if len(s)%2==0:
a = (len(s)/2)
else:
a = ((len(s)-1)/2)
for i in range(a,len(s)):
end+=s[i]
return end
This function flips a string. ("TEST" returns "TSET", "HELLO" returns "OLLEH")
def flip(s):
flipped = ""
for i in range(1,len(s)):
flipped+=s[len(s)-i]
flipped+=s[0]
return flipped
This code takes every product of two 3-digit numbers, and checks if it's a palindrome
for i in range(100,1000):
for q in range(100,1000):
a = i*q
if takeStart(str(a)) == flip(takeEnd(str(a))):
print(str(a))
When this code is run, it outputs:
Traceback (most recent call last):
File "[redacted]", line 39, in <module>
if takeStart(str(a)) == flip(takeEnd(str(a))):
File "[redacted]", line 14, in takeStart
for i in range(a):
TypeError: 'float' object cannot be interpreted as an integer
Alright, I thought I just convert a to an integer and all should be swell.
Doing that appears to remove all errors, but there is NO output whatsoever. (There are new lines every once and a while which makes me think it's working but not outputing any data)
Any ideas on why this is happening?
UPDATE: my code now:
def takeStart(s):
start = ""
if len(s)%2==0:
a = (len(s)//2)
else:
a = (len(s)-1)//2
return start[0:a]
def takeEnd(s):
end = ""
if len(s)%2==0:
a = (len(s)//2)
else:
a = ((len(s)-1)//2)
return end[int(a):len(s)]
def flip(s):
return s[::-1]
for i in range(100,1000):
for q in range(100,1000):
a = i*q
if takeStart(str(a)) == flip(takeEnd(str(a))):
print(str(a))
This is just outputting every single number.
I tested each method and they're returning empty strings. (I'm assuming), which is why every number is passing the palindrome check and printing.
First, using range(int(a)) and range(int(a), len(s)) will solve your error. As Jon Clements points out, you can solve that more easily by just using // instead of / to get integers in the first place. But either way, it's not causing any problems.
Your problem is that ranges, and just about everything related in Python, are half-open. So, your takeStart function is returning all the values up to, but not including, the half-way point—that is, it gives you H for HELLO, T for TEST, BIGG for BIGGERTEST.
Just get rid of the -1 on your a = … lines, and that will solve that problem.
And then it prints out a whole bunch of output lines, all palindromes, which I assume is what you were intending to do.
However, you're still not going to get any odd-length palindromes. For example, with 'MADAM', even when you get the functions right, takeStart(s) is MA, takeEnd(s) is DAM, flip(takeEnd(s)) is MAD, and that's not the same as MAD. Even though your functions are working right, they're not solving the problem. So there's a bug in your design as well as your implementation. If you think about it for a while, you should figure out how to make this work.
And, once you do, you should realize that takeStart and takeEnd can be simplified a lot. (Hint: In which cases do you really need to treat odd and even lengths differently?)
While we're at it, this:
foo = ""
for i in range(x, y):
foo += s[i]
return foo
… is just a verbose, slow, and easy-to-get-wrong way of writing this:
return foo[x:y]
And, likewise, your whole flipped function is just:
return s[::-1]
Pythons range() function does not support floats but numpys arange()
function does
To use arange() function, you need to install and import the numpy package.
We’ll here use the arange() function for generating a range of float numbers.
The arange() has the same signature as the built-in range() method. But we can pass float type arguments as parameters to this function.
import numpy
arange (start, stop, step)
In a full code example this looks as follows:
from numpy import arange
print("Float range using NumPy arange():")
print("\nTest 1:")
for i in arange(0.0, 1.0, 0.1):
print(i, end=', ')

What's causing the 135k/sec page faults in this python code? (trial-division prime sieve)

The following code executes in a work thread, and just happily spins away, receiving pause/report commands, etc. I assume they're soft faults since I'm not having RAM use issues and my hard drive didn't melt over the weekend, and that it's got something to do with how long I've let results accumulate, as I'm only getting ~50 page faults when I started it up a few days ago.
The "counter" attribute is currently at 22,496,115, and "results" has 1,418,641 elements.
The slice of "results" is taken because I was feeling contrary and started the list at 1.
def run(self):
while self.keep_running:
self.lock.acquire()
is_prime = True
self.counter += 1
cutoff_val = pow(self.counter,.5)
for number in self.results[1:]:
if number > cutoff_val:
break
if self.counter % number == 0:
is_prime = False
break
if is_prime:
self.results.append(self.counter)
self.lock.release()
Note: I know that I could use the Sieve of Eratosthenes to optimize the algorithm and probably cut down on the page faults, but that's not the point: I'm trying to pinpoint the exact reason - or at least the worst offender - behind the page faults so I can avoid doing the same kinds of things in the future. This algorithm is used simply for testing UI responsiveness when I need a "stupidly expensive, simple work thread."
Additional setup pieces as requested:
def __init__(self):
self.counter = 0
self.keep_running = False;
self.lock = threading.Lock()
self.results = list()
def __call__(self, *args):
if not self.keep_running:
self.keep_running = True
self.run()
I think that #John Gaines Jr. has pointed out something you need to change. If your list is really big, you don't want to be making copies of it like that.
Here is a good way to loop over the same values as self.results[1:] but without making a copy:
res = iter(self.results) # get an iterator for the list values
next(res) # iterate once to throw away first value
for number in res:
# same code you already have goes here
...
EDIT: The above code is correct and simple, but doesn't scale well. I thought about it and figured there must be something in itertools for this, and sure enough there is:
import itertools as it
res = it.islice(self.results, 1, None)
for number in res:
# same code you already have goes here
...
EDIT: Thank you to #John Gaines Jr. for pointing out that you can use None instead of len(self.results) in the call to it.islice().
From the Python tutorial Lists section:
All slice operations return a new list containing the requested elements. This means that the following slice returns a shallow copy of the list a:
>>> a[:]
['spam', 'eggs', 100, 1234]
So in your for loop, the bit self.results[1:] results in a copy of the results list. If this routine is being called over and over, it could quite easily cause memory thrashing.

Categories

Resources