Develop Reference

Julia code not finishing while Python code does

I am very new to Julia and was trying to implement/rewrite some of my previous Python code as practice. I was using the Project Euler problem 25 as practice
In Python I have
def fibonacci(N):
"""Returns the Nth Fibonacci Number"""
F = [0, 1]
i = 0
while i <= N-2:
F_new = F[i] + F[i+1]
F.append(F_new)
i += 1
return F[N]
N = 0
x = 1000
while len(str(fibonacci(N))) <= x:
if len(str(fibonacci(N))) == x:
print(N)
break
N = N + 1
Which runs and gives me the correct answer in about 6.5 seconds. When trying to do this in Julia below
function fib(N)
F = [0, 1]
global i = 1
while i <= N-2
F_new = F[i] + F[i+1]
append!(F, F_new)
global i += 1
end
return F[N]
end
N = 1
x = 1000
while length(string(fib(N))) <= x
if length(string(fib(N))) == x
print(N-1)
break
end
global N += 1
end
The code seems to run "forever". However in the Julia code only when x<= 20 will the code finish and produce the correct answer. In the Julia code when x>20 the program never ends.
I'm not sure where something could go wrong if it runs for all values below 21? Could somebody explain where the error is happening and why?

Python integers are by default unbounded in size and will grow as needed. Julia on the other hand will default to a signed 64 bit integer if on a 64 bit system. (See docs) This begins to overflow when trying to calculate values above around 19 digits long, hence why this starts around x=20. In order to get the same behavior in Julia, you should use the BigInt type for any values or arguments which can get above this size.

The main problem with your code is what #duckboycool has described. The second advice is to always write functions in Julia. Read the Julia performance tips page for a good start.
Note that you can make the function by #Bill 2X faster by removing the unnecessary if like this:
function test(x = 1000)
N = 0
while ndigits(fib(N)) < x
N += 1
end
return N
end
But if you really want a 16000X faster Julia version, then you can do this:
function euler25()
limit = big(10)^999
a, b = big(1), big(1)
N = 2
while b <= limit
a, b = b, a + b
N += 1
end
return N
end
#btime euler25() = 4782
377.700 μs (9573 allocations: 1.15 MiB)
This runs in 377 μs, because we avoid calculating fib(N) at every step from the beginning. And instead of comparing with the length of a string of the output at each iteration, we just compare with 10^999.

In addition to the earlier answer, note that you should avoid globals if looking at performance, so this is much faster than your global i and x code:
function fib(N)
F = [big"0", big"1"]
for i in 1:N-2
F_new = F[i] + F[i+1]
push!(F, F_new)
end
return F[N]
end
function test(x = 1000)
N = 1
while length(string(fib(N))) <= x
if length(string(fib(N))) == x
print(N-1)
break
end
N += 1
end
end
test()

#AboAmmar shows probably the best "normal" way of writing this. But if you want something even a bit more optimized, you can use in-place BigInt calls. I'm not sure whether I would recommend this, but it's nice to be aware of it.
using Base.GMP.MPZ: add!, set!
function euler25_(limit=big(10)^999)
a, b = big(1), big(1)
N = 2
c = big(0)
while b <= limit
add!(c, a, b)
set!(a, b)
set!(b, c)
N += 1
end
return N
end
This uses the special BigInt functions in the GMP.MPZ library, and writes values in-place, avoiding most of the allocations, and running 2.5x faster on my laptop.

How can I compute faster (x**2-2)%n - Python

x = 2**1000000
n = 2**100000000
(x**2-2)%n is too slow. I found pow() but I can't use it because I can't subtract 2. (pow(x, 2)-2)%n and (x*x-2)%n are also slow. When I tested (x*x-2) it was fast but when I added the modulo operator it was slow. Is there a way to compute (x**2-2)%n faster?

Are you running this in the interpreter? I did some testing and the main slowdown seemed to come from the interpreter trying to display the result.
If you assign the expression to a variable, the interpreter won't try to display the result, and it will be very quick:
x = 2**1000000
n = 2**100000000
result = (x**2-2)%n
Addendum:
I was also originally thinking along the same lines as MikeW's answer, and if you wanted every part of the code to be fast, you could take advantage of Python's internal base 2 representation of integers and use bitwise left shifts:
x = 1 << 1000000
n = 1 << 100000000
This comes with the caveat that this only works because x and n are powers of 2, and you have to be more careful to avoid making an off-by-one error. This answer is a good explanation of how bitshifts basically work, but Python is bit different than other languages like C, C++, or Java because Python integers are unlimited precision, so you can never left shift a bit completely away like you could in other languages.

Some module rules :
1) (a+b)mod(n) = amod(n)+bmod(N)
2) (a.b)mod(n) = amod(n).bmod(n)
So you can transform your equation into :
(x**2-2)%n ==> (x.x - 2)%n ==> (x%n).(x%n) - (2%n)
If n is always greater than 2, (2%n) is 2 itself.
solving (x%n) :
If x and n are always in 2**value ; if x > n then (x%n)= 0 is the answer and if x < n (x%n)=x
So the answer is either 0-(2%n) or x**2-(2%n)

If x is always a power of 2, and n is always a power of 2, then you can you can compute it easily and quickly using bit operations on a byte array, which you can then reconstitute into a "number".
If 2^N is (binary) 1 followed by N zeroes, then (2^N)^2 is (binary) 1 followed by 2N zeros.
2^3 squared is b'1000000'
If you have a number 2^K (binary 1 followed by K zeroes), then 2^K - 2 will be K-1 1s (ones) followed by a zero.
eg 2^4 is 16 = b'10000', 2^4 - 2 is b'1110'
If you require "% 2^M" then in binary, you just select the last (lower) M bits, and disregard the rest .
9999 is b'10011100001111'
9999 % 2^8 is b'00001111'
'
Hence combining the parts, if x=2^A and n=2^B, then
(x^2 - 2 ) % n
will be: (last B bits of) (binary) (2*A - 1 '1's followed by a '0')

If you want to compute (x ** y - z) % n
it will be equivalent to ((x ** y) % n - z) % n
Python pow function includes as optional parameter a modulo, as it is very often used and can be computed in an optimized way. So you should use:
(pow(x, y, n) - z) % n

OP says in comment : it's slow because I assign x to the answer and I repeat the process.
I try this :
x = 2**(1000*1000)
n = 2**(100*1000*1000)
import time
t0=time.time()
for i in range(6):
x = (x*x-2)%n
t1=time.time()
print(i,t1-t0)
t0=t1
print(x<n)
"""
0 0.0
1 0.4962291717529297
2 0.5937404632568359
3 1.9043104648590088
4 5.708504915237427
5 16.74528479576111
True
"""
It shows that in this problem, it's just slow because x grows, doubling the number of digit at each loop :
In [5]: %timeit u=x%n
149 ns ± 6.42 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
the %n takes absolutely no time if x<n.

Python while loop speed

I would like it if someone could please explain to me why the following code has so much additional overhead. At 100k iterations, the speed is the same for either case (2.2 sec). When increasing to 1E6 iterations case "B" never finishes, while case "A" takes only 29 seconds.
Case "A"
while n is not 1:
foo
Case "B"
while n > 1:
foo
Complete code if of any help
def coll(n):
count = 0
# while n is not 1:
while n > 1:
count += 1
if not n % 2:
n /= 2
else:
n = 3*n + 1
return count
for x in range(1,100000):
count = coll(x)

First of all, you should use n > 1 or n != 1, not n is not 1. The fact that the latter works is an implementation detail, and obviously it's not working for you.
The reason it's not working is because there are values of x in your code that cause the Collatz sequence to go over the value of sys.maxint, which turns n into a long. Then, even when it ends up going back down to 1, it's actually 1L; a long, not an int.
Try using while n is not 1 and repr(n) != '1L':, and it'll work as you expect. But don't do that; just use n > 1 or n != 1.

Generally in Python, is is extremely fast to check, as it uses referential equality, so all that needs to be checked is that two objects have the same memory location. Note that the only reason your code works is that most implementations of Python generally maintain a pool of the smaller integers, so that every 1 always refers to the same 1, for example, where there may be multiple objects representing 1000. In those cases, n is 1000 would fail, where n == 1000 would work. Your code is relying on this integer pool, which is risky.
> involves a function call, which is fairly slow in Python: n > 1 translates to n.__gt__(1).

Writing a simple function using while

A Python HOMEWORK Assignment asks me to write a function “that takes as input a positive whole number, and prints out a multiplication, table showing all the whole number multiplications up to and including the input number.”(Also using the while loop)
# This is an example of the output of the function
print_multiplication_table(3)
>>> 1 * 1 = 1
>>> 1 * 2 = 2
>>> 1 * 3 = 3
>>> 2 * 1 = 2
>>> 2 * 2 = 4
>>> 2 * 3 = 6
>>> 3 * 1 = 3
>>> 3 * 2 = 6
>>> 3 * 3 = 9
I know how to start, but don’t know what to do next. I just need some help with the algorithm. Please DO NOT WRITE THE CORRECT CODE, because I want to learn. Instead tell me the logic and reasoning.
Here is my reasoning:
The function should multiply all real numbers to the given value(n) times 1 less than n or (n-1)
The function should multiply all real numbers to n(including n) times two less than n or (n-2)
The function should multiply all real numbers to n(including n) times three less than n or (n-3) and so on... until we reach n
When the function reaches n, the function should also multiply all real numbers to n(including n) times n
The function should then stop or in the while loop "break"
Then the function has to print the results
So this is what I have so far:
def print_multiplication_table(n): # n for a number
if n >=0:
while somehting:
# The code rest of the code that I need help on
else:
return "The input is not a positive whole number.Try anohter input!"
Edit: Here's what I have after all the wonderful answers from everyone
"""
i * j = answer
i is counting from 1 to n
for each i, j is counting from 1 to n
"""
def print_multiplication_table(n): # n for a number
if n >=0:
i = 0
j = 0
while i <n:
i = i + 1
while j <i:
j = j + 1
answer = i * j
print i, " * ",j,"=",answer
else:
return "The input is not a positive whole number.Try another input!"
It's still not completely done!
For example:
print_multiplication_table(2)
# The output
>>>1 * 1 = 1
>>>2 * 2 = 4
And NOT
>>> 1 * 1 = 1
>>> 1 * 2 = 2
>>> 2 * 1 = 2
>>> 2 * 2 = 4
What am I doing wrong?

I'm a little mad about the while loop requirement, because for loops are better suited for this in Python. But learning is learning!
Let's think. Why do a While True? That will never terminate without a break statement, which I think is kind of lame. How about another condition?
What about variables? I think you might need two. One for each number you want to multiply. And make sure you add to them in the while loop.
I'm happy to add to this answer if you need more help.
Your logic is pretty good. But here's a summary of mine:
stop the loop when the product of the 2 numbers is n * n.
In the mean time, print each number and their product. If the first number isn't n, increment it. Once that's n, start incrementing the second one. (This could be done with if statements, but nested loops would be better.) If they're both n, the while block will break because the condition will be met.
As per your comment, here's a little piece of hint-y psuedocode:
while something:
while something else:
do something fun
j += 1
i += 1
where should original assignment of i and j go? What is something, something else, and something fun?

This problem is better implemented using nested loops since you have two counters. First figure out the limits (start, end values) for the two counters. Initialize your counters to lower limits at the beginning of the function, and test the upper limits in the while loops.

The first step towards being able to produce a certain output is to recognize the pattern in that output.
1 * 1 = 1
1 * 2 = 2
1 * 3 = 3
2 * 1 = 2
2 * 2 = 4
2 * 3 = 6
3 * 1 = 3
3 * 2 = 6
3 * 3 = 9
The number on the right of = should be trivial to determine, since we can calculate it by multiplying the other two numbers on each row; obtaining those is the core of the assignment. Think of the two operands of * as two counters, let's call them i and j. We can see that i is counting from 1 to 3, but for each i, j is counting from 1 to 3 (resulting in a total of 9 rows; more generally there will be n2 rows). Therefore, you might try using nested loops, one to loop i (from 1 to n) and another to loop j (from 1 to n) for each i. On each iteration of the nested loop, you can print the string containing i, j and i*j in the desired format.

Single Vs. Multiple For Loops In Python?

When comparing the following code samples in python (the 300 is just an example; in practicality, assume there are 10,000,000 items that we need to loop through):
sum = 0
for i in xrange(0, 300):
sum += i
Vs.
sum = 0
for i in xrange(0, 100):
sum += i
for i in xrange(100, 200):
sum += i
for i in xrange(200, 300):
sum += i
Are both versions the same in terms of running time? I have a problem where I have a huge vector (say, 10,000 items). I don't necessarily have to loop through all of it, unless some conditions are met (these conditions can only be assessed when the first loop -- which using the first 100 items -- is done).
Then, I should continue the looping only if some conditions are met (these conditions can only be assessed when the next 100 items are examined).
Then, I should continue the looping only if some conditions are met (these conditions can only be assessed when the next 100 items are examined).
Same story, until I reach the end of the vector................
Given the fact that I can solve my problem in a modular way, by examining points:
{0, 100}
{100, 200}
{200, 300}
........
{9900, 10,000}
All I am trying is to get a sense whether the second approach is as efficient as the first one. Thanks.
note: the sum is used here for simplicity. In practice, the running time of the mathematical operations that will be used is significantly greater. That's why I am trying to find ways to optimize the running time by examining different coding approaches...

What kind of improvement were you hoping to see?
timeit will help you observe the difference:
$ python -m timeit 'sum = 0
> for i in xrange(0, 300):
> sum += i'
10000 loops, best of 3: 21.8 usec per loop
$ python -m timeit 'sum = 0
> for i in xrange(0, 100):
> sum += i
> for i in xrange(100, 200):
> sum += i
> for i in xrange(200, 300):
> sum += i'
10000 loops, best of 3: 22.8 usec per loop
Personally, I would go for:
$ python -m timeit 'sum(xrange(300))'
100000 loops, best of 3: 5.34 usec per loop
This example is perhaps not the best way to assess whether you'll be able to optimise your actual code, but it does demonstrate how you can test a small snippet for its running time.
Just remember what Donald Knuth said about optimisation ;-)
Here's a way you could checkpoint your iteration and break out if required:
>>> for index, item in enumerate(vect):
... results = do_stuff_with(item)
... if index % 100 == 0 and should_we_stop(results):
... break
What would be even better is if you could have do_stuff_with() return a tuple indicating whether it has finished, then you can check every iteration whether you're done, rather than wait:
>>> for item in vect:
... finished, results = do_stuff_with(item)
... if finished:
... break
Just to reiterate (no pun intended) it's very hard to say how (or even whether) to optimise your actual code without actually seeing it!

If you are not going to loop through all of it, then the cost of setting up a new generator (xrange) is quite small.
So the second approach would be faster, if you are sometimes skipping one or two of those loops.
That said, if your list is only 300 big, then the difference is going to be negligible, in the order of milliseconds or microseconds.

If you want to compare speeds, just time them both:
import timeit
chunk1 = '''
sum = 0
for i in xrange(0, 300):
sum += i
'''
chunk2 = '''
sum = 0
for i in xrange(0, 100):
sum += i
for i in xrange(100, 200):
sum += i
for i in xrange(200, 300):
sum += i
'''
timer1 = timeit.Timer(chunk1)
timer2 = timeit.Timer(chunk2)
print timer1.timeit(100000)
print timer2.timeit(100000)
I get these numbers for 100,000 iterations:
3.44955992699
3.56597089767
As you can see, the second chunk of code is slightly slower.
You might be able to try a while loop and a break statement, so here's what I interpret you as trying to do:
i = 0
while i < limit:
stop = False
for j in xrange(i + 100, i + 300):
if i == foo:
stop = True
break
if stop: break
i += 1