I'm very unfamiliar with Python 2.x and I have some inherited code in many files where I need to count how many times some particular functions are called (to solve a particular problem).
To show what I mean, in another simple language I would do this
Add at the beginning of the code
counter_funcA = 0
counter_funcB = 0
...
In the beginning of function A
counter_funcA += 1
In the beginning of function B
counter_funcB += 1
And so on...
And at the end
print counter_funcA
...
So how would a python expert do this in a nice way?
I guess the operation += is quite atomic -- so there will not be any concurrency issues?
Related
I've been working on solving Problem 31 from Project Euler. I'm almost finished, but my code is giving me what I know is the wrong answer. My dad showed the right answer, and mine is giving me something completely wrong.
coins = (1,2,5,10,20,50,100,200)
amount = 200
def ways(target, avc):
target = amount
if avc <= 1:
return 1
res = 0
while target >= 0:
res = res + ways(target, avc-1)
target = target - coins[avc]
return res
print(ways(amount, 7))
This is the answer I get.
284130
The answer is supposed to 73682.
EDIT: Thank you to everyone who answered my question. Thanks to all of you, I have figured it out!
I see several ways how you can improve the way you're working on problems:
Copy the task description into your source code. This will make you more independent from Internet resources. That allows you to work offline. Even if the page is down or disappears completely, it will enable someone else understanding what problem you're trying to solve.
Use an IDE that does proper syntax highlighting and gives you hints what might be wrong with your code. The IDE will tell you what #aronquemarr mentioned in the comments:
There's also a thing called magic numbers. Many of the numbers in your code can be explained, but why do you start with an avc of 7?. Where does that number come from? 7 does not occur in the problem description.
Use better naming, so you understand better what your code does. What does avc stand for? If you think it's available_coins, that's not correct. There are 8 different coins, not 7.
If it still does not work, reduce the problem to a simpler one. Start with the most simple one you can think of, e.g. make only 1 type of coin available and set the amount to 2 cent. There should only be 1 solution.
To make sure this simple result will never break, introduce an assertion or unit test. They will help you when you change the code to work with larger datasets. They will also change the way you write code. In your case you'll find that accessing the variable coins from an outer scope is probably not a good idea, because it will break your assertion when you switch to the next level. The change that is needed will make your methods more self-contained and robust.
Then, increase the difficulty by having 2 different coins etc.
Examples for the first assertions that I used on this problem:
# Only 1 way of making 1 with only a 1 coin of value 1
assert(find_combinations([1], 1) == 1)
# Still only 1 way of making 1 with two coins of values 1 and 2
assert(find_combinations([1,2], 1) == 1)
# Two ways of making 2 with two coins of values 1 and 2
assert(find_combinations([1,2], 2) == 2)
As soon as the result does no longer match your expectation, use the debugger and step through your code. After every line, check the values in the debugger against the values that you think they should be.
One time, the debugger will be your best friend. And you can never imagine how you did stuff without it. You just need to learn how to use it.
Firstly, read Thomas Weller's answer. They give some excellent suggestions as to how to improve your coding and problem solving.
Secondly, your code works, and gives the correct answer after the following changes:
As suggested, remove the target = amount line. You're resigning the argument of the function to global amount on each call (even on the recursive calls). Having removed that the answer comes up to 3275 - still not the right answer.
The other thing you have to remember is that Python is 0-indexed. Therefore, your simplest case condition ought to read if avc <= 0: ... (not <=1).
Having made these two changes, your code gives the correct answer. Here is your code with these changes:
coins = (1,2,5,10,20,50,100,200)
amount = 200
def ways(target, avc):
if avc <= 0:
return 1
res = 0
while target >= 0:
res = res + ways(target, avc-1)
target = target - coins[avc]
return res
print(ways(amount, 7))
Lastly, there are plenty of Project Euler answers out there. Having solved the yourself, it might be useful to have a look at what others did. For reference, I have not actually solved this Project Euler before, so I had to do that first. Here is my solution. I've added a pile of comments on top of it to explain what it does.
EDIT
I've just noticed something quite worrying: Your code (after fixes) works only if the first element of coins is 1. All the other elements can be shuffled:
# This still works ok
coins = (1,2,200,10,20,50,100,5)
# But this does not
coins = (2,1,5,10,20,50,100,200)
To ensure that this is always the case, you can just do the following:
coins = ... # <- Some not necessarily sorted tuple
coins = tuple(sorted(coins))
In principle there are a few other issues. Your solution breaks for non-unique values coins, and which don't include 1. The former you could fix with the use of sets, and the latter by modifying your if avc <= 0: case (check for the divisibility of the target by the remaining coin). Here is you piece of code with these changes implemented. I've also renamed the variables and the function to be a bit easier to read, and used coins as the argument, rather that avc pointer (which, by the way, I could not stop reading as avec):
unique = lambda x: sorted(set(x)) # Sorted unique list
def find_combinations(target, coins):
''' Find the number of ways coins can make up the target amount '''
coins = unique(coins)
if len(coins)==1: # Only one coin, therefore just check divisibility
return 1 if not target%coins[0] else 0
combinations = 0
while target >= 0:
combinations += find_combinations(target, coins[:-1])
target -= coins[-1]
return combinations
coins = [2,1,5,10,20,50,100,200] # <- Now works
amount = 200
print(find_combinations(amount, coins))
I'm currently writing a simulation in python 3.4 (miniconda).
The entire simulation is quite fast but the measurement of some simulation data is bloody slow and takes about 35% of the entire simulation time. I hope I can increase the performance of the entire simulation if I could get rid of that bottleneck. I spend quite some time to figure out how to do that but unfortunately with little success. The function MeasureValues is called in every period of the simulation run.
If anybody has an idea how to improve the code, I would be really grateful.
Thank you guys.
def MeasureValues(self, CurrentPeriod):
if CurrentPeriod > self.WarmUp:
self.ValueOne[CurrentPeriod] = self.FixedValueOne if self.Futurevalue[CurrentPeriod + self.Reload] > 0 else 0
self.ValueTwo[CurrentPeriod] = self.VarValueTwo * self.AmountValueTwo[CurrentPeriod]
self.ValueThree[CurrentPeriod] = self.VarValueThree * self.AmountValueThree[CurrentPeriod]
self.SumOfValues[CurrentPeriod] = self.ValueOne[CurrentPeriod] + self.ValueTwo[CurrentPeriod] + self.ValueThree[CurrentPeriod]
self.TotalSumOfValues += self.SumOfValues[CurrentPeriod]
self.HelperSumValueFour += self.ValueFour[CurrentPeriod]
self.HelperSumValueTwo += self.AmountValueTwo[CurrentPeriod]
self.HelperSumValueFive += self.ValueFive[CurrentPeriod]
self.RatioOne[CurrentPeriod] = (1 - (self.HelperSumValueFour / self.HelperSumValueFive )) if self.HelperSumValueFive > 0 else 1
self.RatioTwo[CurrentPeriod] = (1 - (self.HelperSumValueTwo / self.HelperSumValueFive )) if self.HelperSumValueFive > 0 else 1
The code looks basic enough that there aren't any obvious gaps for optimisation without substantial restructuring (which I don't know enough about your overall architecture to suggest).
Try installing cython - I believe nowadays you can install it with pip install cython - then use it to see if you can speed up your code any.
As stated in the comments the function is quite simple, and with the provided code I don't see a way to optimize it directly.
You can try different approaches:
PyPy, this may works depending on your current codebase and external dependecies.
Cython as suggested by holdenweb, but you need to redefine a lot of stuff to be static-typed (using cdef) or nothing will change.
rewrite the function in C as a Python extension, this may take some time, especially if you have no experience in C programming.
The PyPy way seems the most reasonable, if it works you will gain a boost for all the simulations code.
This sounds like it should be blindingly obvious but there's something quirky going on.
So I have two scripts. Largely the same but a few variances here and there. They both run big loops and they both use global variables. Yes, I know this is bad. Tidying it up is on my to do list but I'm on a time constraint at the moment.
One script works fine. As expected.
The other works fine... for the most part. Then it bugs out once it iterates through the loop again. It's very strange because the code has very little differences but I did pick up on something which I can't explain.
The variable I use to track my position in the loop is set like this in script 1 (which works flawlessly):
global CurrentMatchScan
while True:
print "=Starting Loop="
Loopcounter = 0
CurrentMatchScan = -1
LoadMatch()
The second script has:
global CurrentMatchScan
while True:
print "=Starting Loop="
Loopcounter = 0
CurrentMatchScan = -1
LoadMatch()
However in the second script PyCharm highlights the = -1 part as
Redclared'CurrentMatchScan' defined above without usage
So the obvious assumption is something further up in the code is using it. I do call a function later on which is placed up there...
def LoadMatch():
nextbox1 = LinkList[CurrentMatchScan + 1]
nextbox2 = LinkList[CurrentMatchScan + 2]
But this is only called once CurrentMatchScan is set... and the first script has the exact same code.
Is my IDE just not noticing it on the other script or something? There's a pretty big issue with the looping on the second one and this is the only difference I can see between the two.
I know this doesn't look like a lot to go on but there's not much else to it that I can see. It's barely actually referenced.
I'd really appreciate anyone who can point out how much of an idiot I'm being and missing something really simple.
/edit:
Based on the fact that it does use CurrentMatchScan and calls LoadMatch() for the first iterations properly I'm starting to doubt this is anything but PyCharm trying to warn me I'm potentially doing something silly.
I think if this was the issue it wouldn't work at all so the warning might be a bit of a red herring when it comes to the issue I'm actually facing.
Is it possible save the evolution of constants of a program, a small one indeed, in one file ? In other word, is there a way to store the informations sent by pdb?
Suppose for example that we have the following code.
a = 1
b = 2
a = a + b
I would like to have something like the following.
a initialized to 1
b initialized to 2
a + b stored in a
I'm a little confused about what, exactly, you're asking. But it sounds like you want to implement logging.
I think I may have implemented this incorrectly because the results do not make sense. I have a Go program that counts to 1000000000:
package main
import (
"fmt"
)
func main() {
for i := 0; i < 1000000000; i++ {}
fmt.Println("Done")
}
It finishes in less than a second. On the other hand I have a Python script:
x = 0
while x < 1000000000:
x+=1
print 'Done'
It finishes in a few minutes.
Why is the Go version so much faster? Are they both counting up to 1000000000 or am I missing something?
One billion is not a very big number. Any reasonably modern machine should be able to do this in a few seconds at most, if it's able to do the work with native types. I verified this by writing an equivalent C program, reading the assembly to make sure that it actually was doing addition, and timing it (it completes in about 1.8 seconds on my machine).
Python, however, doesn't have a concept of natively typed variables (or meaningful type annotations at all), so it has to do hundreds of times as much work in this case. In short, the answer to your headline question is "yes". Go really can be that much faster than Python, even without any kind of compiler trickery like optimizing away a side-effect-free loop.
pypy actually does an impressive job of speeding up this loop
def main():
x = 0
while x < 1000000000:
x+=1
if __name__ == "__main__":
s=time.time()
main()
print time.time() - s
$ python count.py
44.221405983
$ pypy count.py
1.03511095047
~97% speedup!
Clarification for 3 people who didn't "get it". The Python language itself isn't slow. The CPython implementation is a relatively straight forward way of running the code. Pypy is another implementation of the language that does many tricky (especiallt the JIT) things that can make enormous differences. Directly answering the question in the title - Go isn't "that much" faster than Python, Go is that much faster than CPython.
Having said that, the code samples aren't really doing the same thing. Python needs to instantiate 1000000000 of its int objects. Go is just incrementing one memory location.
This scenario will highly favor decent natively-compiled statically-typed languages. Natively compiled statically-typed languages are capable of emitting a very trivial loop of say, 4-6 CPU opcodes that utilizes simple check-condition for termination. This loop has effectively zero branch prediction misses and can be effectively thought of as performing an increment every CPU cycle (this isn't entirely true, but..)
Python implementations have to do significantly more work, primarily due to the dynamic typing. Python must make several different calls (internal and external) just to add two ints together. In Python it must call __add__ (it is effectively i = i.__add__(1), but this syntax will only work in Python 3.x), which in turn has to check the type of the value passed (to make sure it is an int), then it adds the integer values (extracting them from both of the objects), and then the new integer value is wrapped up again in a new object. Finally it re-assigns the new object to the local variable. That's significantly more work than a single opcode to increment, and doesn't even address the loop itself - by comparison, the Go/native version is likely only incrementing a register by side-effect.
Java will fair much better in a trivial benchmark like this and will likely be fairly close to Go; the JIT and static-typing of the counter variable can ensure this (it uses a special integer add JVM instruction). Once again, Python has no such advantage. Now, there are some implementations like PyPy/RPython, which run a static-typing phase and should fare much better than CPython here ..
You've got two things at work here. The first of which is that Go is compiled to machine code and run directly on the CPU while Python is compiled to bytecode run against a (particularly slow) VM.
The second, and more significant, thing impacting performance is that the semantics of the two programs are actually significantly different. The Go version makes a "box" called "x" that holds a number and increments that by 1 on each pass through the program. The Python version actually has to create a new "box" (int object) on each cycle (and, eventually, has to throw them away). We can demonstrate this by modifying your programs slightly:
package main
import (
"fmt"
)
func main() {
for i := 0; i < 10; i++ {
fmt.Printf("%d %p\n", i, &i)
}
}
...and:
x = 0;
while x < 10:
x += 1
print x, id(x)
This is because Go, due to it's C roots, takes a variable name to refer to a place, where Python takes variable names to refer to things. Since an integer is considered a unique, immutable entity in python, we must constantly make new ones. Python should be slower than Go but you've picked a worst-case scenario - in the Benchmarks Game, we see go being, on average, about 25x times faster (100x in the worst case).
You've probably read that, if your Python programs are too slow, you can speed them up by moving things into C. Fortunately, in this case, somebody's already done this for you. If you rewrite your empty loop to use xrange() like so:
for x in xrange(1000000000):
pass
print "Done."
...you'll see it run about twice as fast. If you find loop counters to actually be a major bottleneck in your program, it might be time to investigate a new way of solving the problem.
#troq
I'm a little late to the party but I'd say the answer is yes and no. As #gnibbler pointed out, CPython is slower in the simple implementation but pypy is jit compiled for much faster code when you need it.
If you're doing numeric processing with CPython most will do it with numpy resulting in fast operations on arrays and matrices. Recently I've been doing a lot with numba which allows you to add a simple wrapper to your code. For this one I just added #njit to a function incALot() which runs your code above.
On my machine CPython takes 61 seconds, but with the numba wrapper it takes 7.2 microseconds which will be similar to C and maybe faster than Go. Thats an 8 million times speedup.
So, in Python, if things with numbers seem a bit slow, there are tools to address it - and you still get Python's programmer productivity and the REPL.
def incALot(y):
x = 0
while x < y:
x += 1
#njit('i8(i8)')
def nbIncALot(y):
x = 0
while x < y:
x += 1
return x
size = 1000000000
start = time.time()
incALot(size)
t1 = time.time() - start
start = time.time()
x = nbIncALot(size)
t2 = time.time() - start
print('CPython3 takes %.3fs, Numba takes %.9fs' %(t1, t2))
print('Speedup is: %.1f' % (t1/t2))
print('Just Checking:', x)
CPython3 takes 58.958s, Numba takes 0.000007153s
Speedup is: 8242982.2
Just Checking: 1000000000
Problem is Python is interpreted, GO isn't so there's no real way to bench test speeds. Interpreted languages usually (not always have a vm component) that's where the problem lies, any test you run is being run in interpreted bounds not actual runtime bounds. Go is slightly slower than C in terms of speed and that is mostly due to it using garbage collection instead of manual memory management. That said GO compared to Python is fast because its a compiled language, the only thing lacking in GO is bug testing I stand corrected if I'm wrong.
It is possible that the compiler realized that you didn't use the "i" variable after the loop, so it optimized the final code by removing the loop.
Even if you used it afterwards, the compiler is probably smart enough to substitute the loop with
i = 1000000000;
Hope this helps =)
I'm not familiar with go, but I'd guess that go version ignores the loop since the body of the loop does nothing. On the other hand, in the python version, you are incrementing x in the body of the loop so it's probably actually executing the loop.