I am trying to profile our django unittests (if the tests are faster, we'll run 'em more often). I've ran it through python's built in cProfile profiler, producing a pstats file.
However the signal to noise ration is bad. There are too many functions listed. Lots and lots of django internal functions are called when I make one database query. This makes it hard to see what's going on.
Is there anyway I can "roll up" all function calls that are outside a certain directory?
e.g. if I call a python function outside my directory, and it then calls 5 other functions (all outside my directory), then it should roll all those up, so it looks like there was only one function call, and it should show the cumulative time for the whole thing.
This, obviously, is bad if you want to profile (say) Django, but I don't want to do that.
I looked at the pstats.Stats object, but can't see an obvious way to modify this data.
I have little experience with python, but a lot in performance tuning, so here's a possibility:
Rather than run profiling as part of the unit tests, just do overall execution time measurements. If those don't change much, you're OK.
When you do detect a change, or if you just want to make things run faster, use this method. It has a much higher "signal-to-noise ratio", as you put it.
The difference is you're not hoping the profiler can figure out what you need to be looking at. It's more like a break point in a debugger, where the break occurs not at a place of your choosing, but with good probability at a time of unnecessary slowness.
If on two or more occasions you see it doing something that might be replaced with something better, on average it will pay off, and fixing it will make other problems easier to find by the same method.
Related
I have some pretty fragile code that I want to refactor. It's not very easy to unit test by itself because it interacts with database queries and Django form data.
That in itself is not a big deal. I already have extensive tests that, among other things, end up calling this function and check that results are as expected. But my full test suite takes about 5 minutes and I also don't want to have to fix other outstanding issues while working on this.
What I'd like to do is to run nosetests or nose2 on all my tests, track all test_xxx.py files that called the function of interest and then limit my testing during the refactoring to only that subset of test files.
I plan to use inspect.stack() to do this but was wondering if there is an existing plugin or if someone has done it before. If not, I intend to post whatever I come up with and maybe that will be of use later.
You can simply raise some exception in the function and do one run. All tests that fail do call you function.
I just want to check whether a particular function is called by other function or not. If yes then I have to store it in a different category and the function that does not call a particular function will be stored in different category.
I have 3 .py files with classes and functions in them. I need to check each and every function. e.g. let's say a function trial(). If a function calls this function, then that function is in example category else non-example.
I have no idea what you are asking, but even if it is be technically possible, the one and only answer: don't do that.
If your design is as such that method A needs to know whether it was called from method B or C; then your design is most likely ... broken. Having such dependencies within your code will quickly turn the whole thing un-maintainable. Simply because you will very soon be constantly asking yourself "that path seems fine, but will happen over here?"
One way out of that: create different methods; so that B can call something else as C does; but of course, you should still extract the common parts into one method.
Long story short: my non-answer is: take your current design; and have some other people review it. As you should step back from whatever you are doing right now; and find a way to it differently! You know, most of the times, when you start thinking about strange/awkward ways to solve a problem within your current code, the real problem is your current code.
EDIT: given your comments ... The follow up questions would be: what is the purpose of this classification? How often will it need to take place? You know, will it happen only once (then manual counting might be an option), or after each and any change? For the "manual" thing - ideas such as pycharm are pretty good in analyzing python source code, you can do simple things like "search usages in workspaces" - so the IDE lists you all those methods that invoke some method A. But of course, that works only for one level.
The other option I see: write some test code that imports all your methods; and then see how far the inspect module helps you. Probably you could really iterate through a complete method body and simply search for matching method names.
Is there a tool that is able to measure the frequency of function calls in a project and counts other aspects (for statistics purposes) of Python code?
thank you
The Python profiler should provide you with quite a bit of information:
python -m cProfile -o fooprof myscript.py
You load the results using a simple script:
#!/usr/bin/env python
import pstats
p = pstats.Stats('fooprof')
p.strip_dirs().sort_stats(-1).print_stats()
Alternatively, if you do not specify -o fooprof the results are printed to stdout.
See the documentation at http://docs.python.org/library/profile.html
I'm not sure what "other aspects" you are wanting to count, but this will determine how many times the function is called. If you are interested in the "frequency" of calls, then you can find the average frequency as a function of total execution time and the number of times the function was called. For example:
Suppose foo() was called 100 times in 10 seconds. The average frequency of calls is then 10/second.
I guess you want to do static code analysis. How many locations in your code
call a function.
This is very hard to do in dynamic languages like python, because there are so many
ways functions may be called otherwise than by proper name, and even the python bytecode compiler
won't know always which function is going to be called in a place, and it may even change
during execution. And there's standard OO polymorphism too.
Consider:
def doublefx(f, x):
return f(x)*2
print doublefx(math.sqrt, 9) # 6
f = stdin.readline()
print doublefx(getattr(math, f), 9) # ?
No way any static analyses tool will find out which functions in math.* are going to
be called by this code. Even the first example would be very hard to reason about,
the second is impossible.
The following tool does some static analysis regarding complexity.
http://sourceforge.net/projects/pymetrics/
Other analysis tools like PyLint and PyChecker rather focus on style and probable
errors.
I've never used it, but it looks like cProfile might be a good place to start. (Note that of the three profilers mentioned on that page, one (hotshot) is experimental and not well-supported, one (profile) "adds significant overhead to profiled programs", and the last is cProfile, so probably go with that.) Links to the source code for profile.py and pstats.py are provided at the top of the page.
For my current job I am writing some long-running (think hours to days) scripts that do CPU intensive data-processing. The program flow is very simple - it proceeds into the main loop, completes the main loop, saves output and terminates: The basic structure of my programs tends to be like so:
<import statements>
<constant declarations>
<misc function declarations>
def main():
for blah in blahs():
<lots of local variables>
<lots of tightly coupled computation>
for something in somethings():
<lots more local variables>
<lots more computation>
<etc., etc.>
<save results>
if __name__ == "__main__":
main()
This gets unmanageable quickly, so I want to refactor it into something more manageable. I want to make this more maintainable, without sacrificing execution speed.
Each chuck of code relies on a large number of variables however, so refactoring parts of the computation out to functions would make parameters list grow out of hand very quickly. Should I put this sort of code into a python class, and change the local variables into class variables? It doesn't make a great deal of sense tp me conceptually to turn the program into a class, as the class would never be reused, and only one instance would ever be created per instance.
What is the best practice structure for this kind of program? I am using python but the question is relatively language-agnostic, assuming a modern object-oriented language features.
First off, if your program is going to be running for hours/days then the overhead of switching to using classes/methods instead of putting everything in a giant main is pretty much non-existent.
Additionally, refactoring (even if it does involve passing a lot of variables) should help you improve speed in the long run. Profiling an application which is designed well is much easier because you can pin-point the slow parts and optimize there. Maybe a new library comes along that's highly optimized for your calculations... a well designed program will let you plug it in and test right away. Or perhaps you decide to write a C Module extension to improve the speed of a subset of your calculations, a well designed application will make that easy too.
It's hard to give concrete advice without seeing <lots of tightly coupled computation> and <lots more computation>. But, I would start with making every for block it's own method and go from there.
Not too clean, but works well in little projects...
You can start using modules as if they were singleton instances, and create real classes only when you feel the complexity of the module or the computation justifies them.
If you do that, you would want to use "import module" and not "from module import stuff" -- it's cleaner and will work better if "stuff" can be reassigned. Besides, it's recommended in the Google guidelines.
Using a class (or classes) can help you organize your code.
Simplicity of form (such as through use of class attributes and methods) is important because it helps you see your algorithm, and can help you more easily unit test the parts.
IMO, these benefits far outweigh the slight loss of speed that may come with using OOP.
I know this question is kind of stupid, maybe it's a just a part of writing code but it seems defining simple functions can really hurt performance severely... I've tried this simple test:
def make_legal_foo_string(x):
return "This is a foo string: " + str(x)
def sum_up_to(x):
return x*(x+1)/2
def foo(x):
return [make_legal_foo_string(x),sum_up_to(x),x+1]
def bar(x):
return ''.join([str(foo(x))," -- bar !! "])
it's very good style and makes code clear but it can be three times as slow as just writing it literally. It's inescapable for functions that can have side effects but it's actually almost trivial to define some functions that just should literally be replaced with lines of code every time they appear, translate the source code into that and only then compile. Same I think for magic numbers, it doesn't take a lot of time to read from memory but if they're not supposed to be changed then why not just replace every instance of 'magic' with a literal before the code compiles?
Function call overheads are not big; you won't normally notice them. You only see them in this case because your actual code (x*x) is itself so completely trivial. In any real program that does real work, the amount of time spent in function-calling overhead will be negligably small.
(Not that I'd really recommend using foo, identity and square in the example, in any case; they're so trivial it's just as readable to have them inline and they don't really encapsulate or abstract anything.)
if they're not supposed to be changed then why not just replace every instance of 'magic' with a literal before the code compiles?
Because programs are written for to be easy for you to read and maintain. You could replace constants with their literal values, but it'd make the program harder to work with, for a benefit that is so tiny you'll probably never even be able to measure it: the height of premature optimisation.
Encapsulation is about one thing and one thing only: Readability. If you're really so worried about performance that you're willing to start stripping out encapsulated logic, you may as well just start coding in assembly.
Encapsulation also assists in debugging and feature adding. Consider the following: lets say you have a simple game and need to add code that depletes the players health under some circumstances. Easy, yes?
def DamagePlayer(dmg):
player.health -= dmg;
This IS very trivial code, so it's very tempting to simply scatter "player.health -=" everywhere. But what if later you want to add a powerup that halves damage done to the player while active? If the logic is still encapsulated, it's easy:
def DamagePlayer(dmg):
if player.hasCoolPowerUp:
player.health -= dmg / 2
else
player.health -= dmg
Now, consider if you had neglected to encapulate that bit of logic because of it's simplicity. Now you're looking at coding the same logic into 50 different places, at least one of which you are almost certain to forget, which leads to weird bugs like: "When player has powerup all damage is halved except when hit by AlienSheep enemies..."
Do you want to have problems with Alien Sheep? I don't think so. :)
In all seriousness, the point I'm trying to make is that encapsulation is a very good thing in the right circumstances. Of course, it's also easy to over-encapsulate, which can be just as problematic. Also, there are situations where the speed really truly does matter (though they are rare), and that extra few clock cycles is worth it. About the only way to find the right balance is practice. Don't shun encapsulation because it's slower, though. The benefits usually far outweigh the costs.
I don't know how good python compilers are, but the answer to this question for many languages is that the compiler will optimize calls to small procedures / functions / methods by inlining them. In fact, in some language implementations you generally get better performance by NOT trying to "micro-optimize" the code yourself.
What you are talking about is the effect of inlining functions for gaining efficiency.
It is certainly true in your Python example, that encapsulation hurts performance. But there are some counter example to it:
In Java, defining getter&setter instead of defining public member variables does not result in performance degradation as the JIT inline the getters&setters.
sometimes calling a function repetedly may be better than performing inlining as the code executed may then fit in the cache. Inlining may cause code explosion...
Figuring out what to make into a function and what to just include inline is something of an art. Many factors (performance, readability, maintainability) feed into the equation.
I actually find your example kind of silly in many ways - a function that just returns it's argument? Unless it's an overload that's changing the rules, it's stupid. A function to square things? Again, why bother. Your function 'foo' should probably return a string, so that it can be used directly:
''.join(foo(x)," -- bar !! "])
That's probably a more correct level of encapsulation in this example.
As I say, it really depends on the circumstances. Unfortunately, this is the sort of thing that doesn't lend itself well to examples.
IMO, this is related to Function Call Costs. Which are negligible usually, but not zero. Splitting the code in a lot of very small functions may hurt. Especially in interpreted languages where full optimization is not available.
Inlining functions may improve performance but it may also deteriorate. See, for example, C++ FQA Lite for explanations (“Inlining can make code faster by eliminating function call overhead, or slower by generating too much code, causing instruction cache misses”). This is not C++ specific. Better leave optimizations for compiler/interpreter unless they are really necessary.
By the way, I don't see a huge difference between two versions:
$ python bench.py
fine-grained function decomposition: 5.46632194519
one-liner: 4.46827578545
$ python --version
Python 2.5.2
I think this result is acceptable. See bench.py in the pastebin.
There is a performance hit for using functions, since there is the overhead of jumping to a new address, pushing registers on the stack, and returning at the end. This overhead is very small, however, and even in peformance critial systems worrying about such overhead is most likely premature optimization.
Many languages avoid these issues in small frequenty called functions by using inlining, which is essentially what you do above.
Python does not do inlining. The closest thing you can do is use macros to replace the function calls.
This sort of performance issue is better served by another language, if you need the sort of speed gained by inlining (mostly marginal, and sometimes detremental) then you need to consider not using python for whatever you are working on.
There is a good technical reason why what you suggested is impossible. In Python, functions, constants, and everything else are accessible at runtime and can be changed at any time if necessary; they could also be changed by outside modules. This is an explicit promise of Python and one would need some extremely important reasons to break it.
For example, here's the common logging idiom:
# beginning of the file xxx.py
log = lambda *x: None
def something():
...
log(...)
...
(here log does nothing), and then at some other module or at the interactive prompt:
import xxx
xxx.log = print
xxx.something()
As you see, here log is modified by completely different module --- or by the user --- so that logging now works. That would be impossible if log was optimized away.
Similarly, if an exception was to happen in make_legal_foo_string (this is possible, e.g. if x.__str__() is broken and returns None) you'd be hit with a source quote from a wrong line and even perhaps from a wrong file in your scenario.
There are some tools that in fact apply some optimizations to Python code, but I don't think of the kind you suggested.